Updated Docs

SPEC.md

@@ -14,6 +14,10 @@ The project will be composed of the following files:
 - **`README.md`**: A documentation file providing an overview of the project, setup instructions, and usage examples.
 - **`.gitignore`**: A file to specify which files and directories should be ignored by Git.
 - **`PROGRESS.md`**: A file to track the development progress of the project.
+- **`data_storage.py`**: Handles loading, storing, and calculating baselines from historical data.
+- **`CONSTRAINTS.md`**: Defines constraints and guidelines for the LLM's analysis.
+- **`known_issues.json`**: A JSON file containing a list of known issues to be considered by the LLM.
+- **`AGENTS.md`**: Documents the human and autonomous agents involved in the project.
 
 ## 3. Functional Requirements
 
@@ -26,10 +30,12 @@ The project will be composed of the following files:
     - `HOME_ASSISTANT_TOKEN`
     - `GOOGLE_HOME_SPEAKER_ID`
     - `DAILY_RECAP_TIME`
+    - `NMAP_TARGETS`
+    - `NMAP_SCAN_OPTIONS`
 
 ### 3.2. Data Ingestion and Parsing
 
-- The agent must be able to collect and parse system logs.
+- The agent must be able to collect and parse system logs (syslog and auth.log).
 - The agent must be able to collect and parse network metrics.
 - The parsing of this data should result in a structured format (JSON or Python dictionary).
 
@@ -38,24 +44,25 @@ The project will be composed of the following files:
 - **CPU Temperature**: The agent will monitor the CPU temperature.
 - **GPU Temperature**: The agent will monitor the GPU temperature.
 - **System Login Attempts**: The agent will monitor system login attempts.
+- **Network Scan Results (Nmap)**: The agent will periodically perform Nmap scans to discover hosts and open ports, logging detailed information including IP addresses, host status, and open ports with service details.
 
-### 3.3. LLM Analysis
+### 3.4. LLM Analysis
 
 - The agent must use a local LLM (via Ollama) to analyze the collected data.
-- The agent must construct a specific prompt to guide the LLM in identifying anomalies.
-- The LLM's response will be either "OK" (no anomaly) or a natural language paragraph describing the anomaly, including a severity level (high, medium, low).
+- The agent must construct a specific prompt to guide the LLM in identifying anomalies, incorporating historical baselines and known issues.
+- The LLM's response will be a structured JSON object with `severity` (high, medium, low, none) and `reason` fields.
 
-### 3.4. Alerting
+### 3.5. Alerting
 
 - The agent must be able to send alerts to a Discord webhook.
 - The agent must be able to trigger a text-to-speech (TTS) alert on a Google Home speaker via Home Assistant.
 
-### 3.5. Alerting Logic
+### 3.6. Alerting Logic
 
 - Immediate alerts (Discord and Home Assistant) will only be sent for "high" severity anomalies.
 - A daily recap of all anomalies (high, medium, and low) will be sent at a configurable time.
 
-### 3.6. Main Loop
+### 3.7. Main Loop
 
 - The agent will run in a continuous loop.
 - The loop will execute the data collection, analysis, and alerting steps periodically.
@@ -64,26 +71,33 @@ The project will be composed of the following files:
 ## 4. Data Storage and Baselining
 
 - **4.1. Data Storage**: The agent will store historical monitoring data in a JSON file (`monitoring_data.json`).
-- **4.2. Baselining**: The agent will calculate baseline averages for key metrics (e.g., RTT, packet loss) from the stored historical data. This baseline will be used by the LLM to improve anomaly detection accuracy.
+- **4.2. Baselining**: The agent will calculate baseline averages for key metrics (e.g., RTT, packet loss, temperatures, open ports) from the stored historical data. This baseline will be used by the LLM to improve anomaly detection accuracy.
 
 ## 5. Technical Requirements
 
 - **Language**: Python 3.8+
 - **LLM**: `llama3.1:8b` running on a local Ollama instance.
+- **Prerequisites**: `nmap`, `lm-sensors`
 - **Libraries**:
     - `ollama`
     - `discord-webhook`
     - `requests`
     - `syslog-rfc5424-parser`
-    - `apachelogs`
-    - `jc`
+    - `pingparsing`
+    - `python-nmap`
 
 ## 6. Project Structure
 
 ```
 /
 ├── .gitignore
+├── AGENTS.md
 ├── config.py
+├── CONSTRAINTS.md
+├── data_storage.py
+├── known_issues.json
+├── log_position.txt
+├── auth_log_position.txt
 ├── monitor_agent.py
 ├── PROMPT.md
 ├── README.md
@@ -93,4 +107,4 @@ The project will be composed of the following files:
 ```
 
 ## 7. Testing and Debugging
-The script is equipped with a test mode, that only runs the script once, and not continuously. To enable, change the `TEST_MODE` variable in `config.py` to `True`. Once finished testing, change the variable back to `False`.
+The script is equipped with a test mode, that only runs the script once, and not continuously. To enable, change the `TEST_MODE` variable in `config.py` to `True`. Once finished testing, change the variable back to `False`.