feat: Update baseline calculations and LLM prompts

- Change baseline calculations to use integers instead of floats to simplify data.
- Update LLM constraints and prompt for more accurate anomaly detection.
- Refine known_issues to reduce false positives.
- Update PROGRESS.md with new TODO items.
This commit is contained in:
2025-08-21 12:12:15 -05:00
parent c5a446ea65
commit e119bc7194
5 changed files with 16 additions and 5 deletions

View File

@@ -1,6 +1,8 @@
## LLM Constraints and Guidelines ## LLM Constraints and Guidelines
- Not everything is an anamoly. Err on the side of caution when selecting severity. Its ok not to report anything. You don't have to say anything if you don't want to, or don't need to.
- Please do not report on anything that is older then 24 hours. - Please do not report on anything that is older then 24 hours.
- The server uses a custom DNS server at 192.168.2.112. - The server uses a custom DNS server at 192.168.2.112.
- Please think carefully on if the measured values exceed the averages by any significant margin. A few seconds, or a few degrees in difference do not mean a significant margin. Only report anomolies with delta values greater then 10.
### Important Things to Focus On: ### Important Things to Focus On:
- Security-related events such as failed login attempts, unauthorized access, or unusual network connections. - Security-related events such as failed login attempts, unauthorized access, or unusual network connections.

View File

@@ -60,10 +60,18 @@
36. [x] Create helper function in `data_storage.py` for calculating average metrics. 36. [x] Create helper function in `data_storage.py` for calculating average metrics.
37. [x] Update `README.md` with current project status and improvements. 37. [x] Update `README.md` with current project status and improvements.
38. [x] Create `AGENTS.md` to document human and autonomous agents. 38. [x] Create `AGENTS.md` to document human and autonomous agents.
## Keeping track of Current Objectives
[x] Improve "high" priority detection by explicitly instructing LLM to output severity in structured JSON format. [x] Improve "high" priority detection by explicitly instructing LLM to output severity in structured JSON format.
[x] Implement dynamic contextual information (Known/Resolved Issues Feed) for LLM to improve severity detection. [x] Implement dynamic contextual information (Known/Resolved Issues Feed) for LLM to improve severity detection.
## TODO ## TODO
- [ ] Change baseline calculations to only use integers instead of floats.
- [ ] Add a log file that only keeps records for the past 24 hours.
- [ ] Log all LLM responses to the console.
- [ ] Reduce alerts to only happen between 9am and 12am.
- [ ] Get hostnames of devices in Nmap scan.
- [ ] Filter out RTT fluctuations below 10 seconds.
- [ ] Filter out temperature fluctuations with differences less than 5 degrees.

View File

@@ -19,7 +19,7 @@ def store_data(new_data):
def _calculate_average(data, key1, key2): def _calculate_average(data, key1, key2):
"""Helper function to calculate the average of a nested key in a list of dicts.""" """Helper function to calculate the average of a nested key in a list of dicts."""
values = [d[key1][key2] for d in data if key1 in d and key2 in d[key1] and d[key1][key2] != "N/A"] values = [d[key1][key2] for d in data if key1 in d and key2 in d[key1] and d[key1][key2] != "N/A"]
return sum(values) / len(values) if values else 0 return int(sum(values) / len(values)) if values else 0
def calculate_baselines(): def calculate_baselines():
data = load_data() data = load_data()

View File

@@ -13,7 +13,7 @@
}, },
{ {
"issue": "Port 62078 is open", "issue": "Port 62078 is open",
"resolution": "Port 62078 is used in apple devices for syncing communcation between each other. This is not an amomaly, this is expected and normal behavior used by Apple Devices to communicate." "resolution": "This is normal behavior for Apple devices. Do not report."
}, },
{ {
"issue": "RTT averages are higher then average", "issue": "RTT averages are higher then average",

View File

@@ -195,7 +195,8 @@ def build_llm_prompt(data, baselines, nmap_changes, constraints, known_issues):
**Constraints and Guidelines:** **Constraints and Guidelines:**
{constraints} {constraints}
**Output Request:** If you find an anomaly, provide a report as a single JSON object with two keys: "severity" and "reason". The "severity" must be one of "high", "medium", "low", or "none". The "reason" must be a natural language explanation of the anomaly. Please include specific values if the anomoly has them. If no anomaly is found, return a single JSON object with "severity" set to "none" and "reason" as an empty string. Do not wrap the JSON in markdown or any other formatting. **Output Request:** If you find an anomaly, provide a report as a single JSON object with two keys: "severity" and "reason". The "severity" must be one of "high", "medium", "low", or "none". The "reason" must be a natural language explanation of the anomaly. Please include specific values if the anomoly has them. If no anomaly is found, return a single JSON object with "severity" set to "none" and "reason" as an empty string. Do not wrap the JSON in markdown or any other formatting. Only return the JSON, and nothing else.
**Reasoning Hint:** Think step by step to come to your conclusion. This is very important. **Reasoning Hint:** Think step by step to come to your conclusion. This is very important.
""" """