Inanis_Vault/20-Knowledge/Personal Projects/Blight/AGENTS.md at main

Files

Spencer Grimes 7d373a9b07 vault backup: 2025-11-12 15:17:57 from

2025-11-12 15:17:57 +09:00

20 KiB

Raw Permalink Blame History

Project Overview

Building a centralized AI-powered hub that connects personal devices, processes information, and makes intelligent decisions using Google Gemini API. The system manages markdown notes (Obsidian vault), sends reminders, and executes automations across 20-30 personal devices.

Core Requirements

Scale: Personal use, 20-30 devices maximum
Deployment: Docker Compose on Linux server
AI: Google Gemini API with caching and rate limiting
Notes: Obsidian vault synced via Git (read-write access)
Network: All devices connected via VPN (bidirectional communication)
Notifications: Primary channel is Discord webhooks
Integrations: Home Assistant (optional middleware)
Storage: 7-day data retention for events/logs

Technology Stack

Hub (Docker Compose Services)

API: FastAPI (Python)
Database: PostgreSQL 16
Vector DB: ChromaDB (note embeddings)
Cache/Queue: Redis 7
Worker: Background task processor

Device Agents

Language: Python (cross-platform)
Deployment:
- Linux: systemd service
- Windows: Task Scheduler / Windows Service
- Mobile: Termux (Android) / Shortcuts (iOS)

External Services

AI: Google Gemini API
Notifications: Discord webhooks
Home Automation: Home Assistant REST API
Code Repository: Gitea (local, for agent updates)
Version Control: Git (Obsidian vault sync)

Architecture Decisions

Note Management

Hub has read-write access to Obsidian vault
Git workflow: pull → process → commit → push
Reminder lines deleted after processing (clean removal)
Git conflicts trigger Discord alerts for manual resolution

Device Communication

Devices authenticate with API keys
Agents report events and queue locally if hub offline
Hub can send commands to devices (predefined command set)
Agent state is authoritative over hub's cached state

AI Integration (Gemini)

24-hour cache for similar queries (Redis)
Rate limit: 100 requests/hour
Circuit breaker: 5 failures = 15min pause
Fallback: Simple regex parsing if Gemini unavailable

Data Management

Events/logs: 7-day retention, auto-cleanup
Completed reminders: Delete after processing
Agent timezones: Translate to hub timezone
Incremental note indexing using git diff

Automation Rules

YAML configuration files (version controlled)
Home Assistant handles repeatable triggers
Hub handles one-off events and AI decisions

Project Structure

personal-ai-hub/
├── AGENTS.md                    # This file - update as you progress
├── README.md                    # User-facing documentation
├── docker-compose.yml           # Service orchestration
├── .env.example                 # Environment variables template
├── .gitignore
│
├── hub/
│   ├── Dockerfile
│   ├── requirements.txt
│   ├── main.py                  # FastAPI application entry point
│   ├── worker.py                # Background task processor
│   ├── models.py                # Database models (SQLAlchemy)
│   ├── schemas.py               # Pydantic schemas for API
│   ├── config.yaml.example      # Automation rules template
│   ├── alembic/                 # Database migrations
│   │   ├── alembic.ini
│   │   └── versions/
│   ├── api/
│   │   ├── __init__.py
│   │   ├── devices.py           # Device registration endpoints
│   │   ├── events.py            # Event submission/retrieval
│   │   ├── reminders.py         # Reminder management
│   │   ├── webhooks.py          # Webhook endpoints
│   │   ├── health.py            # Health check endpoint
│   │   └── admin.py             # Admin/management endpoints
│   ├── services/
│   │   ├── __init__.py
│   │   ├── gemini.py            # Gemini API client
│   │   ├── notes.py             # Markdown parser & git operations
│   │   ├── reminders.py         # Reminder scheduling logic
│   │   ├── webhooks.py          # Discord/webhook sender
│   │   ├── home_assistant.py    # HA integration
│   │   ├── automation.py        # YAML rule engine
│   │   ├── vector_store.py      # ChromaDB interface
│   │   └── cache.py             # Redis caching layer
│   ├── utils/
│   │   ├── __init__.py
│   │   ├── auth.py              # API key authentication
│   │   ├── logging.py           # Structured logging
│   │   └── timezone.py          # Timezone conversion
│   └── tests/
│       ├── test_api.py
│       ├── test_notes.py
│       ├── test_reminders.py
│       └── test_automation.py
│
├── agent/
│   ├── agent.py                 # Main agent script
│   ├── requirements.txt
│   ├── config.example.json      # Agent configuration template
│   ├── version.txt              # Current agent version
│   ├── collectors/
│   │   ├── __init__.py
│   │   ├── system_metrics.py   # CPU, memory, disk
│   │   ├── application.py      # Running apps, active window
│   │   └── custom.py           # User-defined collectors
│   ├── executors/
│   │   ├── __init__.py
│   │   └── commands.py         # Command execution handlers
│   ├── install/
│   │   ├── install.sh          # Linux installation script
│   │   ├── install.ps1         # Windows installation script
│   │   ├── systemd/
│   │   │   └── hub-agent.service
│   │   └── windows/
│   │       └── task-scheduler.xml
│   └── tests/
│       └── test_agent.py
│
├── docs/
│   ├── setup.md                 # Initial setup guide
│   ├── api.md                   # API documentation
│   ├── automation-guide.md      # Writing automation rules
│   ├── agent-installation.md    # Device agent setup
│   └── troubleshooting.md       # Common issues
│
├── scripts/
│   ├── backup.sh                # Database backup script
│   ├── cleanup.sh               # Manual data cleanup
│   └── init-vault.sh            # Initialize test vault
│
└── config/
    ├── automation-rules.yaml    # Default automation rules
    └── device-whitelist.yaml    # Optional device restrictions

Development Phases

Phase 1: Foundation ⬜ NOT STARTED

Goal: Basic infrastructure running with device registration and health monitoring

Tasks:

Create docker-compose.yml with all services (postgres, redis, chromadb, hub-api, worker)
Create .env.example with all required environment variables
Set up FastAPI application structure in hub/main.py
Define database models in hub/models.py (Device, Event, Reminder tables)
Create Alembic migration for initial schema
Implement /health endpoint showing system status
Implement device registration endpoint (POST /devices/register)
Implement API key authentication middleware
Set up structured JSON logging
Create basic README.md with quickstart instructions

Acceptance Criteria:

docker-compose up successfully starts all services
Can register a device and receive API key
/health endpoint returns status of all components
Logs are structured and readable
Database migrations apply cleanly

Files to Create:

docker-compose.yml
.env.example
hub/main.py
hub/models.py
hub/api/devices.py
hub/api/health.py
hub/utils/auth.py
hub/utils/logging.py
hub/alembic/versions/001_initial_schema.py
README.md

Progress: 0/10 tasks complete

Phase 2: Device Agent ⬜ NOT STARTED

Goal: Cross-platform agent that reports to hub and executes commands

Tasks:

Create agent/agent.py main script
Implement heartbeat mechanism (report every 5 minutes)
Implement system metrics collection (CPU, memory, disk)
Implement event queue for offline operation
Create device command execution framework
Add auto-update check on startup (version endpoint)
Create Linux systemd service file
Create Windows Task Scheduler XML
Write install.sh for Linux
Write install.ps1 for PowerShell
Document agent configuration format

Acceptance Criteria:

Agent successfully registers with hub on first run
Agent sends heartbeat every 5 minutes
Agent queues events when hub is unreachable
Agent can execute basic commands from hub
Agent installs as service on Linux
Agent installs as scheduled task on Windows
Agent checks for updates on startup

Files to Create:

agent/agent.py
agent/config.example.json
agent/collectors/system_metrics.py
agent/executors/commands.py
agent/install/install.sh
agent/install/install.ps1
agent/install/systemd/hub-agent.service
docs/agent-installation.md
hub/api/devices.py (add version endpoint)

Progress: 0/11 tasks complete

Phase 3: Notes & Reminders ⬜ NOT STARTED

Goal: Parse markdown notes, extract reminders, send Discord notifications

Tasks:

Implement git operations in services/notes.py (pull, commit, push)
Create markdown parser for reminder syntax
Implement reminder extraction with date parsing
Set up ChromaDB for note embeddings
Create note indexing worker (incremental via git diff)
Implement reminder scheduler (checks every minute)
Create Discord webhook sender
Implement reminder deletion from markdown files
Add git conflict detection and Discord alerts
Create reminder management API endpoints
Add error notification for malformed reminder syntax
Write tests for reminder parsing

Reminder Syntax to Support:

@remind 2024-11-15 Review proposal
@remind in 3 days Check on project
@remind daily at 09:00 Stand-up meeting

Acceptance Criteria:

Hub can read notes from mounted Obsidian vault
Hub detects new/modified notes via git diff
Reminders are correctly parsed from markdown
Scheduled reminders trigger at correct time
Discord webhook delivers notification
Processed reminder line is deleted from note
Git commits and pushes changes successfully
Git conflicts are detected and alerted

Files to Create:

hub/services/notes.py
hub/services/reminders.py
hub/services/webhooks.py
hub/services/vector_store.py
hub/api/reminders.py
hub/worker.py (reminder scheduler)
hub/models.py (add Reminder table)
hub/tests/test_notes.py
hub/tests/test_reminders.py
docs/automation-guide.md (reminder syntax section)

Progress: 0/12 tasks complete

Phase 4: AI Integration ⬜ NOT STARTED

Goal: Gemini API integration with caching, rate limiting, and semantic search

Tasks:

Implement Gemini API client in services/gemini.py
Set up Redis caching layer (24hr TTL)
Implement rate limiting (100 req/hour)
Implement circuit breaker (5 failures = 15min pause)
Create fallback regex-based reminder parser
Implement note embedding generation
Create semantic search over notes using ChromaDB
Add cost tracking (log token usage)
Enhance reminder parsing with natural language support
Create Gemini health check for monitoring
Add Gemini context builder (device states + notes)
Write tests for Gemini integration

Acceptance Criteria:

Gemini API successfully processes queries
Responses are cached and reused appropriately
Rate limiting prevents quota exhaustion
Circuit breaker triggers after repeated failures
System falls back to regex parsing when Gemini down
Natural language dates parsed correctly ("next Tuesday")
Semantic search returns relevant notes
Token usage is logged for cost monitoring

Files to Create:

hub/services/gemini.py
hub/services/cache.py
hub/utils/circuit_breaker.py
hub/tests/test_gemini.py
Update hub/services/notes.py (add embeddings)
Update hub/services/reminders.py (add NLP parsing)

Progress: 0/12 tasks complete

Phase 5: Automation Engine ⬜ NOT STARTED

Goal: YAML-based rules that trigger actions based on device events

Tasks:

Create YAML rule schema definition
Implement rule parser in services/automation.py
Create rule evaluation engine
Implement trigger matching (device, event, time)
Implement condition evaluation
Implement action execution (webhook, command, search)
Add Home Assistant REST API client
Create bidirectional HA webhook integration
Add dry-run mode for testing rules
Create rule management API endpoints
Write automation guide documentation
Write tests for automation engine

Example Rule Format:

rules:
  - name: "Evening work reminder"
    trigger:
      device: "laptop"
      event: "work_apps_closed"
      time_after: "17:00"
    action:
      type: "search_notes"
      query: "today's todos"
      notify: discord
    enabled: true

Acceptance Criteria:

YAML rules load correctly from config file
Rules trigger based on device events
Time-based conditions work correctly
Actions execute successfully (Discord, HA, device commands)
Home Assistant can trigger hub via webhook
Hub can trigger HA automations via REST
Dry-run mode shows what would happen without executing
Invalid rules are caught with helpful errors

Files to Create:

hub/services/automation.py
hub/services/home_assistant.py
hub/api/automations.py
config/automation-rules.yaml
docs/automation-guide.md
hub/tests/test_automation.py

Progress: 0/12 tasks complete

Phase 6: Agent Auto-Update ⬜ NOT STARTED

Goal: Agents automatically update from Gitea repository

Tasks:

Create agent version endpoint in hub API
Implement version checking in agent
Create agent download endpoint (proxy to Gitea)
Implement agent self-update logic (download, backup, replace, restart)
Add rollback mechanism (keep last working version)
Create agent release workflow documentation
Test update on Linux
Test update on Windows
Add update notification to Discord
Create version tracking in hub database

Acceptance Criteria:

Agent checks version on startup
Agent downloads new version when available
Agent backs up current version before updating
Agent restarts after successful update
Agent can rollback to previous version on failure
Update process works on both Linux and Windows
Hub tracks which agents are on which versions

Files to Create:

hub/api/admin.py (version endpoints)
Update agent/agent.py (add update logic)
scripts/release-agent.sh (helper for releases)
docs/agent-installation.md (update section)

Progress: 0/10 tasks complete

Phase 7: Monitoring & Polish ⬜ NOT STARTED

Goal: Observability, documentation, and production readiness

Tasks:

Create simple web dashboard (device grid, event stream)
Enhance /health endpoint with detailed metrics
Implement data retention cleanup job (7 days)
Add database backup script
Create troubleshooting documentation
Write comprehensive API documentation
Add Prometheus metrics endpoint (optional)
Set up critical alerts (hub down, device offline >1hr)
Create testing guide
Final end-to-end testing across all components
Performance testing with 30 devices
Security audit (API keys, git credentials, etc.)

Acceptance Criteria:

Dashboard shows real-time system status
Old data automatically cleaned up after 7 days
Database backups run automatically
All documentation is complete and accurate
Critical alerts deliver to Discord
System handles 30 concurrent device connections
No security vulnerabilities in authentication/authorization

Files to Create:

hub/api/dashboard.py (simple UI endpoints)
scripts/backup.sh
scripts/cleanup.sh
docs/troubleshooting.md
docs/api.md
docs/testing.md
Update README.md (comprehensive)

Progress: 0/12 tasks complete

Current Status

Overall Progress: 0% (0/7 phases complete)

Current Phase: Phase 1 - Foundation

Blockers: None

Notes:

Project planning complete, ready to begin implementation
All architecture decisions finalized
Development environment ready (Linux server, Docker, Gitea)

Instructions for Agents

How to Use This File

Start with Phase 1 and work sequentially through phases
Check off tasks as you complete them using [x]
Update progress counters (e.g., "3/10 tasks complete")
Mark phases as complete when all tasks done: ⬜ → 🟡 → ✅
- ⬜ NOT STARTED
- 🟡 IN PROGRESS
- ✅ COMPLETE
Update "Current Status" section with your progress
Add notes in "Blockers" or "Notes" if you encounter issues
Commit changes to this file after each work session

Phase Status Icons

Use these when updating phase headers:

⬜ NOT STARTED - No work begun on this phase
🟡 IN PROGRESS - At least one task started
✅ COMPLETE - All tasks finished and acceptance criteria met

Before Starting a Phase

Read through all tasks and acceptance criteria
Review the files to create
Check dependencies on previous phases
Update phase status to 🟡 IN PROGRESS

When Completing a Task

Mark the task checkbox: - [x]
Update the progress counter
Commit the code changes
Update this file

When Completing a Phase

Verify all acceptance criteria are met
Mark phase as ✅ COMPLETE
Update overall progress percentage
Move to next phase
Add any lessons learned in Notes section

Git Commit Messages

Use conventional commits format:

feat(phase1): implement device registration endpoint
fix(phase3): correct reminder date parsing
docs: update AGENTS.md progress
test(phase2): add agent heartbeat tests

Testing Requirements

Write unit tests for core logic
Write integration tests for API endpoints
Test happy path and error cases
Run tests before marking phase complete

Documentation Requirements

Update relevant docs when adding features
Include code examples in documentation
Keep API docs in sync with implementation
Document configuration options

Environment Setup

Required Environment Variables

# Gemini API
GEMINI_API_KEY=your_key_here

# Database
DATABASE_URL=postgresql://user:pass@db:5432/hub
REDIS_URL=redis://redis:6379

# Git Configuration (for notes)
GIT_USER_NAME="Personal AI Hub"
GIT_USER_EMAIL="hub@yourdomain.local"
GIT_REMOTE_URL=https://gitea.local/user/obsidian-vault.git

# Home Assistant
HOME_ASSISTANT_URL=http://homeassistant.local:8123
HOME_ASSISTANT_TOKEN=your_ha_token

# Discord
DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...

# Agent Updates
GITEA_URL=http://gitea.local
GITEA_TOKEN=your_gitea_token

# Security
API_SECRET_KEY=generate_random_key_here

# Paths
NOTES_PATH=/app/notes
NOTES_ALLOWED_FOLDERS=reminders,tasks,projects

Development Tools Needed

Docker & Docker Compose
Python 3.11+
Git
Text editor / IDE
curl or Postman (API testing)
Access to: Gemini API, Discord webhook, Gitea instance

Helpful Resources

API Clients

FastAPI Docs: https://fastapi.tiangolo.com/
Gemini API: https://ai.google.dev/docs
Discord Webhooks: https://discord.com/developers/docs/resources/webhook

Libraries

SQLAlchemy: https://docs.sqlalchemy.org/
ChromaDB: https://docs.trychroma.com/
Redis-py: https://redis-py.readthedocs.io/
GitPython: https://gitpython.readthedocs.io/

Patterns

Structured Logging: Use JSON format with correlation IDs
Error Handling: Always log, alert on critical errors
API Design: RESTful, versioned endpoints
Testing: Pytest with fixtures for database/API

Contact & Support

If you encounter issues or need clarification:

Check troubleshooting doc
Ask for help

20 KiB Raw Permalink Blame History

Project Overview

Core Requirements

Technology Stack

Hub (Docker Compose Services)

Device Agents

External Services

Architecture Decisions

Note Management

Device Communication

AI Integration (Gemini)

Data Management

Automation Rules

Project Structure

Development Phases

Phase 1: Foundation ⬜ NOT STARTED

Phase 2: Device Agent ⬜ NOT STARTED

Phase 3: Notes & Reminders ⬜ NOT STARTED

Phase 4: AI Integration ⬜ NOT STARTED

Phase 5: Automation Engine ⬜ NOT STARTED

Phase 6: Agent Auto-Update ⬜ NOT STARTED

Phase 7: Monitoring & Polish ⬜ NOT STARTED

Current Status

Instructions for Agents

How to Use This File

Phase Status Icons

Before Starting a Phase

When Completing a Task

When Completing a Phase

Git Commit Messages

Testing Requirements

Documentation Requirements

Environment Setup

Required Environment Variables

Development Tools Needed

Helpful Resources

API Clients

Libraries

Patterns

Contact & Support

20 KiB

Raw Permalink Blame History