feat: scalable entities & patterns architecture

- Entity format: 3-line compact registry (### Name / key facts / status+links)
  Heavy entries promoted to thread files. Cross-domain pointers for shared entities.
- Pattern satellites: core patterns.md cap reduced from 110→70 lines (5.5KB).
  Domain-specific patterns go in satellite files loaded only by owning skill.
- Reflect: pattern routing rules, entity format enforcement (step 3b)
- Housekeeping: entity registry enforcement (step 5b), glacier inactive entities
- Evolve: scorecard metrics for pattern distribution + entity compression ratio
- README: updated entity example to 3-line format

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Marcio Puga
2026-03-23 07:03:34 +11:00
parent b2ff95cc9d
commit d47510b0f6
6 changed files with 45 additions and 15 deletions

View File

@@ -20,6 +20,7 @@ Architecture reference:
Measure (don't edit content): Measure (don't edit content):
- `memory/hot-memory.md` - `memory/hot-memory.md`
- `memory/cog-meta/patterns.md` - `memory/cog-meta/patterns.md`
- Any domain satellite pattern files (e.g. `work/*/patterns.md`)
## Process ## Process
@@ -40,11 +41,19 @@ Review the output of recent housekeeping and reflect runs:
- Did pruning priority order work? Or did it trim wrong things? - Did pruning priority order work? Or did it trim wrong things?
- Are glacier thresholds (50 obs, 10 action items) right? - Are glacier thresholds (50 obs, 10 action items) right?
- Is the 50-line hot-memory cap appropriate? - Is the 50-line hot-memory cap appropriate?
- Is entity format enforcement catching violations?
**Reflect rules check:** **Reflect rules check:**
- Did condensation produce useful patterns, or noise? - Did condensation produce useful patterns, or noise?
- Did thread candidate detection work? - Did thread candidate detection work?
- Is reflect staying in its lane? - Is reflect staying in its lane?
- Are patterns routing to the right file (core vs satellite)?
**Scorecard metrics** — measure and record in evolve-log:
- Core `patterns.md`: line count / 70, byte size / 5.5KB (target: ≤1.0)
- Satellite pattern files: list each with line count (soft cap: 30)
- Entity compression ratio: `(total entity lines across all files) / (total ### entries)` (target: ≤3.0)
- Hot-memory line counts vs caps
### 3. Rule Change Proposals ### 3. Rule Change Proposals

View File

@@ -64,7 +64,15 @@ For each non-glacier memory file:
Only add links where the reference is substantive. Only add links where the reference is substantive.
## 5b. Temporal Fact Maintenance ## 5b. Entity Registry Format Enforcement
Scan all `entities.md` files for registry format compliance:
1. **3-line max**: Any `### entry` with >3 content lines should be compressed. If the entry has an associated detail file (`→ [[link]]`), compress to: name/relationship, pipe-separated key facts, status+link. If no detail file exists and entry is >5 lines, flag as a promotion candidate (suggest creating a thread file).
2. **Glacier candidates**: Entries with `status: inactive` or `last:` date >6 months ago → move to `glacier/{domain}/entities-inactive.md` (leave a stub with archived comment).
3. **Missing metadata**: Flag entries missing `status:` or `last:` fields.
## 5c. Temporal Fact Maintenance
Scan all `entities.md` files for `(until YYYY-MM)` markers with past dates: Scan all `entities.md` files for `(until YYYY-MM)` markers with past dates:
1. If the line has no ~~strikethrough~~, add it 1. If the line has no ~~strikethrough~~, add it

View File

@@ -74,18 +74,31 @@ Check if findings are already captured:
- Distill into a pattern and add/update in `memory/cog-meta/patterns.md` (or domain `patterns.md` if domain-specific) - Distill into a pattern and add/update in `memory/cog-meta/patterns.md` (or domain `patterns.md` if domain-specific)
- Don't delete the observations — they stay as the raw record - Don't delete the observations — they stay as the raw record
**patterns.md size cap — HARD LIMIT: 110 lines / 7KB.** After any updates, check the file size. If it exceeds the cap: **Core patterns.md size cap — HARD LIMIT: 70 lines / 5.5KB.** After any updates, check the file size. If it exceeds the cap:
1. Compress multi-line entries to single lines 1. Compress multi-line entries to single lines
2. Merge entries with overlapping lessons 2. Merge entries with overlapping lessons
3. Remove point-in-time data: counts with date ranges, incident tallies with specific dates 3. Remove point-in-time data: counts with date ranges, incident tallies with specific dates
4. Entries must be **timeless rules** — "what to do" not "what happened" 4. Entries must be **timeless rules** — "what to do" not "what happened"
5. Move domain-specific patterns to satellite files (e.g. `work/acme/patterns.md`) — only universal rules stay in core
**Pattern routing** — when adding a new pattern, decide where it belongs:
- **Core** (`cog-meta/patterns.md`) — universal rules that apply every conversation
- **Domain satellite** (`{domain}/patterns.md`) — rules specific to one domain, loaded only by that skill
- Satellite files have a soft cap of 30 lines each
**Hot-memory relevance** — Review all `hot-memory.md` files: **Hot-memory relevance** — Review all `hot-memory.md` files:
- **Promote**: If a pattern is heating up → add to appropriate `hot-memory.md` - **Promote**: If a pattern is heating up → add to appropriate `hot-memory.md`
- **Demote**: If a hot-memory item has gone quiet (no references in 2+ weeks) → remove from hot-memory - **Demote**: If a hot-memory item has gone quiet (no references in 2+ weeks) → remove from hot-memory
- **Goal**: hot-memory = what matters *right now* - **Goal**: hot-memory = what matters *right now*
### 3b. Detect Thread Candidates ### 3b. Entity Registry Format Enforcement
Scan all `entities.md` files for format compliance:
1. **3-line check**: Any `### entry` with >3 content lines → compress. If the entry has a detail file (`→ [[link]]`), trim to: name line, key facts, status/link. If no detail file exists but entry is >5 lines, flag as a promotion candidate for a thread file.
2. **Status/last fields**: Every entry should have `status: active|inactive` and `last: YYYY-MM-DD`. Scan recent session transcripts to update `last:` dates for mentioned entities.
3. **Cross-domain pointers**: If the same person appears in multiple entity files, ensure one is canonical (full entry) and others are pointers (`see [[link]]`).
### 3c. Detect Thread Candidates
Scan observations for topics that appear across 3+ dates or span 2+ weeks. These are thread candidates. Scan observations for topics that appear across 3+ dates or span 2+ weeks. These are thread candidates.
@@ -94,7 +107,7 @@ For each candidate:
- If not, note it as a suggestion: "Thread candidate: [topic] — [N] fragments across [date range]" - If not, note it as a suggestion: "Thread candidate: [topic] — [N] fragments across [date range]"
- Don't auto-create threads — suggest them - Don't auto-create threads — suggest them
### 3c. Scenario Feedback Loop ### 3d. Scenario Feedback Loop
Scan `memory/cog-meta/scenarios/` for active scenario files. Scan `memory/cog-meta/scenarios/` for active scenario files.

View File

@@ -125,12 +125,10 @@ The `/reflect` skill reads recent session transcripts to review interactions, ca
3. **Observations are append-only**: `- YYYY-MM-DD [tags]: <observation>` — never edit past entries 3. **Observations are append-only**: `- YYYY-MM-DD [tags]: <observation>` — never edit past entries
- Tags: `health`, `habits`, `family`, `milestone`, `work`, `insight`, `regression`, `philosophy`, `mental-health` - Tags: `health`, `habits`, `family`, `milestone`, `work`, `insight`, `regression`, `philosophy`, `mental-health`
4. **Action items**: `- [ ] task (added YYYY-MM-DD)` / `- [x] task (done YYYY-MM-DD)` 4. **Action items**: `- [ ] task (added YYYY-MM-DD)` / `- [x] task (done YYYY-MM-DD)`
5. **Entities**: Edit in place, never delete. For time-bound facts, use inline temporal markers: 5. **Entities**: 3-line compact registry format. Each entry: `### Name (relationship)` / pipe-separated key facts / `status: active|inactive | last: YYYY-MM-DD | → [[link]]`. Max 3 content lines per entry. Heavy entries promoted to detail thread files — entity stub links to thread via `→ [[link]]`. Cross-domain entities: canonical entry in primary domain, pointer in secondary (e.g. `see [[work/acme/entities#Jane]]` in personal). **Temporal validity**: When a fact changes, mark old value as superseded with date. `last:` dates updated by reflect from journal/session scans. Entities inactive 6+ months → housekeeping moves to glacier.
- `(since YYYY-MM)` — when a fact became true (role, relationship, location)
- `(until YYYY-MM)` — when a fact stopped being true (keep the line, don't delete)
- Example: `- Role: Senior Engineer (since 2024-03)` or `- Was at Acme (until 2025-06)`
6. **Hot memory <50 lines**: Prune aggressively, move detail to observations 6. **Hot memory <50 lines**: Prune aggressively, move detail to observations
7. **Self-improvement**: After notable interactions, append to `cog-meta/self-observations.md`. Periodically distill patterns into `cog-meta/patterns.md` 7. **Self-improvement**: After notable interactions, append to `cog-meta/self-observations.md`. Periodically distill patterns into `cog-meta/patterns.md`
- **Pattern routing**: Core patterns (`cog-meta/patterns.md`, ≤70 lines / 5.5KB) are universal rules injected every turn. Domain-specific patterns go in satellite files (e.g. `work/acme/patterns.md`) loaded only by their owning skill. New patterns go to the appropriate satellite if domain-specific, or core if universal. Satellite files have a soft cap of 30 lines each.
8. **Single Source of Truth (SSOT)**: Each fact lives in ONE canonical file. Other files reference via `[[link]]`, never copy. 8. **Single Source of Truth (SSOT)**: Each fact lives in ONE canonical file. Other files reference via `[[link]]`, never copy.
- `entities.md` — people, organizations, named things - `entities.md` — people, organizations, named things
- `action-items.md` — all tasks - `action-items.md` — all tasks
@@ -175,7 +173,7 @@ When responding to any query:
| `hot-memory.md` | Rewrite freely | | `hot-memory.md` | Rewrite freely |
| `observations.md` | Append only | | `observations.md` | Append only |
| `action-items.md` | Append new, check off done | | `action-items.md` | Append new, check off done |
| `entities.md` | Edit in place by `### Name` header. `(since)`/`(until)` for time-bound facts | | `entities.md` | 3-line registry: `### Name` / key facts / `status\|last\|→[[link]]`. Max 3 content lines per entry |
| `calendar.md` | Edit in place | | `calendar.md` | Edit in place |
| `health.md` | Edit `## Current State`, append to `## History` | | `health.md` | Edit `## Current State`, append to `## History` |
| `philosophy.md` | Edit in place | | `philosophy.md` | Edit in place |

View File

@@ -94,15 +94,16 @@ Here's what builds up over time. None of this is pre-filled — it emerges from
- 2026-03-12 [insight]: Realized I've been avoiding the budget conversation. Not about money — about control. - 2026-03-12 [insight]: Realized I've been avoiding the budget conversation. Not about money — about control.
``` ```
**`memory/work/acme/entities.md`** — people and context: **`memory/work/acme/entities.md`** — compact 3-line registry:
```markdown ```markdown
### Sarah Chen ### Sarah Chen (Engineering Manager)
- Role: Engineering Manager (direct report to VP Eng) - Direct report to VP Eng | Joined Jan 2025 | Runs platform team | Prefers async over meetings
- Context: Joined Jan 2025. Runs platform team. Prefers async updates over meetings. - status: active | last: 2026-03-10
- Notes: Advocated for my promotion. Values shipping over polish.
``` ```
Heavy entries get promoted to thread files — the entity stub just links: `→ [[work/acme/sarah-chen]]`.
### Progressive Condensation ### Progressive Condensation
Two processes: Two processes:

View File

@@ -1,4 +1,5 @@
<!-- L0: Distilled interaction and workflow rules from experience --> <!-- L0: Distilled interaction and workflow rules from experience -->
# Cog — Learned Patterns # Cog — Learned Patterns
<!-- Edit in place. Distill self-observations into actionable patterns. Timeless rules only. HARD LIMIT: 110 lines / 7.5KB. --> <!-- Edit in place. Distill self-observations into actionable patterns. Timeless rules only. HARD LIMIT: 70 lines / 5.5KB. -->
<!-- Domain-specific patterns go in satellite files (e.g. work/acme/patterns.md), loaded only by their owning skill. -->