Engram PAS Bus Shared Memory via Context

Engram PAS Bus Shared Memory via Context 1/19/26 Trent Carter An Engram-Bus in Verdict is a pointer-first shared memory fabric that sits above LightRAG and SAM and is wired through PAS so every agent “shares” repo and each other’s state without copying it into prompts: children emit only tiny Engrams (micro-claims + confidence + TTL + scope) that point to canonical sources (file spans, artifacts, commits, SAM entries), the parent distills/broadcasts only the top-K Engrams + pointer packs with a strict dereference budget, and the gateway enforces a hard “≤5% inline context” envelope by rejecting oversized messages and requiring resends in Engram+Pointer form—so models pull exact spans only when authorized, keeping prompts small, deterministic, and anchored to ground truth.

⸻

SPEC_Engram_Bus_PAS_Shared_Memory Status: Draft (v0.1) Owner: Verdict Core (Gateway + Orchestrator) Audience: Architect/Director/Manager/Programmer agents; Gateway, RAG, SAM implementers Goal: Make PAS agents share repo + each other’s “important pieces” while using ~5% of context, via pointer-first Engrams + enforced budgets. 0. Non-Negotiables

Pointer-first always. Agents must reference canonical sources; large inline context is disallowed except by explicit budget grants.

Gateway-enforced budgets. Prompts cannot “politely request concision”; enforcement must be deterministic.

Two memory planes only.

• Truth plane: repo + artifacts + commits (LightRAG pointers)

• Episodic plane: decisions/preferences/workflow continuity (SAM pointers)

Engram Bus is the glue layer that indexes + routes tiny claims across planes.

1. Definitions 1.1 Engram A small, structured memory atom:

• A _claim_ (1–3 sentences, tightly scoped)

• A set of _pointers_ to canonical sources

• Metadata: confidence, TTL, scope, tags, provenance

1.2 Pointer An address, not content. Examples:

• repo:path/to/file.py#L120-L188@

• artifact:SPEC_Foo.md#sec=BudgetRules@

• sam:decision:publish-domain@2026-01-08

1.3 Dereference Fetching the actual content behind a pointer. Must be budgeted and (optionally) granted by the parent. 2. System Overview 2.1 Components

• Engram Store (ES): Stores Engrams + indexes (hash-key, optional embedding).

• Engram Query (EQ): Retrieves top-K Engrams for a query key / embedding.

• Pointer Resolver (PR): Resolves pointers to file spans / artifacts / SAM entries.

• Budget Enforcer (BE): Rejects oversized messages, caps dereferences per turn.

• PAS Orchestrator: Allocates attention (grants dereferences), distills children output.

2.2 Data Flow Child → Parent (Compression)

• Child outputs: Engram[] + PointerPack + minimal response

Parent → Children (Broadcast)

• Parent outputs: SharedBriefMicro + role-targeted PointerPack + budgets + grants

LightRAG/SAM integration

• LightRAG returns pointers first; excerpts only under budget.

• SAM returns pointer + short statement; full entry only under budget.

3. Hard Context Envelope (5% Rule) 3.1 Inline Context Budget Define per message:

• MAX_INLINE_TOKENS (e.g., 800 tokens)

• MAX_ENGRAMS (e.g., 12)

• MAX_ENGRAM_CHARS (e.g., 500)

• MAX_INLINE_CODE_CHARS (e.g., 0 by default; only allowed with grants)

3.2 Dereference Budget Per agent turn:

• MAX_REPO_SPANS (e.g., 3)

• MAX_ARTIFACT_SECTIONS (e.g., 2)

• MAX_SAM_ITEMS (e.g., 2)

• MAX_TOTAL_DEREF_TOKENS (e.g., 1200)

3.3 Enforcement Rules (Deterministic) Budget Enforcer must:

• Reject responses exceeding inline token/char limits with error BUDGET_EXCEEDED.

• Require a resend formatted as Engrams + pointers.

• Reject dereference attempts beyond budget with DEREF_DENIED unless grant present.

4. Engram Schema 4.1 JSON Schema (v0.1)

{

  "$schema": "https://json-schema.org/draft/2020-12/schema",

  "$id": "verdict://schemas/engram.v0.1.json",

  "type": "object",

  "required": ["id", "kind", "claim", "pointers", "confidence", "ttl", "scope", "provenance"],

  "properties": {

    "id": { "type": "string", "description": "ULID/UUID" },

    "kind": { "type": "string", "enum": ["fact", "decision", "risk", "todo", "constraint", "diff", "test", "perf", "policy"] },

    "claim": { "type": "string", "maxLength": 500 },

    "pointers": {

      "type": "array",

      "minItems": 1,

      "maxItems": 12,

      "items": { "$ref": "verdict://schemas/pointer.v0.1.json" }

    },

    "confidence": { "type": "number", "minimum": 0.0, "maximum": 1.0 },

    "ttl": { "type": "string", "description": "ISO-8601 duration, e.g. PT6H, P7D" },

    "scope": { "type": "string", "enum": ["run", "project", "org", "global"] },

    "tags": { "type": "array", "maxItems": 12, "items": { "type": "string", "maxLength": 40 } },

    "hash_keys": { "type": "array", "maxItems": 32, "items": { "type": "string", "maxLength": 80 } },

    "embedding_ref": { "type": "string", "description": "Optional pointer to embedding storage/id" },

    "provenance": {

      "type": "object",

      "required": ["created_at", "created_by", "source"],

      "properties": {

        "created_at": { "type": "string", "format": "date-time" },

        "created_by": { "type": "string", "description": "agent_id" },

        "source": { "type": "string", "enum": ["rag", "sam", "agent", "tool"] }

      },

      "additionalProperties": false

    }

  },

  "additionalProperties": false

}

4.2 Pointer Schema (v0.1)

{

  "$schema": "https://json-schema.org/draft/2020-12/schema",

  "$id": "verdict://schemas/pointer.v0.1.json",

  "type": "object",

  "required": ["type", "ref"],

  "properties": {

    "type": { "type": "string", "enum": ["repo", "artifact", "sam", "url", "test", "diff"] },

    "ref": { "type": "string", "maxLength": 300 },

    "digest": { "type": "string", "description": "Optional content hash for integrity" },

    "span": { "type": "string", "description": "Optional Lx-Ly or section id", "maxLength": 80 }

  },

  "additionalProperties": false

}

5. Engram Indexing + Retrieval 5.1 Hash-key Generation (Engram-like) Purpose: fast candidate recall without heavy embedding search.

• Generate hash_keys from:

• normalized tokens / identifiers

• short n-grams (2–5) from claim + referenced file identifiers

• stable “concept keys” (service:gateway, budget:reservation, etc.)

Collision policy:

• Collisions acceptable; rank by (scope match, recency, confidence, pointer overlap).

• Keep top-N per key with TTL eviction.

5.2 Optional Embedding Ranker Use embedding only as a gating/rerank stage after hash recall:

Hash recall top 100 candidates

Embedding rerank to top K (K ≤ 12)

6. APIs 6.1 Engram Store API

POST /engram

  -> stores Engram (validated)

GET /engram/query?keys=...&k=...

  -> returns Engram[]

GET /engram/query_by_embedding?embedding_ref=...&k=...

  -> returns Engram[]

DELETE /engram/{id}

  -> removes Engram (admin/tool)

6.2 Pointer Resolver API

POST /pointer/deref

  body: { "pointer": {...}, "budget_token": "...", "max_tokens": 400 }

  -> returns { "excerpt": "...", "pointer": {...}, "content_digest": "..." }

6.3 Budget Enforcer API (Gateway internal)

• validate_agent_message(message) -> ok | BUDGET_EXCEEDED

• validate_deref(agent_id, pointer_type) -> ok | DEREF_DENIED

• issue_grant(parent_id, child_id, pointer, cap) -> budget_token

7. PAS Message Contracts 7.1 Child → Parent (Required Format)

ROLE:

TASK_STATUS:

ENGRAMS:

  - kind: risk

    claim: "Potential contradiction between SPEC budget rules and current gateway reservation flow."

    confidence: 0.72

    ttl: "P7D"

    scope: "project"

    pointers:

      - { type: "artifact", ref: "artifact:SPEC_Budget_Enforcement.md#sec=Rules@" }

      - { type: "repo", ref: "repo:services/gateway/vendor_client.py#L280-L360@" }

POINTER_PACK:

  - { type: "repo", ref: "repo:services/gateway/routes/completions.py#L900-L1060@" }

DEREF_REQUESTS:

  - pointer: { type: "repo", ref: "repo:services/ledger/reservation.py#L1-L220@" }

    reason: "Need to confirm reservation atomicity behavior."

OUTPUT:

  summary: "Top risks identified; need deref grant for ledger code."

  next: "Await grant or alternative pointers."

7.2 Parent → Child (Required Format)

ROLE:

SHARED_BRIEF_MICRO:

  - "Goal: implement Engram Bus with 5% context rule; pointer-first only."

  - "Hard constraint: budget enforcement in gateway; reject oversize."

BUDGETS:

  max_engrams: 10

  max_repo_spans: 3

  max_sam_items: 2

  max_inline_tokens: 800

GRANTS:

  - budget_token: ""

    pointer: { type: "repo", ref: "repo:services/ledger/reservation.py#L1-L220@" }

    cap_tokens: 500

TARGET_POINTER_PACK:

  - { type: "artifact", ref: "artifact:SPEC_Engram_Bus_PAS_Shared_Memory.md#sec=Budgets@" }

INSTRUCTIONS:

  - "Confirm atomicity + idempotency; emit 3-5 Engrams max; no inline code."

8. Failure Modes and Guardrails 8.1 Common Failures

• Context bloat: agent pastes code/spec text

• Pointer rot: pointers missing SHA or refer to moved lines

• Unanchored claims: Engram without pointers

• Hallucinated deref: agent claims it dereferenced without resolver proof

• Engram spam: too many low-quality Engrams pollute store

8.2 Guardrails

• Reject Engrams with:

• no pointers

• pointers without a resolvable reference format

• claims > max length

• Require deref responses to include:

• content_digest returned by resolver

• the exact pointer used

8.3 Pointer Integrity Pointer Resolver returns content_digest. Store it in Engram provenance when deref’d. 9. Determinism Targets (v0.1)

• Budget enforcement: fully deterministic

• Engram validation: schema-validated

• Hash-key generation: deterministic normalization

• Rerank: may be probabilistic if embedding similarity used; constrain by fixed K and stable tie-breakers (recency, confidence, scope).

10. Acceptance Criteria 10.1 Functional

• Agents can coordinate on a repo task with:

• ≤ 800 inline tokens per message

• ≤ 3 repo dereferences per turn

• No inline code unless granted

• Parent can broadcast a SharedBriefMicro that is ≤ 30 lines.

10.2 Correctness

• Every Engram used in decision-making has ≥ 1 pointer.

• Dereferenced excerpts are provably tied to pointers via content_digest.

• Oversized messages are rejected and successfully resent in compliant format.

10.3 Performance

• Engram query (hash recall) p95 < 20ms (local store target)

• Pointer deref p95 < 120ms (local FS) / < 300ms (remote artifact store)

11. Test Matrix 11.1 Budget Tests

• Send message with 1200 inline tokens → expect BUDGET_EXCEEDED

• Attempt 4th repo deref without grant → DEREF_DENIED

• Attempt inline code without grant → rejected

11.2 Pointer Tests

• Pointer missing SHA → rejected (v0.1)

• Pointer refers to nonexistent file → deref fails with clear error

11.3 Engram Quality Tests

• Engram without pointers → rejected

• Engram over length → rejected

• Engram TTL expired → excluded from queries

12. Rollout Plan (Minimal, Safe) Phase 0 (Deterministic core)

• Schemas + store + query by hash

• Pointer resolver for repo/artifact/SAM

• Budget enforcement + message rejection loop

• PAS templates enforced

Phase 1 (Quality upgrades)

• Optional embedding rerank

• Duplicate/collision handling improvements

• Scope-based routing heuristics (who sees what)

Phase 2 (Optimization)

• Prefetch based on deterministic pointer packs

• Offload large indexes; caching strategies

• Cross-run learning of high-value Engrams (careful with privacy / scope)

⸻

If you want, next I’ll produce a repo-ready patch plan: exact filenames/paths (Gateway middleware, Engram store service, schemas directory, and unit tests) and the precise interfaces each PAS role must implement so this becomes a real contract rather than “prompt etiquette.”