Memory as a Service (Proposal)¶
Status: draft proposal. Not part of the canonical numbered set. If accepted, the use-policy fields fold into 04 Data Model §9 (
knowledge_fact) and the Knowledge Synthesis & Wiki Layer proposal; the boundary and topology fold into 02 System Architecture and 05 Persistence, Storage & Ingestion; the decision record (ADR-029 below) is appended to the decision log. Do not run the docs index/changelog maintainer against this file until it is promoted. The ADR number is provisional (latest accepted is ADR-027). This proposal pairs with the Service Library & Capability Surface proposal (ADR-028), which establishes the library and substrate rule this memory service is the first Convex-substrate member of.
Precis¶
Memory should be designed as a service with a hard boundary, even while it physically lives inside the Thinklio Convex deployment. Every read and write of memory — including from Thinklio's own agents — goes through a stable MCP/HTTP contract and a single internal module, never through ad-hoc ctx.db access scattered across the codebase. The store stays Convex (it is reactive, vectorised, durably compiled — see Knowledge Synthesis & Wiki Layer); what changes is that access is mediated.
The payoff is leverage. Once memory is a clean, governed, boundaried service inside Thinklio, the question "co-located vs its own deployment" becomes a configuration/topology choice, exactly mirroring the tenancy tiers of ADR-022. Default: co-located, keeping co-transactional governance and in-deployment foreign keys. Graduated: its own Convex deployment(s) — which is precisely what a future personal-memory product (see Personal Memory as a Service) needs. Same code; the deployment is configuration.
The proposal also adopts one concrete primitive from Open Brain: the instruction-vs-evidence use-policy on memory atoms (agent-written memory is evidence until promoted), plus recall traces.
1. Background — memory is the most core-coupled service¶
Among candidate services, memory is uniquely coupled to the core, which is why it must not be naively extracted to its own deployment today:
knowledge_fact.sourceInteractionIdis av.id("interaction")— a hard foreign key into Thinklio's own data.- Extraction writes a fact and marks wiki sections dirty in one transaction.
- Governance middleware reads a policy and writes a fact in-process, zero network hop — the headline ADR-017 benefit.
- The wiki compiler (single-writer Tier 2 Workflow) and the unified retrieval pipeline (wiki + facts + library, wiki proposal §10) assume co-located data.
A stateless proxy like Cliniko pays nothing to be a separate service; memory pays atomicity, cross-deployment IDs, and in-process governance. The resolution is not "don't make it a service" — it is "make it a service boundary now, defer the deployment split until something forces it." The boundary is cheap; the deployment split is not.
2. What memory is — and what it is not¶
The boundary is only coherent if we are precise about what sits behind it. Per the Open Brain guardrail and Thinklio's existing selective-distillation design, memory is the distilled, governed layer, not the raw stream.
| Behind the memory boundary (memory) | Feeds memory (sources) | Never memory (separate) |
|---|---|---|
knowledge_fact atoms (provenance, confidence, scope) |
Chat transcripts (message) |
Credentials / connections (the vault, doc 07) |
| Compiled wiki pages & promoted derivations (wiki proposal) | Audit log (audit_log) |
|
Curated documents & chunks (library_item) |
Uploaded files before extraction (media) |
|
| Identity / profile / preferences (the "about me") | Tool-call results before distillation |
Two notes. First, profile/preferences/the people-graph is a first-class memory category that the current model under-specifies; it is the most portable and the most valuable for personalisation, and it matters most for the personal-memory product. Second, raw transcripts and audit logs are sources, not memory — they feed extraction; you port distilled facts and profile, not gigabytes of logs. Credentials stay in the vault and never enter memory.
3. The boundary discipline (the cheap-now move)¶
The single discipline that makes the future split a config flip rather than a rewrite:
All memory access — including Thinklio's own agents — goes through the memory service's API/module boundary. No unrelated function reaches into memory tables directly.
This is the same "route access through a clean interface" instinct doc 15 §7 already records for tenancy, applied to the memory subsystem. Concretely:
- A single internal module (
convex/memory/*) owns all reads/writes of the memory tables; it exposes typed functions to the rest of the app and an MCP/HTTP facade to outside callers. - Agents call memory through that facade contract, identical to how an outside app would. Internal callers get a fast in-process path; the contract is the same.
- Retrieval, extraction, and the wiki compiler are memory-domain logic and live inside the service module, not scattered across agent code.
If this holds, lifting memory into its own deployment later means re-pointing the facade at a remote deployment and rewriting foreign keys to opaque references — mechanical, not architectural.
4. Topology as configuration (mirrors ADR-022)¶
Memory deployment topology is modelled on the tenancy tiers, not invented fresh:
| Co-located (default) | Dedicated (graduated) | |
|---|---|---|
| Where memory lives | In the Thinklio Convex deployment | Its own Convex deployment(s) |
| Access path | In-process fast path behind the facade | Facade over the network (MCP/HTTP) |
| Foreign keys | Native v.id(...) into Thinklio data |
Opaque references across the boundary |
| Governance | Co-transactional, in-process | Use-policy enforced in the memory service; caller trusted/granted |
| Why move | — | Personal-memory product; multi-app reuse at scale; a whale tenant's dedicated memory; data residency |
| Cost paid | None | Cross-deployment refs, eventual consistency on ingest, duplicated use-policy enforcement |
The decision of which Convex database memory belongs to becomes a per-deployment configuration — the same code serving the tenanted Thinklio store or a personal-product store by pointing at a different Convex deployment. That is the leverage thesis from the design discussion made concrete.
5. The governance split¶
The boundary partitions governance cleanly, with no duplication:
- Inside the memory service: epistemic & use-policy governance. Is this atom sourced or inferred? May it be used as an instruction? Has it been reviewed? This is memory-domain logic and travels with memory wherever it deploys.
- At the caller: budget, credit, and rate governance. How much LLM/tool spend this principal may drive, throttling, attribution. This stays with whoever invokes memory and is enforced by the existing harness.
This split is why "memory as a service" does not fracture the governance story: the two halves have clear, non-overlapping owners.
6. The use-policy primitive (the Open Brain borrow)¶
Open Brain's agent-memory schema enforces, as column defaults, the rule that agent-written memory is evidence-only until a human (or a trusted import, or a confidence threshold) promotes it to instruction-grade. Thinklio's wiki proposal already separates epistemic status (truthStatus: inferred, reviewState: machine | human_reviewed | flagged); this primitive adds the orthogonal permission axis: may an agent act on this?
Add to knowledge_fact (and any memory atom) — singular naming to match the current schema and doc 04 §9; reconcile with the plural-vs-singular open question noted in the wiki proposal §13.1:
// additive fields on knowledge_fact (and derivation, wiki_page where applicable)
usePolicy: v.object({
canUseAsEvidence: v.boolean(), // default true — may inform reasoning, always cited
canUseAsInstruction: v.boolean(), // default false — may be treated as a directive/rule
requiresHumanConfirmation: v.boolean(), // default true — promotion needs human or trusted import
}),
reviewStatus: v.union( // default "pending"
v.literal("pending"),
v.literal("confirmed"), // human-confirmed or trusted-import
v.literal("rejected"),
),
promotedBy: v.optional(v.object({ // provenance of any promotion to instruction-grade
type: v.union(v.literal("human"), v.literal("trusted_import"), v.literal("threshold")),
principalId: v.optional(v.string()),
at: v.number(),
})),
Why now. Adding the columns is nearly free; retrofitting them across a populated store is not (the same "cheap now, expensive later" logic doc 15 §7 records for tenancy). They are also the honest answer to the wiki layer's drift risk: the compiled wiki may render inferred/evidence atoms with appropriate hedging, but may treat only instruction-grade atoms as directives. The use-policy axis is the governance dial the Karpathy-style compiled layer needs to stay honest over time.
Relationship to existing fields. truthStatus/reviewState answer is it true / sourced? usePolicy/reviewStatus answer may an agent act on it? They are orthogonal: a sourced fact can still be evidence-only (not a standing instruction), and a human can confirm an inferred derivation as instruction-grade. Retrieval and the wiki compiler read both axes.
7. Recall traces¶
Thinklio has audit_log (what was written) but not retrieval-level provenance (which memory was surfaced into which agent turn). Add a recall trace so that "why did the agent believe that?" is answerable and so the personal-memory product can show users what a connected tool actually read:
recall_trace: defineTable({
accountId: v.optional(v.id("account")), // optional: personal-domain recalls are user-rooted
principalId: v.string(), // who recalled (user | agent)
interactionId: v.optional(v.id("interaction")),
query: v.string(),
results: v.array(v.object({ // what was surfaced, and how it ranked
sourceType: v.string(), // fact | wiki_page | library_item | derivation
sourceId: v.string(),
score: v.number(),
usedInResponse: v.boolean(),
})),
at: v.number(),
})
.index("by_principal", ["principalId"])
.index("by_interaction", ["interactionId"]),
Recall traces are memory-domain data and live inside the service boundary. They double as the audit substrate for GDPR "what did this tool see about me" in the personal-memory product.
8. The service contract (sketch)¶
The memory service exposes a small, stable surface — projected per protocol per the capability-surface model (ADR-028):
memory.search(query, scope, limit, filters)→ ranked atoms with provenance and use-policy.memory.capture(content, scope, sourceRef)→ new atom, defaulted evidence-only/pending.memory.promote(atomId, by)→ governed transition to instruction-grade.memory.get(atomId)/memory.relations(atomId)→ fetch + graph edges.memory.recallTrace(...)→ write/read recall provenance.- Read models for the compiled wiki (browse pre-synthesised pages with citations).
The same contract serves Thinklio's agents (in-process), fallbot, future Thinklio apps, and — when graduated — a personal-memory product, each as a governed principal.
9. Build sequence¶
- Establish the boundary in place. Move all memory reads/writes behind
convex/memory/*; nothing else touches memory tables. (Cheap; do first — it is the load-bearing discipline.) - Add use-policy + reviewStatus + recall_trace as additive fields/tables, with defaults. (Cheap; do before the store is populated.)
- Expose the MCP/HTTP facade over the boundary (Convex
httpActionstreamable transport, or thin facade — see ADR-028 open question 1). - Wire the use-policy axis into retrieval and the wiki compiler (evidence vs instruction handling, promotion path).
- Defer the deployment split until the personal-memory product, multi-app reuse, or a residency requirement forces it — at which point it is a config change, not a rewrite.
10. Draft decision record (draft ADR-029)¶
To be appended to decision-log.md on acceptance. Number provisional.
ADR-029: Memory as a Boundaried Convex Service¶
Date: 2026-06-15 Status: Proposed
Context: Memory (knowledge facts, compiled wiki, curated documents, profile) is the most core-coupled candidate service: hard foreign keys into interactions, transactional ingestion, and in-process governance make a naive extraction costly. Yet a clean, reusable memory boundary is the enabler for both multi-app reuse and a future portable personal-memory product.
Decision: Keep memory in Convex and, for now, co-located in the Thinklio deployment, but mediate all access — including Thinklio's own agents — through a single internal module and a stable MCP/HTTP contract. Treat the co-located-vs-dedicated deployment choice as a configuration/topology decision modelled on ADR-022's tenancy tiers. Partition governance: epistemic/use-policy inside the service, budget/credit at the caller. Adopt the instruction-vs-evidence use-policy primitive (evidence-only by default, human/trusted promotion to instruction-grade) and recall traces.
Reasoning: The boundary is cheap and makes the eventual deployment split mechanical rather than architectural, while preserving co-transactional governance and native foreign keys today. The use-policy axis is the governance dial that keeps the compiled wiki honest over time and is near-free to add before the store is populated, expensive to retrofit after.
Implications:
- New discipline: no direct memory-table access outside
convex/memory/*. knowledge_fact(and applicable atoms) gainusePolicy,reviewStatus,promotedBy; newrecall_tracetable.- Retrieval and the wiki compiler read both the epistemic and the use-policy axes.
- The deployment split is deferred and, when triggered, is a configuration change — the same code serving a tenanted or a personal-product Convex deployment.
- First Convex-substrate member of the service library (ADR-028).
11. Open questions¶
- Facade host — Convex
httpActionstreamable-HTTP vs a thin separate facade (shared with ADR-028 Q1). - Promotion thresholds — what
confidence/corroboration/accessCountthresholds (if any) may auto-promote evidence to instruction-grade without a human, and per-account policy control of those thresholds. - Profile/preferences modelling — whether the "about me" category is new tables or a typed use of
knowledge_fact+note; resolve before the personal-memory product depends on it. - Cross-deployment foreign keys — the opaque-reference scheme for
sourceInteractionIdand friends once memory graduates; design alongside the doc 15 §6.3 export/import ID remapping so they share machinery. - Singular vs plural table naming — inherits the wiki proposal §13.1 reconciliation; must be settled before any of these ship.
12. Revision history¶
| Date | Change |
|---|---|
| 2026-06-15 | Initial draft proposal. |