Structured Memory Observations

Structured memory observations are the inspectable metadata layer for CoWork OS archive memory. They keep the existing local-first memories table authoritative, then add a sidecar index that makes memory easier to search, audit, redact, suppress, and expand only when the agent needs full detail.

The design was inspired by claude-mem-style observation cards, but implemented as a CoWork-native SQLite sidecar rather than a new external memory store.

Current Concept

CoWork memory now has four complementary shapes:

• Facts: short, prompt-visible facts and rules about you and your workspaces, kept as notes in the memory folder (~/CoWork Memory); commitments and notes about other people stay in memory_items. Both are written only through MemoryWriter (see Memory Engine).
• Archive memory: durable local memory rows stored in memories.
• Structured observations: metadata rows keyed by memory_id that describe archive memories with title, narrative, facts, concepts, provenance, files, tools, source events, privacy state, and migration status.
• Durable runtime context: optional task-scoped message and compaction-summary rows used only for active-task recall through context_recall.

Structured observations are not a replacement for archive memory. They are an index and control plane over it.

Durable runtime context is not a replacement for structured observations. It is a runtime continuity lane for long active tasks, especially after context compaction. See Durable Runtime Context.

Dreaming works on the memory folder, not on the archive: it reads the folder and recent task conversations and never edits archive rows or observation metadata. See Dreaming.

Data Model

Structured metadata is stored in memory_observation_metadata, keyed by memory_id.

The sidecar includes:

• workspace_id, task_id, origin, and observation_type
• title, subtitle, narrative, facts, and concepts
• files_read, files_modified, tools, and source_event_ids
• content_hash, capture_reason, privacy_state, generated_by, and migration_status
• created_at and updated_at

Full-text search indexes observation text, facts, concepts, file paths, and tool names through the observation FTS table. The original memories row remains the source for full content and existing archive recall behavior.

Dreaming state is stored outside the observation sidecar: each dream is recorded in the memory folder's git directory, and its pending changes are a branch you review. The older dreaming_runs and dreaming_candidates tables from the retired curator are kept for history only.

This separation keeps observation metadata descriptive and keeps memory curation reviewable.

Capture Rules

MemoryService.capture(...) still creates the base archive row. When structured observations are enabled, it also asks MemoryObservationService to create metadata for the new memory.

Automatic capture from task events is salience-gated (src/electron/memory/memory-capture-salience.ts). The daemon archives only:

• completed task outcomes (task title, request excerpt and result summary)
• user feedback and decisions
• tool errors, failed steps, task errors and verification failures (compact)
• user corrections, captured once where they are detected

Other writers add accepted insights and curated promotions, Chronicle-derived context (always private), Playbook outcomes and explicit memory_remember calls of kind outcome, error or note (facts go to memory_items, see memory-engine.md).

Raw telemetry is not archived: tool calls and results, step progress, plan JSON, assistant and user messages, file create/modify events and passed verifications stay in the task timeline. Events from memory tools (memory_recall, memory_remember, memory_forget, context_recall, their deprecated aliases and kg_*) are never archived, so recalled memories are not re-captured. Secrets are redacted before storage.

Dreaming does not change capture rules. It reviews already-captured observations and transcript evidence after the fact, then proposes maintenance when evidence suggests that memory is missing, duplicated, stale, contradicted, or too noisy.

Privacy Controls

Inline privacy markers are handled before automatic capture:

• <no-memory> disables automatic memory capture for that task content.
• <private>...</private> redacts that segment from captured memory and marks affected derived memory private when needed.

Observation privacy states are:

• normal: searchable and recallable under normal memory rules.
• private: local-only and excluded from external mirroring.
• redacted: content was replaced and the row is excluded from prompt recall.
• suppressed: hidden from prompt recall, agent recall tools and default Memory Inspector search results.

Prompt recall checks both old prompt-recall ignore markers and observation privacy state. That means suppressed and redacted observations are excluded from both search-based recall and recent-memory prompt recall, and the agent read path (memory_recall, both its listing and its detail: "full" expansion) filters them as well. Private rows are never injected into prompts. The tool has no privacy-filter parameter, so the model cannot widen this, and Rebuild never loosens an existing privacy state.

Access profile boundary

Memory search, observation details, durable-context recall, and background memory reads remain subject to the effective task access profile, workspace scope, and tool restrictions. Memory tools cannot use a recall request to widen filesystem, network, command-tool, or cross-task access. Memory writes also remain subject to the separate Memory Write policy; an access profile cannot turn a blocked or unavailable write into an allowed one. The no-prompt runtime controls whether an optional review queue is used, while sensitive external payloads remain blocked regardless of queue mode.

Backfill And Migration

Backfill is deterministic and local. It derives titles, narratives, facts, concepts, and file paths from existing memory content and summaries without per-row LLM calls.

Backfill does not run as a synchronous startup write path. Startup only initializes the service. The Memory Hub can show status and trigger a rebuild action explicitly.

Backfill status tracks:

• total memory rows
• processed rows
• failed rows
• pending rows
• running state
• last run time
• last error

If metadata creation fails for a row, that row increments failed, not processed.

Progressive Recall

Agents recall progressively with one tool, memory_recall (audit §8.3):

1. Index (default): ranked results across memory items, this archive, earlier conversations and workspace knowledge, each with an id (memory:…, archive:…, event:…, kg:…, doc:…), lane, kind, snippet, date, provenance, relevance and an estimated token cost of the full text.
2. Full detail: memory_recall with ids (and detail: "full") returns the complete text of the selected results only. Reading an archive row in full counts as a use (reference count and last reference), which retention and tier promotion see; a listing does not.

Archive results come from the same hybrid search as before (MemoryService.searchForRecallAsync), with suppressed and redacted rows removed. Expansion of an archive: id is scoped to the active workspace, plus non-private imported rows; it is refused for any other workspace's row.

context_recall is the narrower task-scoped workflow for compacted active-task recall: it searches sanitized task messages and source-linked compaction summaries (durable runtime context, when enabled) and then the active task's conversation index, and expands one result by id. It always works on the active task.

The earlier tools (search_memories, memory_search_index, memory_timeline, memory_details, context_grep, context_describe) were hidden aliases for one release and are now removed; use memory_recall and context_recall.

Memory Hub Inspector

The Memory Hub Inspector is the primary user surface for structured observations. It is under Settings > Memory > Settings > Advanced.

It supports:

• search over observation metadata
• privacy filtering
• compact result rows with source labels and token estimates
• a detail drawer for title, narrative, facts, provenance, and timeline context
• metadata editing
• promotion to memory: your line in the memory folder (the workspace's file, or me.md for facts about you); commitments, or a memory folder that is off, keep the earlier memory_items path
• marking private
• suppressing prompt recall
• redaction
• confirmed soft-delete
• backfill status and explicit rebuild

Delete from the inspector is a soft-delete operation. It marks the observation suppressed, keeps a minimal local metadata record, marks the underlying memory private, and excludes it from default search, prompt recall and agent recall tools. It does not directly delete another workspace's memory row. Hard deletion happens through task delete and Clear All Memories (see Workspace Memory Flow) and through archive retention, which honours the workspace's retention_days.

Dream changes that touch your own notes are reviewed as diffs in Settings > Memory > Review; applied changes are commits that can be undone.

IPC And Security Boundary

Mutation IPC calls must include workspaceId and memoryId.

The service verifies ownership by loading the observation with both IDs before mutating it. Update, delete, redact, and promote cannot act on a memory outside the requested workspace.

Observation update patches are validated before reaching the service:

• only known editable fields are accepted
• privacyState is an enum
• arrays are capped
• strings have length limits
• unknown keys are rejected

This protects the FTS index from accidental oversized writes and prevents invalid array payloads from throwing inside metadata serialization.

Testing Expectations

Structured observation changes should include tests for:

• fresh database schema creation
• legacy database migration
• deterministic backfill and idempotency
• failed metadata writes reporting failed
• FTS insert/update/delete behavior
• workspace-scoped mutations
• soft-delete suppression
• <no-memory> and <private> handling
• prompt recall exclusion for redacted and suppressed observations
• progressive recall tool behavior
• durable runtime context enablement, active-task scoping, clear-memory deletion, durable-result echo filtering, direct-fact ranking, large-payload references, and summary-DAG parent links
• Memory Hub Inspector loading, editing, redaction, promotion, suppression, and rebuild flows
• Dreaming evidence use, Review tab accept/reject, and undo of applied curation operations

The native SQLite suite may be skipped on machines where better-sqlite3 cannot load. Keep mock-level tests for service behavior that must remain covered without native SQLite, especially startup backfill behavior, failure accounting, workspace scoping, and prompt-recall suppression.