Async SQLite migration plan

Date: 2026-09-27. Status: DB0 and DB1 landed; DB2 landed behind COWORK_DB_WORKER=1; DB3 (timeline writes and projections in the worker) landed behind COWORK_DB_WORKER_TIMELINE=1; DB4 (usage reports in a read-only reporting reader, usage rollups and post-startup maintenance in the worker, memory hybrid search in the FTS worker) landed behind COWORK_DB_WORKER_REPORTS=1; DB5 (revision-checked settings and policy transactions, with host encryption and worker or host storage) landed, its worker route behind COWORK_DB_WORKER_SETTINGS=1; DB6's lifecycle, schema bootstrap, memory capture and read overlay landed; the per-domain moves are tracked by the dependency audit.

Revised 2026-09-27 after a source review. The revision adds call-site counts and build, lock, and test constraints; a host-thread reduction phase (DB1) that needs no worker; lock, preemption, and durability rules; and enforcement from DB1. Phases after DB0 were renumbered.

Source baseline: local checkout at eff5d1262, including pre-existing uncommitted changes. This is a source-backed plan, not a release audit or a fresh performance measurement. Implementation must record its own baseline and preserve concurrent work.

Outcome

Move application SQLite execution off the Electron main thread and the Node daemon/CLI event loops, preserving durable acknowledgements, transaction boundaries, event ordering, profile isolation, and encrypted settings behavior. Retain SQLite and better-sqlite3; execute synchronous SQL inside dedicated workers behind asynchronous application APIs.

Success means a slow query or lock wait cannot stop the host event loop from handling cancellation, IPC, or network callbacks. Writer throughput remains bounded by SQLite, storage, and query cost. This project also measures and reduces that work through batching, bounded reads, and query/index improvements. Reductions that need no worker ship first, so users benefit before the worker protocol is complete.

Evidence and scope

The inventory must cover raw SQL outside the database directory, type-only versus runtime imports, singleton access, scheduled callbacks, synchronous getters, cross-repository transactions, and separate database files such as checkpoint locks. Every remaining synchronous application-thread database path needs an owner and removal phase. The counts above are text-search counts at the source baseline; DB0 replaces them with a call graph.

Architecture decisions

1. One write-capable worker per runtime/profile connection. A desktop process, standalone daemon, and direct CLI process can each own a worker. SQLite continues to arbitrate concurrent writers to the same profile. A shared cross-process database broker would be a separate change, justified by measured contention after this migration. The worker is single-threaded, so any wait inside it (a busy-timeout sleep or a long statement) stalls every queued command in that runtime even while the host event loop stays responsive. The contract below keeps such waits short.
2. Keep heavy reads separate. Retain the FTS read worker and add a bounded reporting reader only if profiling supports it. Correctness-sensitive reads and reads immediately following a write use the write worker. Read workers open short-lived snapshots after the required commit barrier; eventual consistency is allowed only for documented search/reporting uses.
3. Typed domain commands. Add a request/result protocol and asynchronous facade. Requests contain validated, cloneable data and explicit profile identity. Database handles, transaction callbacks, functions, arbitrary renderer SQL, and Electron objects do not cross the boundary. Existing synchronous repositories can remain implementation details inside the worker. Convert in two tiers. For the long tail, expose allowlisted repository methods whose arguments and results are plain cloneable data through typed one-to-one async proxies, so conversion is mechanical. Write domain commands by hand for transaction groups and measured hot paths. Each proxy call runs as its own transaction, so a caller that composes several repository calls inside one transaction needs a domain command.
4. Transactions execute entirely inside the worker. A command such as appendUserMessage or resolveApproval includes all dependent reads, checks, and writes. Never hold a transaction open while calling the host, network, or another worker. Existing same-connection invariants are preserved. Write commands begin IMMEDIATE (.immediate() in better-sqlite3), so the write lock is acquired up front through the busy handler instead of failing with SQLITE_BUSY_SNAPSHOT when a read upgrades to a write. Group writes into one transaction only where atomicity requires it or a measurement shows a gain: in DB1, wrapping each timeline event and its projections in one transaction with a savepoint per projection cut throughput by 15 to 20%, and a plain wrapping transaction gained nothing over autocommit (see the DB1 subsection).
5. Separate acceptance from commitment. A queued request is not a successful write. Durable API responses resolve after commit, at the durability level chosen in decision 9. Host state, completion notifications, approval execution, and external acknowledgements that depend on persistence follow that commit. Ephemeral streaming updates retain their existing non-durable semantics.
6. Bound pending work by count and bytes. Preserve order for dependent operations within a session. Reserve capacity for control operations, while retaining their prerequisite writes; give maintenance limited service to prevent starvation. Backpressure propagates to producers. Only explicitly ephemeral progress or recomputable counters may be coalesced or discarded.
7. No automatic synchronous fallback for migrated domains. Worker unavailability produces a bounded, explicit failure or documented read-only degradation. Backend rollback happens after draining/restarting the runtime, never by replaying an uncertain write on a different connection.
8. Bound every command's cost by construction. A dispatched statement cannot be interrupted, so deadlines and cancellation take effect only while a command is queued. Commands use limits, pagination, or chunking so that the worst case of any single command fits the latency budget; review rejects unbounded scans and deletes, and full VACUUM outside an idle window. Terminating the worker remains a recovery path, not a timeout mechanism.
9. Set the durability level explicitly. Set synchronous on every connection instead of inheriting a default that depends on the build and on whether the run created the database. In WAL mode, NORMAL survives process and worker crashes but can lose the latest commits on power loss or an OS crash; FULL survives those at the cost of an fsync per commit. Because the worker serializes commands, it can raise the level around specific commands. Decide whether approvals, consent revocations, and external-effect admission need FULL, and state the resulting crash model with the acceptance results.
10. Enforce the boundary from DB1. A CI ratchet allows the counts of runtime better-sqlite3 imports, getDatabase() calls, and .prepare( calls outside worker modules only to decrease, with each remaining site listed in the exception register. Type-aware no-floating-promises and no-misused-promises rules cover migrated modules and their callers. DB7 turns the ratchet into a zero-exception dependency gate. Without the ratchet, concurrent feature work adds synchronous SQL faster than the migration removes it.

SQLite WAL supports overlapping readers and a writer, but still permits only one writer at a time; moving queries into workers isolates blocking from the application event loop. See the SQLite WAL documentation and better-sqlite3 worker documentation.

Request, recovery, and consistency contract

• Requests carry a transport request ID, worker generation, operation name, ordering key when needed, deadline, and bounded arguments. Retryable mutations additionally carry a stable logical operation ID across reconnects/restarts.
• Reuse existing event IDs and domain idempotency keys. For mutations without an adequate receipt, store a receipt/result reference atomically with the mutation, with a documented retention/retry horizon. An ID alone does not make replay safe; the same logical ID with conflicting input must fail.
• If a worker dies after commit but before reply, reconcile the logical operation against durable state after reopening. Unknown outcomes remain explicit. Do not blindly retry increments, approvals, external-effect admission, or other mutations.
• Cancellation before dispatch removes queued work. Cancellation or timeout during execution does not prove rollback, and a dispatched statement cannot be interrupted (decision 8). Settle/reconcile the outcome and reject stale-generation replies. Killing a worker is a failure/recovery path, not normal query cancellation.
• Serialize dependent session operations before asynchronous preprocessing can reorder them. Allocate durable sequence values transactionally where multiple runtimes may write; preserve existing ephemeral timeline ordering conventions.
• Retry lock conflicts only after establishing rollback/no commit, with a bounded total deadline and asynchronous backoff. Preserve session order during retries. Measure busy waits in DB0, then replace the 5-second in-connection wait in write workers with a short one. A failed BEGIN IMMEDIATE establishes that nothing committed; the scheduler then parks writes in session order, keeps serving reads that do not depend on a parked write, and retries with a single probe under backoff. Replacing one long block with a retry storm is not acceptable.
• Bound request payloads, result sizes, worker message queues, and host-side decoding work. Keep large history scans paginated and process reductions in the worker where possible.
• Treat every unexpected worker exit, including exit code zero, as loss of service. Reject or reconcile in-flight work, bound restarts, and prevent a crash loop from growing queues.

Delivery phases

Each phase is independently reviewable. A phase is complete only with its listed evidence. Ship each phase's improvements as they land rather than holding them for the full migration. Re-estimate the full migration after DB0 produces the call graph; the repository surface is too broad for a credible fixed PR count today.

DB0 precedes DB1 so the baseline predates the reductions. DB1 and DB2 can proceed in parallel; DB2 precedes any worker-routed domain. DB3 and DB5 must respect their discovered transaction dependencies. DB6 and DB7 close the migration; finishing the reductions or the pilot alone is not completion.

DB0: scope decision

DB0 attributes host event-loop delay by source. If synchronous filesystem, child-process, or CPU work outweighs SQLite at the tested task counts, this project alone cannot meet the responsiveness gate below. Record then whether to add a complementary change that moves AgentDaemon out of the Electron main process into a utilityProcess, building on the existing Node-only daemon. That change isolates the UI from all agent work but does not remove blocking between sessions that share a runtime, so it complements the worker migration rather than replacing it.

DB1: event transaction shape

The proposed shape was one IMMEDIATE transaction per event: insert the event, run each same-connection projection in its own savepoint so a failure rolls back only that projection, then commit. It was implemented and measured on the DB0 workload with instrumentation off, and rejected:

At synchronous = NORMAL a WAL commit does not fsync, so merging commits saves little, while savepoints add statements and statement-journal work. Timeline persistence keeps separate statements. Revisit grouping inside the worker only with a measurement, for example when batching several events per command. Note that the instrumentation counts JavaScript run inside a wrapped transaction as SQLite time, so compare write shapes by throughput and event-loop delay, not by SQLite share.

DB3: event authority and async callers

As built: the host prepares each TaskEvent (and the activity and usage rows derived from it) and hands the rows to TimelineWriter, which the worker inserts in ordered batches together with a timeline_projection_outbox row; the worker then projects outbox entries and deletes each entry with its projection writes. Every insert is keyed by a host-generated id and ignored if present, and only the insert that takes effect writes the outbox row, so either side may commit a row. Host reads stay consistent without an overlay: before any repository read or write of a task's events (or of the activity feed), pending rows for it are committed on the host first ("flush through"); async readers such as runtime checkpoints wait for the worker instead. Lifecycle, approval, input, user-message, follow-up, and snapshot events were committed before logEvent returns until DB6 slice C2. Now they go to the worker like other rows. Their authority stays elsewhere: task status in the task row, and approvals and input requests in their own rows, all committed before the caller proceeds. completeTask resolves only after its milestone rows are committed. Readers that act on derived state (stale-turn checks, replay evaluation, the progress view) await projections first. Projection reads of task_events are bounded by the projected event, because later events already exist when projections trail inserts.

What this does not cover: a process crash can lose accepted non-milestone rows that neither side has committed yet (at most one batch); reporting queries that read task_events, activity_feed, or llm_call_events with raw SQL are eventually consistent by milliseconds; and memory capture still writes on the host, so a foreign write lock still stalls it until the memory domain moves (DB6).

persistTimelineEvent writes a TaskEvent before best-effort protocol, contract, and progress projections, each committing on its own. Record the authority used by each existing endpoint before changing this flow. Preserve its authoritative commit and use durable repair cursors/receipts or existing replay facilities for derived state. Do not turn optional projections into a new universal failure condition incidentally. Where a user operation already requires several writes atomically, dispatch them as one worker command.

Audit every logEvent(): void caller, EventEmitter handler, timer callback, and constructor dependency. A mechanical async signature change can create floating promises and reorder completion. The decision 10 lint rules catch unawaited promises; the audit covers ordering that lint cannot see. Make required persistence explicitly awaited; route genuinely detached maintenance through a supervised error/drain mechanism. Durable completion, queue-slot release, and approval-controlled effects must respect required preceding writes.

DB4: heavy reads and maintenance

As built: a second worker, the reporting reader, opens the database read-only and runs report scans; it never queues behind writes. Usage insights reports run there with a plan the host picks from projector state (rollups, rollups plus a raw tail during backfill, or the raw window), so parity reduces to the same query code on another connection. A reader failure or a cancelled request rejects the IPC call; it never becomes an empty report or a host rerun. Queued reads can be cancelled with an AbortSignal; a statement that already started cannot be interrupted (better-sqlite3 is built without the progress callback), so report commands are bounded by their period clamp.

Usage rollups (the legacy telemetry backfill, resets, and per-day rebuilds) run in the write worker as bounded commands: the backfill walks task_events by rowid in 2,000-row chunks, and rebuilds send at most 16 workspace-days per command. Each chunk selects its rowids with a plain forward scan before loading details, because in one statement SQLite sorted through the type index and ran the per-row routing lookups for every matching row before applying the limit (13 s for one chunk on the heavy fixture). Without the worker the same chunks run on the host, 50 rows at a time with a macrotask yield between them; before DB4 the backfill yielded only to microtasks, so it blocked the event loop for its whole length.

The whole hybrid memory search (lexical FTS, the embedding scan, and the full-row rerank) runs in the FTS worker, using one ranking function shared with the host. The worker keeps its own embedding cache; every repository write or delete of an embedding reaches it as an invalidation, and a 10-minute full reload bounds staleness from cascaded deletes. The embedding backfill's scans for missing embeddings run there too; computing and storing embeddings stays a host write. A worker failure raises an error instead of degrading to a host search.

Post-startup maintenance runs as bounded, idempotent chunks: run-duration backfill (100 tasks), payload sanitizing (5,000-rowid ranges, now covering every oversized payload instead of the 500 largest), orphan repairs (one statement per command), and orphan task-event deletion (1,000 rows, forward rowid cursor). Schema migrations stay atomic in DatabaseManager and are unchanged.

The usage lookups of a row's latest routing change and provider log use one indexed branch per event-type column, which took raw reports from 15 to 18 s to under a second on the heavy profile. The briefing and Box Brain recall use the async memory search. Legacy events converted on read are written back after the read, one task per transaction, in the worker when it runs.

Embedding backfill batches are written in the worker, and timeline cursors re-anchor on their row so paging survives a legacy task's conversion. Remaining on the host: capture-path embedding writes (with DB6), and with the host backend the raw report scan and a legacy session's deferred write-back.

DB5: encryption and freshness

As built: secure_settings rows carry a revision taken from one monotonic clock (secure_settings_revision_clock), so a revision is never reused, even after a delete. Existing rows read as revision 0 until their next write. The host decrypts, decides and encrypts; commitSecureSettingsWrites then compares the revisions the host read and moves only ciphertext, inside an IMMEDIATE transaction on the host or in the worker (secureSettings.commit). A conflict re-reads, re-applies the change, and re-encrypts. No settings transaction calls the keychain, key derivation, or the network: save() encrypts first, and keychain adoption decrypts and encrypts its canary before its transaction.

• Refused writes stay refused and say so. save() returns false; update throws SecureSettingsWriteRefusedError, so the permission, guardrail and credential paths reject instead of reporting success.

• Decisions do not overwrite each other. Permission rules remembered by an approval are applied to the latest stored settings. A settings edit made from an older snapshot keeps rules added since that snapshot. Credential fulfilment commits with its request row in one transaction and only while the request is pending. Resolving a credential decides on the latest vault and cannot write an older vault over a revocation.

• Revocations apply before the next protected action. Permission and guardrail caches compare their revision with the stored row on every load, one indexed read, so an edit by any process (the CLI, the daemon) applies to the next check. A recurring approval decided before a rule was revoked no longer re-activates it when written late. The unreadable-category memo is keyed by revision, so a row repaired by another process is read again.

• Pulse. Decisions and delivery results are planned on the host (decrypt, decide, encrypt, build the package) and committed with the consent windows, outbox and lease in one transaction under the settings revision that was read (pulse.commit, pulse.claim in the worker, the same functions on the host). A late response still cannot reverse consent: the revision, identity and lease must all still match.

• Keychain rows are only written by a process that can read them. The daemon and CLI run without safeStorage; they used to back up and replace an os: row they could not read, losing the desktop app's settings. They now refuse such writes (save() returns false, update throws, Pulse decisions fail with settings_write_refused and report no consent), and keep writing the rows they can read. Only the desktop app can use the keychain, and it is the process that verifies the keychain identity; its refusal flag covers every keychain write.

• Plain saves no longer overwrite blindly. The repository keeps, per category, the revision and ciphertext it last read or wrote (never plaintext). save() commits against that revision; if another writer changed the row meanwhile, it merges field by field against the value it last read (secure-settings-merge.ts): fields this save did not change take the other writer's value, and fields both changed keep this save's value, with a logged warning. This covers all plain save call sites without changing them. A save of a category this process never read is still unconditional.

Electron safeStorage is a main-process API; keep encryption and decryption in the existing host environment. Preserve Node fallback behavior and keychain identity checks without transferring plaintext through generic SQL messages. See Electron safeStorage.

Allow host caches only for explicitly cacheable settings. Populate them during awaited startup and update them after committed writes. Security and consent decisions use an authoritative revision check at their action boundary; periodic invalidation alone is insufficient. Cross-process settings edits require refresh/version checks. Pulse's encrypted settings update and consent/outbox mutation stay on the same worker connection in one transaction.

DB6: startup and shutdown

In progress. Landed so far:

• Migration serialization. DatabaseManager takes a crash-recoverable lock file (cowork-os.db.migration.lock, created exclusively) before it opens, configures and initializes the database; a lock whose process is gone, or older than 10 minutes, is broken. Four processes opening one fresh profile at once all succeed. That test found a real bug: applyConnectionPragmas switched the journal mode before setting busy_timeout, so a second process failed at once with SQLITE_BUSY; the timeout now comes first.

• Schema version gate. Initialization stamps PRAGMA user_version (CURRENT_SCHEMA_VERSION, 1). A profile from a newer build fails clearly (UnsupportedSchemaVersionError, a dialog on desktop) before anything touches it. The write worker and the reporting reader report ready only for that exact version, and the reader starts after the writer is ready.

• Pragmas on every connection. Writers set the lock wait, WAL and synchronous = NORMAL; read-only connections (the reader, the FTS worker, CLI discovery) set their lock wait.

• Shutdown records. Each runtime records its run in maintenance_state. A clean shutdown removes the record; a failed shutdown step or an undrained worker marks it incomplete, and a crash leaves it behind. The next start reports such runs (logged, and kept under last_incomplete_shutdown). Accepted writes are recovered from the database on that start: timeline outbox entries drain, deferred conversions re-run on read, and rollups rebuild from their watermarks. Close runs a PASSIVE checkpoint, which never waits on other connections. The direct CLI now closes its host connection too.

• Memory capture. A capture's memory row, embedding and observation commit in one transaction, in the worker when it runs (memory.capture) and on the host otherwise, instead of three to four separate host writes.

• Reads never write. Task event reads (full history, typed and limited reads, the replay tail, event cursors, multi-task reads, the latest timeline page) and activity-feed reads merge the writer's accepted, uncommitted rows in memory instead of committing them on the host first. Write-side hooks (deletes, payload updates, the next seq, migrations, pruning) still commit pending rows first.

• Schema startup in a worker. The runtimes open the profile with DatabaseManager.open(): the host prepares the filesystem (directory permissions, legacy directory migration, retired temp folders) and passes the absolute database path to a bootstrap worker, which takes the migration lock, checks the version, initializes and stamps the schema, and exits; the host then opens a connection that does not initialize. Errors keep their type. A build missing the worker entry initializes in-thread, with an error logged.

• Dependency audit. npm run qa:db:audit (also a test) requires every file in the ratchet register to match a rule in scripts/qa/sqlite-audit-rules.json naming its domain, how it is reached, and its migration plan; an unexplained file fails the check.

Under a foreign write lock, the host no longer waits for the lock on the task hot path. Since DB6 slice C2, milestone events (lifecycle, approval, input, snapshot) are committed by the worker. Boundaries that need one durable await TimelineWriter.committed, which waits without blocking the host thread; it falls back to a host commit only when the worker is no longer ready. The trade-off, accepted when closing DB6: a process crash (not a shutdown, which commits everything) can lose accepted non-milestone rows that neither side committed yet. That is normally the batch in flight, at most the writer's high-water mark when the worker falls behind. Milestones are committed before the task proceeds, and an interrupted task resumes from the committed rows.

• Domain moves: mailbox. A migrated domain lists its statements in a catalog (src/electron/database/statements/); its services call them by name through a statement port. The port runs them in the write worker when the domain's flag is on (COWORK_DB_WORKER_MAILBOX for mailbox), on the host connection otherwise. The worker's statements.read/statements.write commands resolve names in the domain catalog and never accept SQL. Mailbox (the mailbox services and AgentMail) moved first: its 281 prepare sites are gone from the register, parity is tested on both backends, and npm run qa:db:mailbox measures the sync against UI reads.

• Transaction units; domain moves: memory. A domain can register transaction units: pure, validated (db, args) functions that run several statements with logic between them in one IMMEDIATE transaction (or one read snapshot for read-only units), on the host or in the worker. Store classes get one unit per method and an async facade. Memory (tiers, observations, durable context, transcripts, the markdown index, Playbook evidence, dreaming, the Box brain and the knowledge graph) moved this way; its check-then-write sequences are single units, parity is tested on both backends, and npm run qa:db:memory measures indexing against recall.

• Domain moves: control plane; report units. The control plane's queries and its issue/run state machine are units (checkout, task attachment rows, release and lifecycle sync are single transactions). Task rows, company workspace provisioning and template import stay on the host with the storage layer. The planner and API handlers use burst-gated catalog statements. A read-only unit can be a report: it runs on the reporting reader when one is running (cost summaries). npm run qa:db:control-plane measures it.

• Domain moves: reports. Standup, agent reviews and briefing counts are reports-domain units (reads on the reporting reader, generation as single write units); usage insights already ran there through DB4.

• Storage layer, slice A. 25 leaf repositories are synchronous stores run as storage-domain units behind async facades with their old names (COWORK_DB_WORKER_STORAGE).

• Storage layer, slice B. Channels, channel users and sessions, memories and embeddings follow the same pattern. Channel config crosses the worker boundary sealed, and check-then-write sequences became single units. The work-session repositories stay synchronous stores shared by the session services, with read units for host-only readers.

• Services domain, Everyday Agent. The Everyday Agent is a store behind services-domain units. Admin policies are read on the host and passed to each unit. A refusal that marks a preview is returned from the unit and thrown after it commits.

• Services domain, evals. Eval cases, suites and runs are a store behind services-domain units. Case creation and each suite run are single units, and the host drops its cached task rows after a case is linked.

• Services domain, work contexts and session membership. Work contexts and session membership are stores behind services-domain units. Each authorization check shares a unit with the write it guards. Client bindings and the cached local principal stay on the host.

• Services domain, contact identity. Contact identities are a store behind services-domain units. The knowledge-graph match runs on the host, and the mailbox contact resolution is one unit.

• Services domain, managed agents. The managed repositories and the service's membership, audit and routine SQL moved behind services-domain units. The workspace permission checks are async, and an AST audit confirms every call is awaited.

• Services domain, subconscious. The subconscious repositories and the loop's own SQL moved behind services-domain units. The cross-domain evidence read is one reporting unit, and the target rekey is one write unit.

• Services domain, core learning. The 10 core learning repositories are stores behind services-domain units.

• Services domain, routines; approval ordering. Routine and workflow SQL moved to stores behind services-domain units. The workflow units are clocked, and a run upsert is one unit. Slice A's approval ordering gap is closed: the daemon denies pending approvals through the host store before a task's terminal update.

• Services domain, agents. The background and domain services move area by area into one services domain (COWORK_DB_WORKER_SERVICES). The 12 agent repositories are stores behind async facades, and the daemon hot path keeps the stores. The async-SQLite lint pass now also flags a promise returned unawaited inside try, which skips its catch. The migration had introduced 15 such returns; all are fixed.

• Storage layer, slice C2: async milestones. Measurements showed the daemon's remaining task SQL is non-blocking reads. The host stall under a foreign lock came from milestone events committing on the host inside logEvent, so C2 moved those commits to the worker instead of converting the daemon's repository calls. The daemon keeps its stores.

• Storage layer, slice C1. Tasks and workspaces follow outside the daemon hot path. The daemon, runtime wiring, host transactions and timeline transports keep the stores. Authorization guards that became async are awaited everywhere, which an AST audit and the typed promise lint check. Slice C2 (the daemon hot path and task events) follows.

• DB6 close-out. The rest of the services backlog moved behind services-domain units. That covers:

  • mission control, the activity feed, automation outcomes, councils, event triggers and supervisor exchanges;
  • the improvement loop, hooks, first-task, briefing and YouTube transcripts;
  • context policies, ACP, the file hub, temp workspace pruning and orchestration graphs;
  • Numbat security records, recurring approvals, usage telemetry, Pulse's reads and the channel tools' reads.

  The same moves closed the agent runtime, security, IPC, telemetry and runtime-wiring backlogs. The mailbox's thread sync is two round trips, and its reads use the reader connection when one is running.

DB6 status

DB6 is closed. Exit criteria:

• Dependency audit: every file in the register has a rule, and no rule is a backlog.
  • Shared SQL: 82 files run as units.
  • Documented stays: the storage-layer stores the daemon keeps, lifecycle, the renderer, memory checkpoint handles, and CLI discovery.
  • Wiring only: 17 files that pass connection handles to worker-backed facades, with no SQL of their own.
  • Reviewed exceptions: the daemon hot path and the work-session services beside it; settings (plain saves behind synchronous settings managers, the protected-credential vault commit, the legacy settings fallback); service schema DDL at construction; and two one-time startup migrations.
• Schema startup: the schema bootstraps in its own worker. Per-service CREATE ... IF NOT EXISTS at construction is a reviewed exception: it is idempotent and writes nothing once the tables exist.
• Host constructors and raw handles: application code has no raw SQL outside the reviewed exceptions. The remaining getDatabase() calls hand connections to worker-backed facades and ports.
• Shutdown: the runtime closes the reporting reader, then drains and closes the write worker. The timeline writer commits everything on shutdown.

Gaps closed:

• Slice A ordering: closed with the routines area.
• C2 crash loss: accepted, bounded and documented; see above.
• Mailbox: batched thread upsert and reader-routed reads.
• Unawaited returns inside try: the pre-existing ones are fixed.
• Floating promises on database paths: fixed. The daemon's completeTask is fire-and-forget in the executor, the router's outgoing-message log, and the queue manager. Promise-lint findings in channel adapters and UI windows, which do not touch the database, remain.

DB7 candidates recorded by the reviews, and how DB7 handled them:

• Async settings managers: scoped as a reviewed exception. The managers save rarely and synchronously; the settings transactions that decide policy already run in the worker (DB5).
• A combined vault commit for protected credentials: scoped as a reviewed exception. The commit is one short host transaction next to the keychain write.
• Removing the host fallback: not done. The host backend is the rollback path. It is chosen per run and never mid-run.

DB7 status

DB7 is closed. Evidence is in the baseline's "DB7 rollout" section. Exit criteria:

• Default on:
  • The worker and every domain flag are on by default in the desktop app, the daemon and the CLI (DATABASE_WORKER_ROLLOUT).
  • The desktop app awaits worker startup, so each run picks one backend.
• Rollback:
  • The kill switch is COWORK_DB_WORKER=0. To turn one domain off, set its flag to 0.
  • Rolling back is a restart on the host backend, demonstrated worker → host → worker on one profile.
• Workload:
  • At the supported active-task levels (1 and 8 tasks, and 50 submitted with 8 running), host loop p99 is 19–33 ms.
  • Under a 2 s foreign lock, loop p99 is 27 ms and the longest stall is 41 ms.
  • Throughput does not regress.
  • 20 and 40 tasks exceed the 50 ms gate, but both are above the supported cap and 3–4× better than the host backend.
• Recovery matrix:
  • These cases run against a real worker: disk full, the stale-snapshot read-then-write, two runtimes on one profile, and integrity after a crash.
  • They join the termination, lock, payload, timeout, saturation and restart suites from DB2–DB6.
• Packaged artifacts: the desktop artifact smoke checks the worker entrypoints and the native module in the ASAR, and launches on a disposable profile.
• Zero-exception gate:
  • Every audit rule names an owner. qa:db:audit fails when a file is matched only by a backstop rule, or when a rule has no owner.
  • The ratchet fails on any new SQL outside a unit store.
• Residual exceptions, each scoped and owned in sqlite-audit-rules.json:
  • the daemon hot path and the work-session services beside it;
  • synchronous settings saves, the protected-credential vault commit, and the legacy settings fallback;
  • per-service schema DDL at construction;
  • two one-time startup migrations;
  • the storage-layer stores the daemon keeps;
  • lifecycle, memory checkpoint handles, the renderer, and CLI discovery.
• Docs: architecture.md, performance-stability.md, memory-fts-performance.md and the troubleshooting runbook describe the ownership, the rollout and the recovery contract.

Resolve the exact profile directory in the host and pass an absolute DB path to workers; do not let a worker rediscover Electron paths or silently fall back to another profile. Split filesystem/keychain preparation from SQL initialization. Worker startup reports ready only after schema compatibility and migration complete. Read workers start afterward.

Define and qualify cross-process migration serialization with a crash-recoverable local migration lock and transactional schema-version rechecks. Apply connection-local pragmas, including the explicit synchronous level, on every connection. Keep same-profile desktop/daemon/CLI coexistence in the test matrix. Existing unsupported schema versions must fail clearly.

On shutdown: stop new task admission and background producers; let active producers settle their required writes; seal DB admission; drain accepted operations; close read workers and write connection; join worker termination. If a deadline expires, report incomplete shutdown and reconcile on next startup. Preserve current non-quiescent shutdown fences. Checkpoints are bounded and tolerate other readers; a busy checkpoint does not mean committed data is lost.

Proposed acceptance criteria

These are proposed gates to calibrate against DB0 on named hardware, not measured current performance. Establish them before comparing implementations.

• Correctness: zero lost acknowledged mutations, duplicate logical writes, per-session durable sequence violations, false durable-completion responses, or stale-policy approvals in the deterministic suite. Crash recovery either resolves an operation or exposes its uncertainty. Acknowledged means committed at the recorded synchronous level: process and worker crashes lose no acknowledged mutation, and the stated power-loss expectation follows the level chosen in decision 9.
• Host responsiveness: on the database-focused workload, p99 host event-loop delay below 50 ms and p99 lightweight IPC/health response below 100 ms at supported active-task levels. A separate process holding a write lock for two seconds must produce neither a matching two-second host stall nor a stall of reads queued in the same runtime. Attribute other synchronous filesystem/CPU work separately; if DB0 shows it dominates, apply this gate to database-attributable delay and follow the DB0 scope decision.
• Commit latency: proposed p95 authoritative event commit below 100 ms and p99 below 500 ms in steady state without injected lock faults. Lock-fault cases require bounded deadlines and correct outcomes instead of these latency thresholds.
• Throughput: useful completed operations/second must not regress by more than 10% against DB0 on the same fixture at 1 and 8 tasks; higher concurrency must satisfy the agreed queue/latency limits. Investigate worker messaging overhead and batching if it fails.
• Capacity: queue count, queued bytes, result sizes, retry duration, and worker restart count never exceed configured limits. After a burst, queued work drains and host/worker RSS and WAL size return to a documented steady range. Saturation yields explicit backpressure and preserves critical writes.
• Coverage: test 1/8/20/40 active tasks through supported admission routes and 50 submitted tasks with queueing. Use a benchmark-only override only if evaluating 50 active tasks; do not claim production support from bypassing the existing cap.
• Isolation: all workload and failure injection runs use disposable profiles and synthetic fixtures. Test two runtimes against the same disposable profile as well as separate profiles. Never seed QA tasks or lock experiments into the user's profile.

Verification and release checks

Use real SQLite and actual worker messaging for atomicity, contention, and crash tests. Unit mocks are suitable for caller branching, but cannot establish persistence behavior. These suites fail rather than skip when the native module cannot load, and they spawn the built worker entrypoint instead of mocking worker_threads. Run the worker smoke tests under Node and inside Electron with the native build that ships. Reuse protocol/replay, timeline, retention, approvals, Pulse, keychain identity, and graceful-shutdown regression suites.

Include: termination before dispatch/during transaction/after commit before reply; disk-full or failed commit; locked database; a deferred read-then-write racing a cross-process commit (the SQLITE_BUSY_SNAPSHOT case); a projection failure and a whole-transaction abort inside the event transaction; maintenance scheduled while tasks are active (pruning stays within its chunk budget and VACUUM defers); invalid request/result payload; worker exit zero; pending-request timeout; concurrent settings changes; restart during projection repair; old-schema upgrade; FTS unavailability; long-lived readers; and cancellation while saturated. Verify offline integrity/foreign-key checks on fixtures after recovery.

Run focused tests per changed domain, formatting/lint/type checks, and the ratchet, then build:electron, build:daemon, and build:cli. Daemon/CLI TypeScript configs currently include their entrypoint trees: workers referred to only by a runtime filename may not be emitted automatically. Add explicit build coverage and inspect actual output files. Qualify packaged Electron ASAR/native-module loading and Linux Node native ABI, plus shipped npm/desktop worker assets. Run repository release gates when preparing a release.

Rollout and rollback

During migration, legacy and worker connections may coexist only for explicitly partitioned operation groups; no transaction crosses the boundary. Shared-table readers need committed-write barriers. Maintain an exception register, seeded in DB1 from the ratchet counts, and shrink it each phase. Avoid routing two implementations as writers for the same logical operation.

Select backend mode at startup for a coherent domain. Compare read results against a controlled snapshot when useful; never shadow-write production mutations. Keep schema additions backward compatible and avoid destructive changes in this project. To roll back: stop admission, drain and reconcile, restart on the prior backend, and verify pending operations and schema compatibility. Never hot-switch after an uncertain write or blindly replay it.

Completion requires all application-thread SQLite access to be removed or covered by a documented, reviewed exception such as an offline maintenance utility. Update docs/architecture.md, docs/memory-fts-performance.md, docs/performance-stability.md, and runtime runbooks to match the implemented ownership and recovery contracts.

First implementation slice

Ship DB0's instrumentation as its own change, then the inventory and disposable benchmark. Land DB1's reductions as small independent changes measured against that baseline: the FTS fallback and exit-code fixes, bounded maintenance, the metric-retention and quota-scan fixes, the explicit synchronous level, and the ratchet. In parallel, build DB2's worker client with the pruning and FTS pilot commands, and prove commit-before-reply crash recovery and lock-wait isolation. Then move timeline event persistence (DB3) and use the measurements to size the remaining domain migrations.