HRC persistence, encrypted state and recovery#

Design 1, 31 October 2026. Product decisions · Protocol · Integration

This specification selects the first implementation's storage and encrypted-state boundaries. It does not claim a qualified database, cryptographic implementation or live conversation service. The accepted limits are a 30-day/10-GiB service history, seven-day/512-MiB local history, at least seven-day deduplication and at most 24-hour retry windows. New and recovered devices receive future encrypted messages by default. Account recovery uses a pre-enrolled owner-held signing key with no administrator override.

1. Storage engine and ownership#

Use one local SQLCipher database per HRC service and per device profile. Use SQLite WAL with synchronous=FULL, foreign keys enabled and one application-owned writer. Readers use bounded transactions on the same host. Do not put WAL databases on network filesystems or split an operation's atomic state across attached databases. Check every required setting after opening; an unknown SQLite PRAGMA can be silently ignored. SQLite WAL · Durability settings

The selected initial native baseline is SQLCipher Community 4.20.0, tag commit 355d4db92d3404e71faa8088d071a71d767807f4, with SQLite 3.54.1. Its release and tagged VERSION agree. This includes the upstream WAL-reset fix; old unpatched system SQLite is not an interchangeable dependency. Record and verify the built engine, source identity and enabled features in packaging. SQLCipher release · SQLite release

Use rusqlite 0.37 with its sqlcipher feature and the controlled native library. The inspected build script makes that feature override plain bundled; do not enable bundled-sqlcipher and accidentally select a different vendored engine. The build must resolve only the approved SQLCipher headers/library, then verify non-empty cipher_version, expected SQLite version/source identity, FTS5 and a real encrypted round trip. Wrong-key opening must fail without creating or replacing a database. Binding selection

Keys enter through a private descriptor or interactive secret input, not command arguments, environment variables, diagnostic SQL or logs. Client unlocking and unattended service/bot key provision have explicit local configuration. An account-recovery signing key is never reused as a database key or MLS key. Encrypt archive content, indexes, MLS state, drafts and approval records together. Require memory-only temporary storage in the build and connection configuration; page encryption alone does not protect every SQLite temporary file. SQLCipher storage design

The service stores community plaintext inside its encrypted-at-rest database, but stores only ciphertext and permitted metadata for encrypted rooms. It never places confidential message plaintext in a service search index. Database encryption does not prevent an authorised running endpoint from reading content or protect against a compromised host with its live key.

Feature owners retain their own policy. The persistence boundary owns transactions, schema conversion, bounded queries, durability and recovery errors; it does not independently decide delegation or room rights.

2. Atomic service operations#

One writer serialises room events and home-owned security controls. Each committed transaction advances a global commit counter and relevant room/policy revisions. Event identity and room order are independent of timestamps. Event, outcome/deduplication row, consumed approval/challenge and authoritative state change commit together.

Use explicit transactions after bounded admission. Acquire a fresh action-bound HyperNet check outside the writer critical section, then recheck its deadline and all current home state at the serial authorisation point. Retain the decision's evidence references with the outcome. Durability may complete after that point; a timeout never proves that a committed effect did not happen. Publication and Committed acknowledgement follow successful durable completion and the continuity guard described below.

An uncertain database result stops further effects until the writer can resolve it. Never roll back a published event by changing its outcome label. Operation.Resolve reads the durable record; it does not reenact the operation. A missing/closed retry window cannot be recreated from client timestamps. A terminal admitted rejection retains its immutable action digest and outcome for the deduplication period.

Keep the one-writer lock free of network calls, UI work, model calls and unbounded scans. Use a short bounded database busy timeout; return Overloaded before admission when the writer cannot accept work. An I/O stall after authorisation produces pending/unknown state. HRC pause/revoke must not acknowledge a boundary while an older unfenced commit could still finish after it.

3. Retention, quotas and indexed access#

The accepted service-history quota includes retained event bodies, attribution, search indexes and derived previews. The local-history quota includes retained plaintext/ciphertext archives, local indexes and previews. Persistent security state, deduplication and active cryptographic state have separate measured budgets; exhausting any budget rejects new work rather than discarding still-required outcomes or shortening agreed retention. Drafts and pending outbox records are never silently evicted to make room.

Expiry immediately excludes content from authorised delivery/history/search predicates. A bounded deletion transaction removes the expired event content, its index entries and previews together. Retain only the minimal outcome/identity metadata still required by the deduplication or audit contract. An accepted message's history expiry does not reopen its operation ID. Initial cleanup target remains 60 seconds, subject to qualification.

Use FTS5 for service-readable text and for authorised decrypted local archives only. Every query first restricts permitted room/history/sender scope and applies current recipient ignore filters. Search hit IDs are rechecked against current access before returning snippets or context. SQL parameters and a typed search grammar are mandatory; raw user input is not an FTS expression. Queries have result/byte limits, progress cancellation and a bounded transaction lifetime. Search failure does not alter message state. FTS5

Version the normalised search representation and its Unicode data separately from original content. A schema/index-version change rebuilds derived data from authorised retained content. It never rewrites message bytes, signed Actions or ciphertext. The text profile fixes Unicode17 normalisation and a case-sensitive trigram index over derived text, with an explicit bounded short-query path. The default FTS tokenizer's Unicode version is not silently treated as the editor's version.

A read transaction can prevent WAL checkpoint progress. The service must cancel readers at their documented deadline, perform bounded checkpoints and stop new admission at configured physical-storage/WAL ceilings. Logical content quotas are not promises that total disk use equals payload bytes. Measure encryption, index, deduplication and journal overhead in the pilot. Do not delete live WAL files or checkpoints to force quota recovery.

4. Continuity, backups and migration#

Maintain a private continuity guard outside the database snapshot set. It records service identity/recovery generation, global commit counter and the current commit-chain digest. Each transaction advances that digest from its previous value and a domain-separated deterministic encoding of the counter, operation/outcome digests and bounded authoritative change record. This is a constant-cost continuity record for bounded transactions, not a full database rescan or proof against an operator who can rewrite every trusted state source. For a mutation, commit the database durably, atomically replace and synchronise the guard, then publish/acknowledge. Finish guard persistence before starting the next mutating transaction, including maintenance. Therefore a supported crash can leave the database at most one transaction ahead. Recover that case only by validating the retained previous counter/digest and the exact next chain link, then advance the guard before service resumes. A database behind, further ahead or inconsistent with the guard fails closed.

The guard is not a separate copy of business state or a distributed consensus system. It prevents supported restore procedures from silently accepting an old database as current. It cannot detect a malicious administrator deliberately rolling back every local continuity source together. Do not claim otherwise. Loss of the guard or inability to prove continuity requires recovery handling, not an automatic reset to zero.

Back up through a consistent database snapshot API or a stopped, checkpointed service, never by copying only the main file of a live WAL database. Encrypt and protect snapshots like live content. Automatic backups are disabled until explicitly configured with destination, retention and disclosure policy; their retention must be visible separately from active conversation history. SQLite backup API

Restoring the same active service identity is supported only if the snapshot matches the retained guard or a proven complete subsequent log. An older snapshot may be opened in an isolated read-only recovery view with expiry/access restrictions, but it cannot resume effects under the old identity. If current state is lost, create an explicitly new HRC service identity and seek fresh admission/consent. Do not resurrect grants, retry windows or federation agreements from an old backup. This first profile does not provide replicated failover or arbitrary point-in-time active restoration.

Migrations are versioned and transactional where SQLite permits. Run them before service admission, with exclusive ownership and an authorised consistent backup. Preserve old outcomes and retry-window closure. If a migration cannot complete, keep the original usable state or refuse startup explicitly; never partially serve a mixed schema. Database/key corruption must not trigger automatic reinitialisation.

Device profiles apply the same continuity principle to crypto state, outbox and received cursors. An old device snapshot cannot resume sender ratchets. If current state cannot be established, retire the endpoint and enrol a new one under the accepted recovery policy, receiving future messages only.

5. Persistent MLS conversations#

Use OpenMLS 0.9.1 and the same initial ciphersuite as Suite's inspected mail implementation, MLS_128_DHKEMX25519_AES128GCM_SHA256_Ed25519. Each direct conversation or private group has a persistent MLS group with one leaf per authorised device/agent endpoint. Suite's per-letter fresh-group lifecycle is not the chat lifecycle. Use public membership handshakes, encrypted application messages and no external joins in the first profile. Suite Mail implementation

Compose the existing RustCrypto provider with upstream openmls_sqlite_storage 0.3.0. Its storage provider can borrow the connection underlying an HRC-owned transaction. Standard storage helpers do not start nested transactions. This allows MLS changes and HRC outbox/inbox/approval/cursor changes to commit together; no replacement implementation of the complete storage trait is needed. Run provider migrations separately before live operations. Its rusqlite bundled dependency makes the explicit SQLCipher linkage checks above essential. Provider · Manifest

OpenMLS processing can write receive state before later cryptographic validation, and Welcome processing can consume a package before every check succeeds. On cryptographic, credential or group-transition failure, roll back the MLS transaction, discard the mutated in-memory group and reload committed state. Application-content rejection after successful MLS authentication follows the separate durable outcome below. Do not rely on cloning MlsGroup; its clone support is not a production API. Message processing · Welcome processing

A home-committed application event can contain invalid UTF-8, excessive plaintext or another invalid application body even when its MLS authentication succeeds. Consume that receive generation and atomically retain a bounded RejectedDelivery record plus the home cursor, without archiving/indexing the invalid content or triggering notifications/agents. The record binds event ID/digest and a stable reason, not plaintext. Reconnect must not decrypt or reject that same event forever. Approval inspection uses the same distinction, but records a rejected candidate without advancing a room cursor; later matching delivery resolves that candidate outcome once.

For a home-authenticated committed application event whose MLS cryptography or sender binding fails, roll back its tentative MLS changes first, then separately persist the rejected-delivery outcome and cursor. This skips only that application event; it does not install its claimed ratchet, epoch or identity. Later authentic traffic may consume bounded skipped generations under the selected ratchet policy. If that cannot proceed within the limit, show resynchronisation required and use explicit recovery. An invalid membership commit is different: preserve a visible failed-transition position and suspend dependent epoch processing and sends; do not skip it into an unverified successor epoch. All rejection records and reports have quotas and coalescing limits.

Sender retries use the exact stored ciphertext and Action. They never restore earlier sender state or encrypt a retry again. A transaction rollback before exposure can safely discard its tentative outputs; once ciphertext leaves the device, its generation is consumed even if the service later rejects publication.

6. Epoch ordering, removal and failed transitions#

The home orders one public commit candidate from the expected epoch and checks exposed signatures, roster bindings and HRC policy. The author retains its pending commit until the home outcome is resolved, then merges that exact winner. Recipients stage, validate policy/cryptography, merge and durably acknowledge the exact epoch and commit digest. Distinguish home-ordered from verified and durable at this endpoint.

A service's public-group checks cannot validate the private confirmation value as a full member can. Do not claim that one server acknowledgement proves every endpoint accepted a commit. Invalid confirmation, credentials or membership at an endpoint freeze that endpoint's sends and trigger a bounded discrepancy report. A confirmed inconsistent group transition freezes new group sends while recovery is established. A malicious member or delivery service can still deny service. Public group validation · Member confirmation check

Membership removal, account/device recovery and withdrawal of an agent's read access establish an immediate home send barrier. Resume only after an eligible endpoint has durably verified the required removal transition and the home records that readiness for the exact new epoch. Other endpoints must verify that same transition before sending or displaying its messages. Readiness is an authorised endpoint statement, not a proof against colluding malicious members.

If no eligible member can complete the transition, remain unavailable for new encrypted sends. Recovery can explicitly initialise a fresh group with the current authorised membership and a new group identity; it never falls back to keys held by removed members. Preserve a visible discontinuity and old-history attribution. Direct conversations follow the same rules as groups.

For the ordered initial profile, retain no old epoch secrets after successful transition. Retained decrypted archives are separate content under their own accepted policy. Process prior committed application events before their successor commit. Offline endpoints catch up through the ordered chain within bounded work; if required cryptographic history is unavailable, explicit fresh admission gives future messages only. Sender-ratchet tolerance and forward-distance limits must be pinned and tested before enabling encrypted sends; neither unlimited catch-up nor silent message loss is acceptable.

7. Exact-content approval#

The owner must already be an authorised member able to decrypt the candidate. Consume the candidate once in that owner's normal transactional MLS state and save a private PendingApproval containing exact ciphertext/Action digests, verified plaintext and metadata, actor/grant, group, epoch and decision. Display this as a proposed message, not committed room history. Approve signs the exact immutable action; rejection does not rewind the sender or recipient ratchet.

When matching ciphertext later appears in home history, use the verified pending record instead of decrypting it again, then advance the room cursor in the delivery transaction. A duplicate preview returns the existing candidate result. A different digest under the same operation conflicts. Expiry, changed content or changed epoch requires a new candidate and approval.

Allow at most one outstanding encrypted candidate per sending endpoint and room. That sender cannot publish a later generation while the candidate is unresolved; process preceding committed traffic before preview. Bound abandoned-generation gaps within the configured ratchet limit and require an authorised epoch change or rejoin before the limit is exceeded. A rejection may consume a generation without creating a room event. Exact approval does not make a draft durable conversation history.

8. Account recovery and device continuity#

Account creation binds the initial authorised controller endpoints and an independently generated recovery public key. The owner keeps the recovery signing secret separately from ordinary device and MLS secrets. The account's initial identity descriptor and subsequent controller-authorised changes establish a verifiable continuity chain. Trusted contacts retain that anchor; a service assertion alone must not silently replace it. First-contact verification still depends on the explicit identity/enrolment trust model.

Recovery requires fresh HyperNet admission, possession of the new device key and a one-use home challenge signed by the currently enrolled recovery key. Bind account identity, current security generation, home service, replacement device/transport identity, nonce and expiry. The recovery credential permits this narrow transition, not arbitrary messages, decryption or agent delegation. An administrator cannot supply a replacement signature.

In one home-ordered transaction, advance the account security generation, replace lost-device authority, invalidate pending approvals and pause affected delegations. Reject queued old-generation effects. Emit a visible recovery/device-change event and require encrypted membership removals/additions before new messages. The new endpoint receives no historical MLS keys or archive import automatically. Losing all devices and the recovery key requires a new account, not an administrative alias to the previous identity.

Recovery-key rotation requires the currently enrolled recovery key or an authorised surviving controller under the explicit rotation operation, proof of the replacement key and a current account revision. Notify other enrolled devices and record the new key generation. Recovery is one-use for the challenged transition; rotate/reconfirm recovery material deliberately after use. Never export a recovery secret through an agent grant or ordinary diagnostic/export command.

Federated recovery cannot instantly acknowledge control on unreachable homes. Bind an owner's security generation in its home-owned delegation and in any source assertion for a remotely represented owner. The origin refuses new old-generation assertions; already-issued evidence retains only its original finite lifetime. Request remote pause/withdrawal barriers and show which homes have acknowledged them. A local recovery acknowledgement must not claim global completion. First-profile delegated actors and represented foreign accounts must use the agreed origin or the receiving home, without third-network transitive identity assertions.

9. Implementation evidence#

Qualify one SQLCipher transaction spanning real OpenMLS writes and HRC state, including failure injection before/after commit and guard persistence. Check wrong-key refusal, migration recovery, snapshot/floor mismatch, index expiry, quota rejection and bounded WAL growth. Inspect representative database, WAL and temporary output for planted plaintext markers without logging keys or private messages.

Exercise exact ciphertext retry, invalid cryptographic input rollback, authenticated invalid-text rejection with cursor progress across restart, one-use Welcome, competing and uncertain commits, preview/approval/rejection/duplicate delivery, old-generation recovery requests, lost devices, removal with no available member and explicit fresh-group recovery. Verify that new/recovered endpoints can read future messages while historical encrypted content remains unavailable. Reuse existing cryptographic vectors and meaningful boundary regressions; passing tests establish only the qualified profile.