Materialized Project Statistics Programme¶

Status and authority¶

This technical plan is the architecture gate for the complete pre-calculated/materialized statistics programme. It records intended architecture and staged delivery only. Approval authorizes completion of the detailed Phase 0 catalogue, mutation-ownership matrix and benchmark specification in a separate review; it does not authorize Phase 1 implementation, staging activation, production activation, data migration or legacy removal.

The programme replaces repeated full-project MongoDB aggregation with a rebuildable read projection while preserving the current authoritative Project and Study aggregates. It covers valid screening and annotation statistics at project, stage, membership/reviewer, question, and domain reconciliation scopes, together with durable per-project history.

The programme deliberately does not revive the broad agreement/kappa implementation from pull request

2534. That implementation can compare screening decisions originating in different stages. Any future¶

inter-rater agreement measure requires a separately reviewed same-stage cohort and denominator.

Decision summary¶

The following decisions are proposed for approval:

Project and Study remain authoritative. Statistics are a disposable, rebuildable projection and must not become a second source of domain truth.
One logical ProjectStatisticsProjection is physically partitioned into naturally keyed mutable current documents by metric family and scope. A small shared project summary is acceptable; one ever-growing payload document and copy-on-write generation trees are not the default.
Current and historical readers use the same catalogue keys, scope identifiers, authorization rules, provenance fields, and projection-revision model. Pages may adapt that contract to presentation DTOs, but may not create page-specific statistics stores or history formats.
Ordinary supported same-MongoDB mutations calculate deterministic signed bucket moves and commit the source change, targeted scoped statistics increments/decrements and an immutable unique delta/history record in one transaction. Asynchronous fenced processing is reserved for bulk, cross-system, non-additive or unsupported operations.
A requested statistics bundle is served against one project committed revision or entirely from the authoritative calculation. Unchanged Fresh scopes may have an older LastChangedRevision while still being valid through that project revision; mixed source snapshots and partly materialized responses are rejected.
History stores immutable, real observations at bounded checkpoints. It never invents pre-enable data, averages observations during compaction, or rewrites old snapshots after a repair.
Materialized serving is disabled by default. A global kill switch, family flags, consumer flags, project allowlist, and shadow parity mode make every cutover independently reversible.
Pull request #2985 supplies screening mechanics and tests, not the final storage or API shape. Its useful work is reconciled in Phase 2 after the shared Phase 1 contract exists.
SignalR and page migration begin only after the projection contract is approved and proven. SignalR publishes committed revision/scope invalidations; clients fetch an authorized snapshot rather than receiving a recalculated full aggregate on every Study event.
Production activation and legacy removal remain separate explicit user decisions after staging soak.
Bounded state-profile distributions are first-class statistics. Their stable profiles and formulas mirror approved live calculations; clients cannot request arbitrary dimension cross-products. Project profiles preserve valid combinations such as different reviewers recording both include and exclude decisions. Equivalent membership/reviewer profiles are returned only when authorized.
The March-April plans and M008-M011 are primary design evidence. The reconciliation ledger is authoritative whenever this plan summarizes a recovered decision.

Programme MVP and progressive enhancement¶

The minimum usable runtime slice is:

Phase 1 shared dark foundation;
Phase 2 project screening materialization; and
the first Phase 5 screening-only API/Project Overview consumer cutover.

That screening-only consumer must request only the Phase 2 family; it must not use the broad FullStats bundle, which would coherently fall back until annotation and membership families also exist. The slice must reduce authoritative aggregation load, preserve current externally visible results, provide truthful history, and remain instantly reversible. Annotation and question families, additional pages, enhanced history UI, automatic repair, and any statistically validated agreement measure remain ordered enhancements rather than being forced into the first runtime cutover.

Problem and current architecture¶

StudyStatsQuery.GetFullProjectStatsAsync currently derives a broad FullStats response from a MongoDB aggregation. The response contains project screening, stage annotation, membership screening, and membership-stage annotation data. It is invoked by REST controllers, synchronous Project DTO mapping, reviewer-specific queries, and SignalR subscriptions.

The present hot paths have four consequences:

full multi-facet aggregation is repeated on ordinary reads;
Project DTO mapping blocks on asynchronous I/O through .Result;
every qualifying Project or Study change can trigger another full SignalR aggregation; and
there is no shared, versioned history or reliable source provenance.

Current main also contains separate live ProjectReviewerStats/StageReviewerStats count queries and a manually refreshed annotation question-answer tally embedded in Project. Those are part of the Phase 0 catalogue; they must not be duplicated under new names.

An annotation unit belongs to an individual Study and reviewer annotation. No demonstrated cross-study aggregate, shared unit definition, consumer contract, or performance need exists, so unit materialization is outside this programme. Outcome-level materialization is likewise outside this programme: the shared scope key supports versioned question identity only. Either domain may return only through a future separately approved feature with a concrete authoritative aggregate and consumer.

Operational progress such as import jobs, export jobs, Bulk PDF delivery, or live SignalR presence is not project statistical history. It stays in its existing operational model unless a later independent feature explicitly reclassifies it.

Scope and authoritative metric catalogue¶

Catalogue rules¶

Every materialized metric requires the following catalogue fields before implementation:

stable metric key and human meaning;
aggregation grain and scope identifiers;
authoritative source fields and inclusion/exclusion filters;
exact numerator, denominator, tally or grouping equation;
configuration inputs and compatibility digest;
mutation authorities and invalidation triggers;
current API, page, SignalR, export, or report consumers;
authorization class, including own-row and peer-row rules;
whether it is durable, checkpoint-history eligible, derived at query time, or intentionally live; and
exact parity representation, including ordering and rounding rules.

Integer counters and tallies require exact equality. A later floating-point metric must define a tolerance and rounding rule in its catalogue entry; no implicit tolerance is allowed.

Catalogue baseline¶

Family	Scope	Current metrics or intended definition	Phase 0 disposition
Project screening	Project	Study count plus bounded distributions over approved screening-state profiles: included/excluded reviewer decisions, decision-count/status, sufficiency, started and overscreening states, including valid conflicting include/exclude combinations	Approved current authoritative family; Phase 2
Membership screening	Project + membership/reviewer	Per-reviewer study distributions over approved decision, availability, sufficiency and completion profiles; included, excluded, screened, available, unavailable and total remain compatible views	Approved current authoritative family; Phase 4 after project screening
Stage annotation	Project + stage	One stage-scoped payload with fixed included/unexcluded and excluded class buckets, each containing approved annotation-session profiles such as no session, incomplete/in-progress and completed; domain-reconciliation states remain separately identified	Approved current authoritative family; Phase 3
Membership-stage annotation	Project + membership + stage	One membership-stage payload with the same fixed inclusion-class buckets over approved available, unavailable, incomplete/in-progress and completed candidate-session profiles; reconciliation and capacity-derived values remain compatible views	Approved current authoritative family; Phase 3
Reviewer screening	Project + membership/reviewer	Screened, available, unavailable and total from the separate reviewer query	Catalogue against membership screening to remove aliases; Phase 4
Reviewer annotation	Project + membership + stage	Included/unexcluded and excluded in-progress/completed/available values, total and capacity status	Catalogue against membership-stage annotation; Phase ¾
Question answers	Project + stage + question/version	Existing distinct-study and annotation-answer tally, with active/suppressed/version semantics made explicit	Replace manual refresh only after catalogue approval; Phase 3
Outcome-level statistics	N/A	No approved authoritative cross-study formula or consumer	Explicitly outside this programme; a future feature requires separate approval and definition
Search/population	Project + systematic search	Approved live study-population/search counts and their contribution to project totals; pending import/deletion rows remain excluded exactly as authoritative queries define	Catalogue exact current formulas and consumers; Phase 2/4
Project/stage derived summaries	Catalogue-specific	Valid totals or presentation-ready summaries derived from approved scopes	Derive without duplicating source counters; Phase 4
Domain reconciliation	Project + stage + membership where applicable	Candidate/reconciliation session availability, started, in-progress and completed states	Included with annotation families; not statistics parity repair
Agreement/kappa	Same-stage cohort only	No approved formula	Excluded until a separate statistical specification is approved
Operational progress/presence	Job or live connection	Import/export/Bulk PDF progress and active connections	Intentionally outside this programme

Question scopes must carry the immutable definition/version identifiers required to interpret an old observation. Renaming or deleting a definition cannot silently change historical meaning.

Bounded study-state profile distributions¶

A state-profile distribution is a catalogue-defined set of meaningful current Study classifications, not a generic analytics cube. Each profile family specifies stable profile keys, exact formulas, source fields, stage/configuration inputs, exclusivity or permitted overlap, ordering and schema/source version. The dimensions must mirror an approved authoritative live calculation. Adding a dimension or changing a formula is a catalogue/schema change, not a query option.

The initial profile families are:

Project screening: counts of studies by meaningful combinations of reviewer include/exclude decisions and screening-decision number/status, plus approved sufficiency, started and overscreening classifications. A Study may validly have include and exclude decisions from different reviewers; those conflicting combinations are preserved and tallied rather than labelled impossible.
Membership/reviewer screening: equivalent counts from the authorized individual reviewer's perspective, retaining that reviewer's decision/status and the approved project context without leaking peer decisions.
Project/stage annotation: counts of studies by approved candidate-session state combinations, such as no session, incomplete/in-progress and completed. The stage-scoped payload contains the bounded, fixed included/unexcluded and excluded class buckets used by the current authoritative calculation; inclusion class is not a separate physical scope key or caller-selected dimension.
Membership/reviewer-stage annotation: equivalent authorized per-reviewer distributions over that reviewer's availability and incomplete/in-progress/completed session states, with both fixed inclusion classes in one membership-stage payload.

Profiles are omitted as impossible only when an enforced application invariant proves the state cannot exist. Observed combinations that are unusual, internally conflicting between reviewers, or merely absent from a fixture remain representable. Unknown combinations caused by schema drift make the family Incompatible or Stale and force authoritative fallback; they are never silently dropped into an other bucket. The catalogue may retain simple totals as named projections of these distributions, but the materialized contract must not collapse the distributions into totals alone.

Target architecture¶

Source-of-truth boundary¶

Project owns configuration, stages, memberships, permissions and question definitions. Study owns screening decisions, annotation sessions, domain reconciliation sessions and annotations. Statistics consume facts from those aggregates but cannot enforce or redefine their invariants.

For an ordinary supported mutation, the domain transition produces deterministic before/after classifications and versioned signed bucket moves. The MongoDB transaction commits the authoritative source mutation, targeted current-statistics increments/decrements and an immutable uniquely identified delta/history record together. The value object describes a domain transition; repository mapping owns the physical field paths.

The operation identity is allocated before entering the driver's retryable transaction callback. External effects such as SignalR, broker publication, email, object storage or remote API calls are prohibited inside that callback. Required revision/scope invalidations use the durable transactional outbox; other effects may run only after a confirmed commit.

Logical data model¶

flowchart LR
    P[Supported Project / Study mutation] -->|one transaction| S[Authoritative source]
    P -->|signed bucket moves| C[Scoped current statistics]
    P -->|same operation ID in every mode| I[Bounded source-operation receipt]
    P -->|same operation ID| D[Immutable delta / provenance record]
    P --> O[Durable transactional notification outbox]
    O -->|leased at-least-once dispatch after commit| N[SignalR or broker invalidation]
    B[Bulk / cross-system / unsupported operation] --> F[Durable affected-scope fence]
    F --> R[Bounded bulk deltas or authoritative scoped rebuild]
    C --> Q[Authorized current/history query]
    D --> H[Bounded checkpoint sets with scope references]
    Q -->|Fresh, compatible, one revision| M[Materialized response]
    Q -->|Missing, stale, rebuilding, incompatible, fenced or disabled| A[Authoritative response]

`ProjectStatisticsGlobalControl` and `ProjectStatisticsControl`¶

One durable global control document is the safety authority for write/serve mode, a monotonic WriteEpoch and a monotonic GlobalClientInvalidationRevision. Generated runtime flags request transitions but their per-process caches are not correctness authorities. Every statistics-aware source transaction reads this document inside the transaction; every materialized read includes it in the same snapshot as project control/scopes/fences. A missing, incompatible or non-Enabled global mode always means authoritative serving.

One small control document per project contains catalogue/storage/source versions, per-family lifecycle summaries, configuration digests, CommittedProjectionRevision, a monotonic SourceInvalidationRevision, a monotonic ClientInvalidationRevision, last reconciled revision, parity/rebuild failure metadata, applied WriteEpoch, delta replay/identity and source-operation idempotency floors and deletion state. Every source transaction that can change an authorized statistics response advances both SourceInvalidationRevision and ClientInvalidationRevision, even when materialized writes are disabled. Every supported point materialization transaction also advances its small projection commit clock atomically with source/receipt/current/delta writes but allocates only one ClientInvalidationRevision for the combined change. A projection-only repair/rebuild publication advances CommittedProjectionRevision and ClientInvalidationRevision without changing SourceInvalidationRevision. This deliberately accepts one scalar shared write for the expected reviewer population; it does not contain or rewrite metric payload or history, and the concurrency benchmark determines whether this clock needs a measured alternative.

`ProjectStatisticsSummary` and `ProjectStatisticsScope`¶

A small project summary may hold approved project-wide totals and bounded profile buckets; it is also family/token-addressed when it participates in staged publication. Other current payloads are mutable, naturally partitioned scope documents addressed by:

(ProjectId, MetricFamily, ScopeKey, SchemaVersion, PublicationGeneration, PublicationToken).

ScopeKey is a canonical typed tuple in a fixed family-specific order: project/stage, membership/reviewer, membership + stage, stage + question definition/version, or systematic search. A generic untyped scope ID is prohibited. The family publication guard's VisibleGeneration and VisibleToken first select, without considering schema/source compatibility, the exact published row at the visible generation whose token equals VisibleToken, when that row exists; otherwise they select the greatest published row below the visible generation for an unchanged scope. Only after that newest eligible row or tombstone has been selected may the reader validate its versions and digest. An incompatible selected row makes the complete requested bundle fall back authoritatively; a reader must never descend to an older compatible row and thereby hide an incompatible newer publication. Candidate generation numbers are never skipped: the only candidate is VisibleGeneration + 1, and an abandoned candidate must be resumed or have every row and marker with its token purged before that number can be reused with another token. The atomic guard flip is the publication fact, so a row below the visible generation could only have come from an earlier successful flip; an unpublished token can never become eligible merely because a later generation was published. Thus a candidate may replace only affected scopes while unchanged scopes continue to resolve to their older published rows; no complete payload copy or parent-chain traversal is required. The unique/index design must support this bounded per-scope descending lookup and prohibit two rows for the same scope/generation/token.

A removed natural scope is represented by an explicit tombstone in the selected row for an ordinary transaction, or at the deletion publication generation for staged work; absence never means deletion because it would allow lookup to fall through to an older row. A tombstone is a terminal selected result (normally mapped to not-found/omitted by the authorized query contract), is included in publication manifests and cleanup reachability, and is retained until the catalogue and retention floors prove that no supported current or historical request can resolve through it. In steady state, ordinary point transactions read the guard and selected row in-transaction and update that row. While a family is Fenced, point mutations commit source plus ordered deltas for candidate catch-up but cannot expose a generation above VisibleGeneration. Catch-up must include every typed scope named by those deltas, not only scopes in the operation's initial source snapshot. For a newly affected scope, it copies the selected published base row into the candidate generation; if none exists, it uses the catalogue's canonical empty state immediately before a proven creating delta or an authoritative scope snapshot at a captured source/replay boundary. It applies only moves after that base boundary and idempotently adds the row/tombstone and full scope identity to the bounded candidate manifest under the stable delta + scope identity. If the boundary is unprovable or the addition cannot satisfy manifest or document admission, the whole publication aborts/restarts; the guard cannot flip around the scope. Each document carries LastChangedRevision, AppliedWriteEpoch, configuration/content digest and schema/source versions. LastChangedRevision advances only when that scope changes; it is not required to equal the project CommittedProjectionRevision.

Targeted MongoDB $inc/$dec operations or equivalent signed bucket moves update only affected fields. A tally transition decrements the exact old key and increments the exact new key. A small shared project summary is acceptable for the expected small reviewer population; ½/5/10-reviewer benchmarks must measure write conflicts, transaction retries and p50/p95. Striped counters or further splitting are permitted only when those measurements prove a hotspot.

Every physical current document has a hard dynamic-key and encoded-BSON admission ceiling below MongoDB's 16 MiB limit; Phase 0 sets a measured limit no greater than 8 MiB and 50,000 dynamic keys. Fixed catalogue profile buckets remain in the small summary, but the sparse ScreeningGroupedCount / screeningTallyCounts[(NumberScreened, IncludedCount)] distribution is explicitly data-dependent. A transaction that would introduce a new tally key first guards stored key/byte accounting and a conservative encoded-size upper bound. If the result would cross either ceiling, it does not attempt the MongoDB update: it atomically marks the screening family Stale with typed StatisticsScopeCapacityExceeded and, in the same transaction, marks Stale or fences every other family the event matrix lists as dependent on that screening transition (the project, stage and membership-stage annotation populations at the inclusion boundary) unless those families' moves are independently bounded and are applied, so no dependent Fresh row survives a source-only screening commit; it then commits the source through the source-only receipt/invalidation path, and makes that project ineligible for materialized screening serving until a separately approved bounded representation or a measured lower-cardinality state permits rebuild. Backfill/rebuild calculates exact candidate BSON size before publication and follows the same no-publication fallback. Existing-key updates still require the resulting encoded-size guard. No family-wide storage budget substitutes for this per-document admission rule.

`ProjectStatisticsSourceOperationReceipt`¶

Every statistics-affecting source mutation, including a source-only fallback mutation, atomically inserts one minimal immutable receipt keyed by project plus stable namespaced OperationId. The operation envelope binds a server-issued immutable OperationCreatedAt and expected source-before revision outside the retryable callback; broker redelivery persists that envelope and an API caller cannot replace its age. The envelope's expected source-before revision is a staleness-floor admission guard, not a per-attempt equality bind: a lost write conflict inside the same delivery reloads the newer source, recalculates the deterministic transition and commits under the same immutable envelope, admitted only while the reloaded source revision is at or above the envelope's expected revision and above the idempotency floor. A catalogue mutation with explicit compare-and-set semantics instead treats a lost conflict as terminal: the caller receives the typed conflict and must issue a new operation/envelope rather than rebasing. The receipt records the envelope identity and its expected revision plus the actual committed source before/after revisions, source/content digest, operation kind, observed time and resulting source/client invalidation revisions, so redelivery deduplicates against the receipt even when the commit legitimately rebased past the envelope's expected revision. A materialized Delta references the same identity but is not the sole deduplication authority. Unknown-commit recovery and redelivery first read this receipt: an identical receipt is a successful no-op and conflicting content fails closed. Receipt equivalence excludes the server mint timestamp: independently minted envelopes for the same stable operation, expected revision, operation kind and submitted-content digest are duplicates. The original receipt's creation/observation times remain immutable retention metadata, and envelope age and source-revision admission floors still apply before any new source transaction. Screening and annotation saves bind the actual submitted content as well as configuration before receipt lookup; matching configuration alone is insufficient. Unordered submitted sets and object keys use canonical ordering, while ordered answer arrays retain order. Submission entity creation timestamps are observational metadata and are excluded; scientific time-point values remain content. A bulk or import operation that commits source work in multiple transactions inserts one receipt per source transaction, not one for the parent operation: every forward and inverse source child batch commits its own receipt atomically in that batch's transaction, keyed in a distinct child namespace by the parent OperationId plus direction and stable child batch ID, recording that batch's own source before/after revisions and digest. The parent operation envelope still governs admission and operation-level deduplication, but per-batch retry and unknown-commit recovery read exactly the matching child receipt, so a retried later batch can neither collide with an earlier batch's receipt nor reapply an authoritative mutation that lacks its own all-mode receipt.

Receipts are retained through the measured maximum transaction-retry, broker-redelivery and audit windows, then reclaimed only after advancing a durable SourceOperationIdempotencyFloorObservedAt. Envelopes at or below that floor are rejected as terminal-too-old rather than reapplied, and the guarded source-before revision is a second protection after receipt reclamation. Those windows bound only single-transaction receipts. Every child receipt of a staged bulk or import operation whose coordinator is non-terminal (Preparing, SourceComplete, AllFamiliesAdmitted or Aborting) is pinned: the reclamation watermark excludes it and the idempotency floor never advances past the oldest pinned receipt's envelope, however long the operation has been active, because those receipts are the per-batch retry authority and the BulkSourceIncomplete abort watermark. Pins are released only by the coordinator's terminal transition, after which the ordinary windows apply. The receipt collection has independent measured row/byte ceilings; if safe reclamation cannot keep it below them, an idempotency-requiring source mutation fails with typed StatisticsIdempotencyCapacityExceeded before changing source data. Reclamation never discards a pinned receipt to relieve that pressure; a long-running staged operation whose pinned receipts alone breach the ordinary admission ceiling is aborted through the ownership-checked protocol rather than having its receipts reclaimed. That abort must itself be able to write receipts, because every inverse deletion/restoration batch inserts its own namespaced child receipt while the forward receipts stay pinned throughout Aborting. Two rules keep that path open. First, a staged operation reserves its receipt capacity before loading any source row: the coordinator's admission records a bounded maximum forward batch count derived from the bounded manifest capacity and reserves receipt rows/bytes for that many forward child receipts plus an initial equal number of inverse child receipts; if the reservation cannot be admitted, the operation fails typed before any hidden row exists, and the reservation is released only by the coordinator's terminal transition. That initial inverse allowance is a floor, not a bound: an inverse batch that restores before-images (a bulk update, or any future separately approved update-in-place import) can need more inverse transactions than forward transactions when the before-images are larger than the after-images, because the before-images, not the after-images, determine the inverse BSON size. Each forward child transaction therefore computes the worst-case inverse batch count for the exact before-images it is about to replace (using the same point-path size limits) and atomically extends the operation's inverse receipt reservation before it commits — by max(worstCaseInverseCount − 1, 0) rows plus the bytes beyond the preallocated share, because admission already reserved one inverse receipt for this forward batch, so the common one-forward/one-inverse case extends by nothing rather than double-counting; a forward batch whose inverse extension cannot be admitted fails typed without committing, and the operation aborts with every already-committed batch's inverse capacity already reserved. The invariant is that reserved inverse capacity always covers the worst-case rollback of every committed forward batch, so an Aborting operation can never be refused an inverse receipt by the ordinary ceiling. Second, the receipt ceiling carries a fixed terminal-cleanup reserve above the ordinary admission ceiling that ordinary point receipts and new coordinator reservations can never consume: inverse child receipts, PublicationAbort records and terminal coordinator transitions may draw on it, so a ceiling breach by ordinary traffic closes new admission while every already-admitted abort still completes. Receipt capacity tests must prove that an import aborted at the ordinary ceiling completes its full inverse prefix and reaches Aborted. The write/serve kill switches disable projection work, not these correctness receipts or the required notification outbox.

`ProjectStatisticsRevisionRecord`¶

An immutable revision record is uniquely keyed by project and a stable namespaced record ID across every schema/source version. A point Delta uses its OperationId; a bulk child delta uses a deterministic operation + batch identity; and each ReplacementPublication uses a deterministic PublicationId = (OperationId, MetricFamily). Thus one all-family transaction inserts one distinct, retry-stable record per family without weakening project-wide operation deduplication. Abort kinds use their operation ID in a separate fixed namespace. Schema/source versions are validated record content, not part of the uniqueness boundary. Every record stores the assigned CommittedProjectionRevision and is ordered and indexed by (ProjectId, CommittedProjectionRevision, RecordId). Replay cursors contain both revision and RecordId; all records sharing a revision are a bounded sibling group, and a range is complete only after its cursor passes the final stable RecordId in that group. A Delta kind records source aggregate identity and before/after revision, affected typed scopes, canonical replayable signed moves, versions, trigger/reason, observed/calculated time and redelivery provenance. A ReplacementPublication kind records one affected family, candidate publication generation/token, a bounded immutable manifest ID/reference, scope/tombstone counts, complete manifest digest, authoritative source watermark, versions and trigger/provenance. It never embeds the affected scope set. Immutable manifest pages contain that set in bounded rows written and verified before publication; the revision record has no synthetic signed moves and is an explicit replay barrier. A rebuild whose replay range crosses that barrier must restart from at least the replacement source boundary. A bounded PublicationAbort kind records the operation, fixed affected-family set, capacity or other typed reason, source watermark, candidate manifest digests and cleanup proof; it asserts no materialized publication and is never treated as a replacement barrier. A bounded SourceImportAbort kind additionally records the hidden import token, source rollback-manifest digest/count and verified zero-row boundary; it records removal of an unexposed prefix rather than inventing a user-visible history point.

Source/source-operation-receipt/current delta writes and their Delta record commit atomically. A bulk/rebuild guard flip assigns a distinct new CommittedProjectionRevision and commits its ReplacementPublication record, project control, family guard and notification outbox atomically. A digest-only Delta row is prohibited while its revision is above the durable replay floor.

A duplicate key with the same revisions, versions and digest is a successful no-op. A duplicate operation ID with different revisions, versions or content fails closed, marks affected scopes Stale and produces parity-investigation evidence. Revision records are the projection replay/deduplication authority and fine-grained historical/provenance ledger; source-operation receipts are the source-mutation idempotency authority in every mode.

`ProjectStatisticsCheckpointSet`, reference pages and observations¶

User-queryable history consists of immutable sparse checkpoint-set roots keyed by project and a CheckpointId = (CheckpointSourceRevision, CheckpointProjectionRevision, CheckpointModeEpoch), the physical three-component key capturing SourceInvalidationRevision, CommittedProjectionRevision and the fleet-wide mode epoch recorded on the singleton control row; the two revision components are the display form, and every uniqueness, CAS and ordering rule below operates on the complete three-component key. The source component prevents collisions during source-only fallback; the projection component permits a repair, rebuild or schema publication at an unchanged authoritative source revision to append a truthful checkpoint; the mode-epoch component means a stage-one mode transition changes every enabled project's checkpoint identity at once, so the daily producer treats it as an advance and an authoritative build under the new formulas gets a distinct ID rather than colliding with the old-epoch checkpoint and failing closed on its digest. Each fixed-size root records page count, total reference count and a Merkle root over the ordered reference-page content digests for the complete catalogue of metric/family/typed-scope observations in that logical checkpoint. It also stores the selected immutable BuildToken, which unambiguously addresses its pages. Immutable reference pages are keyed by checkpoint build token and ordinal. References have one deterministic total order by (MetricFamily, MetricKey, ScopeType, canonical ScopeTuple, ObservationId); pages record first/last full reference key, reference count, content digest and a bounded Merkle inclusion proof to the root. They are capped initially at 500 references and 256 KiB each (Phase 0 capacity tests may lower either bound), and reference immutable observation rows by occurrence identity. Every immutable observation document carries the same encoded-BSON admission ceiling as current scope documents (no greater than 8 MiB); its encoded bytes count against the build's reserved byte budget before any artifact is written, and an authoritative result whose scope payload cannot be admitted — including a scope that is Stale precisely because it exceeded the current-document ceiling — records typed CheckpointCapacityExceeded unavailability for that build instead of attempting a MongoDB insert that would fail at the BSON limit. ObservationId includes metric/family/typed scope identity, the first-observed three-component checkpoint identity (source revision, projection revision, mode epoch) and the content digest, so an identical payload recalculated under a new mode epoch is a distinct immutable observation with its own provenance. A compound build-token/full-reference-key-range index locates only pages that can contain requested metric-scope pairs, including deterministic handling of equal typed scopes in different families. The root never contains a growing list of page IDs, so no checkpoint document can approach the MongoDB BSON limit. Publication also has an initial hard ceiling of 256 pages/64 MiB per checkpoint; Phase 0 must replace it with a measured equal-or-lower limit before production. Exceeding it records typed CheckpointCapacityExceeded unavailability and leaves current serving unaffected. A changed scope creates a new observation even when its value returns to content seen before (A→B→A has two distinct A occurrences). An unchanged scope may reference its immediately preceding immutable observation only when an uninterrupted durable scope-change proof covers every component of the checkpoint identity from that occurrence through the new checkpoint boundary: every source revision, every projection record (projection-only repair or schema publication) and every compatibility or mode-epoch transition. A mode-epoch change is never covered by such a proof, so a mode-dependent scope always receives a new observation under the new epoch even when its payload is identical. A minimal source-operation receipt, a compacted range without scope evidence, or any source-only interval does not prove unchangedness. When the proof is absent, an authoritative build creates a new observation at the captured boundary even if its content digest equals the prior value; this records a real later observation without claiming which intermediate values occurred. The root records the checkpoint source revision/time, projection revision, ClientInvalidationRevision, delta/source watermark, trigger and compatibility metadata, while every observation retains its own first-observed source/projection revision, time and versions. Repeating a checkpoint at the same CheckpointId is an idempotent no-op only when the complete identity and digest match; a different digest at that exact pair fails closed.

New observations and pages are written as immutable unpublished rows under one build token. Their source must be either one authoritative snapshot or one materialized snapshot. An authoritative build captures the source revision and every authoritative visibility boundary used by its query, including the bounded hidden-import/operation identity where applicable. It may publish while projection writes are disabled or the affected projection scopes are Stale, Rebuilding or fenced: publication compare-and-swaps the captured global WriteEpoch, source revision and authoritative visibility boundaries, and rejects only a concurrent change to those captured inputs. It does not require projection Freshness or a zero-fence guard on projection-side publication guards. Source-visibility fences are never waived: ActiveInclusionInfoCalculationJob holds one stable token across its three Study update passes, and the definition-rewrite token holds one across a whole-Project annotation rewrite, so an unchanged token is no evidence of a coherent source, and an authoritative build must verify that no source-visibility fence of either kind is active for any affected family both at snapshot capture and again inside the publication compare-and-swap. An active token rejects the checkpoint build with the same typed StatisticsInclusionRecalculationInProgress unavailability that current authoritative bundles return, so a checkpoint can never permanently publish the partially rewritten population those bundles refuse to serve.

A materialized build instead requires compatible global/project mode and epoch, every included scope to be Fresh, and every relevant publication guard to report zero active fences. The publisher verifies every ordinal, count, byte bound, reference and full manifest digest, then atomically inserts the small root with its BuildToken and compare-and-swap against the captured global WriteEpoch, project committed/source watermark and bounded family publication-guard GuardVersion plus visible generation/token pairs. On this path, any stale/rebuilding/incompatible scope, active fence or changed guard rejects publication. Both paths preserve the captured source/projection provenance in the checkpoint identity and root. The root is the sole publication point: pages without a root are invisible and garbage-collected after a bounded build lease. Publication verifies the complete ordered page set. A scoped query reads only the full-reference-key-range pages it needs, verifies each page content digest and inclusion proof against the root, and fails typed-unavailable on any missing/duplicate range, bad proof or unresolved requested scope. It never rehashes unrelated pages. This avoids an unbounded multi-document transaction while preserving atomic visibility of a complete logical bundle. Checkpoint publication never mutates current statistics and never fabricates a past value.

`ProjectStatisticsCheckpointBuild`¶

Every unpublished checkpoint build has one bounded durable marker keyed by project + BuildToken and a unique active-checkpoint identity for its intended CheckpointId. It records lifecycle (Building, Published, Abandoned, Cleaning or Cleaned), random BuildToken, lease owner, monotonic lease generation and expiry, captured source/projection/visibility boundaries, expected/actual page/reference/ byte counts and digest, and a restartable cleanup cursor. A worker may write or publish only while a CAS proves its current unexpired lease generation. It renews in bounded intervals; after expiry, a successor may CAS-increment the lease generation and resume the same token, or mark it Abandoned and clean it before a different token can build the same CheckpointId. An expired owner can neither renew, publish nor delete under its old generation.

A singleton checkpoint-build admission row per project permits exactly one unpublished build token across all CheckpointIds and reserves that build's bounded page/reference/byte budget before any artifact is written. A request for a newer CheckpointId while that slot is occupied coalesces only a bounded latest requested identity; it does not allocate a second token or write pages. An expired build is resumed under the same token or moved through Abandoned/Cleaning until a digest-verified cleanup proves zero unpublished artifacts, after which one CAS releases the slot and admits the coalesced successor. Successful root publication releases the slot the same way, because its artifacts have become published-history storage: the root-publication transaction's admission-row CAS hands the slot directly to the recorded coalesced successor, allocating its token and reserving its budget, and only releases the slot empty when no successor is recorded. That handoff is atomic with the successor's existence: the same transaction inserts the successor's ProjectStatisticsCheckpointBuild marker in Building with its initial lease owner, generation and expiry, so a crash immediately after commit leaves an ordinary expired-lease build that any worker can resume or abandon under the marker protocol. An admission row whose active token has no marker is therefore an invariant violation, not a recovery state; the ownership-checked recovery transition treats it exactly like an expired build — CAS the row's token to Abandoned, run cleanup for that token (which proves zero artifacts) and release or hand off the slot — rather than waiting for a marker that will never appear. The admission row is never cleared or deleted while it carries a coalesced identity, so a checkpoint requested during a build is admitted by that build's own success or cleanup and never depends on an external caller requesting it again. The admission row, marker and partial unique active-token index are defence in depth: crash recovery cannot accumulate one 64 MiB candidate for each successive CheckpointId.

The root-publication transaction verifies the complete build marker and CASes it from Building to Published while inserting the root; a root and marker can therefore never disagree about which BuildToken won. Reference pages and newly created observations carry the creator BuildToken. An observation from another build is reusable only when already reachable from a published root, so cleanup never races a reference to unpublished content. For an expired build with no root, cleanup first CASes the marker to Abandoned/Cleaning, then walks its pages and creator-token observations in bounded idempotent batches, rechecking that no root, active lease, published checkpoint or snapshot-reader reference makes them reachable before deletion. It finally marks the build Cleaned; crash recovery resumes from its cursor and digest. Building markers are never TTL-deleted. Terminal markers are retained through audit/redelivery/ snapshot floors and reclaimed under measured per-project row/byte ceilings; capacity pressure rejects a new checkpoint build with typed unavailability and never affects current serving or deletes a live build.

Global and project `ProjectStatisticsNotificationOutbox`¶

A singleton global notification slot carries the monotonic GlobalClientInvalidationRevision from ProjectStatisticsGlobalControl. A global write/serve-mode transition updates that control and slot in one bounded transaction and the dispatcher broadcasts the revision to every statistics subscription; it never enumerates or updates project controls/outboxes. Every subscription observes this global identity, and every current response carries the observed GlobalClientInvalidationRevision so reconnect/refetch cannot retain data from an earlier global serving epoch. Delivery is leased, at least once and monotonically coalesced like the project slots. Clients deduplicate it by (StatisticsGlobal, GlobalClientInvalidationRevision). Failure to admit the singleton slot aborts the global transition before its durable mode changes.

Every source commit that can change an authorized current response advances the project's independent SourceInvalidationRevision and ClientInvalidationRevision and upserts a durable, bounded project + metric-family notification slot in the same MongoDB transaction as the source write, whether the projection path publishes a delta or falls back to source-only operation. A projection-only repair, rebuild or schema publication advances ClientInvalidationRevision and the affected slots in its atomic publication transaction without advancing SourceInvalidationRevision. The fixed catalogue also reserves one project-wide visibility slot; every family subscription observes it. A source-visibility transition such as staged-import reveal advances that slot so one atomic event invalidates all affected family clients even when their individual materialized publications are still fenced or later stall. The slot monotonically coalesces to the greatest target ClientInvalidationRevision and a family-wide invalidation; it never contains statistics payloads or an expanding scope/revision list. A restartable dispatcher claims a slot with a lease, publishes at least once after commit, and compare-and-sets DeliveredThroughClientInvalidationRevision only for the target it actually sent. If a concurrent transaction advances the target, the slot remains pending and is sent again. Clients discard revisions they have already observed and refetch through the authorized query contract. Notifications and current responses carry ClientInvalidationRevision, SourceInvalidationRevision and CommittedProjectionRevision; clients deduplicate a family-slot invalidation by (ProjectId, MetricFamily, ClientInvalidationRevision) and a project-wide visibility invalidation by (ProjectId, ProjectVisibility, ClientInvalidationRevision). They must not discard a second affected family merely because another family came from the same source/projection commit. A consumer that refetches a complete subscribed bundle still observes every subscribed family slot; the project-wide slot is the single extra identity only for catalogue-defined project visibility transitions and does not replace ordinary family subscriptions. SourceInvalidationRevision and CommittedProjectionRevision remain source and projection/history provenance. Attempts, failures and bounded delivery receipts are retained for a measured deduplication/audit window.

The number and byte size of live slots is bounded by the approved metric-family catalogue per project, and measured backlog-age/byte ceilings are enforced. Crossing a ceiling atomically stops materialized writes for that family and marks it non-servable, but does not suppress the fixed-size coalescing notification slot: source writes still advance SourceInvalidationRevision and ClientInvalidationRevision plus the slot so clients refetch authoritatively until dispatch and reconciliation recover. A slot update failure aborts the source transaction after SignalR cutover; silently committing an unnotified source change is prohibited. This is a repository-owned durable MongoDB outbox; an in-memory MassTransit outbox or an unrecorded post-commit callback does not satisfy the contract.

`ProjectStatisticsRebuildLease`¶

Leases are separate documents keyed by project + metric family + canonical scope key. Each records its token, monotonic lease generation, owner, expiry and captured projection/source watermark. Scopes in different metric families can rebuild and publish concurrently without overwriting a single control field. Within one family, the family guard's sole candidate/operation owner serializes candidate publication: another scope rebuild must wait or fail typed-busy unless it joins the same explicitly coordinated candidate operation.

`ProjectStatisticsPublicationGuard`, `ProjectStatisticsPublicationOperation` and¶

ProjectStatisticsPublicationMarker

A small guard per project and bounded metric family records independent GuardVersion, VisibleGeneration, VisibleToken, the sole candidate generation/token, lifecycle, active-fence count and last published revision. While fenced it also records the retry-stable ActiveOperationId and operation kind that owns the family admission. GuardVersion is the monotonic CAS counter: every fence, candidate, lifecycle, ownership or visibility change advances it. VisibleGeneration advances only in a successful publication flip. At most one candidate publication per family may be active, and its generation is always VisibleGeneration + 1. An abandoned token must be resumed or completely purged before a replacement candidate can reuse that number. This gives checkpoint and serving queries a constant-size CAS authority over many scopes without confusing fence activity with publication. A missing, changing or non-Fresh guard always falls back authoritatively. The initial bounded transaction compare-and-sets every affected family from no owner (or the same operation on retry) to the same operation ID before any batched scope fence or source write. A retry with the same ID resumes; a different operation cannot overlap those families and fails typed-busy rather than adopting or stranding another operation's fence. That initial admission transaction also inserts or verifies one bounded coordinator keyed by project + operation ID. The coordinator records operation kind, fixed ordered affected-family set, lifecycle (Preparing, SourceComplete, AllFamiliesAdmitted, Completed, Aborting or Aborted, in that protocol order), captured project revisions and one captured guard version/marker identity per affected family. SourceComplete is recorded by an ownership-checked transaction after the last forward child receipt and seals the complete child-batch and manifest digest and revokes the loader-lease generation in the same compare-and-set, so no forward batch can commit after the sealed boundary. For a staged import, only a coordinator in SourceComplete may select the post-source PublicationAbort path, and recovery of an import coordinator that has not sealed source completion always treats the load as incomplete and takes the SourceImportAbortPending rollback path, so a crash between the last source batch and publication can neither expose a partial import nor roll back a fully loaded one by guesswork. A non-import bulk update or deletion is different because its committed batches are authoritative and are never rolled back: it may enter PublicationAbort from Preparing with typed reason BulkSourceIncomplete, recording the completed forward-batch boundary as its partial source watermark as defined under bulk and import operations, as well as from SourceComplete or AllFamiliesAdmitted for post-source causes. The family catalogue bounds this record; it never contains scope, Study or manifest-page lists. AllFamiliesAdmitted -> SourceComplete is the sole non-terminal reverse transition (never back to Preparing, which would re-admit forward batches): the same owner uses it after a final-publication CAS loses to a supported concurrent change, atomically discarding the stale capture before catch-up and re-admission. Every other owner or lifecycle mismatch fails closed.

A marker per project + bulk operation + family records the candidate publication generation/token, captured/included revision, bounded immutable manifest ID, expected scope/tombstone/page and completed batch counts, rolling manifest digest and phase. Immutable manifest pages are keyed by marker/build token and bounded by the same page-count/byte-ceiling pattern as checkpoint reference pages. Candidate scope rows are written in bounded batches at that generation and are invisible until the guard advances VisibleGeneration. Marker updates are idempotent by stable batch ID. A single-family final publication touches only its marker, coordinator, family guard, project control, bounded ReplacementPublication reference and notification outbox; a multi-family final publication touches the same bounded record set once per affected family. It never embeds a manifest or rewrites all scopes or fences. Recovery reads the coordinator first, then verifies exactly the named family markers/guards; it cannot infer operation-wide admission from a subset of family rows.

`ProjectStatisticsOperationFence`¶

A durable operation fence is keyed by project + operation ID + metric family + canonical scope key. It records the operation kind, affected scope, lifecycle state, restartable batch progress, creation/expiry times and the source snapshot boundary required for publication. Active fences are indexed by scope and are independent of the control document so a point mutation cannot accidentally clear a bulk, import, configuration or deletion fence. Creating, changing or clearing a fence also advances the corresponding family publication guard.

Lifecycle¶

Each metric family/scope follows:

Missing/Incompatible -> Stale -> Rebuilding -> Fresh.

A supported source mutation commits source, signed scoped changes and immutable delta history atomically and advances CommittedProjectionRevision. There is no ordinary Pending interval.
Every mutation authority must either update each affected scope or mark it Stale/fenced in that same transaction before advancing the commit revision. Unaffected Fresh scopes remain valid through the new commit revision even though their LastChangedRevision is older.
A bulk, cross-system, non-additive, unbounded configuration or otherwise unsupported mutation creates an operation fence and marks only affected scopes Stale before source changes can make old data unsafe.
Rebuilding data always uses authoritative reads.
A rebuild publishes only when its lease, source snapshot boundary and captured committed-delta watermark still match.
A newer committed delta is replayed only from a complete, ordered, gap-free (CommittedProjectionRevision, RecordId) range whose canonical signed moves remain above the replay floor. Every bounded page resumes after that full cursor and consumes every bounded sibling record at a shared revision before declaring the revision complete; otherwise compare-and-swap publication fails and the rebuild restarts from a newer source boundary.
No visible scope can become servable while an active operation fence covers it, even if a supported point event is committed during that operation. A fully verified candidate row may be stamped Fresh in a bounded pre-publication batch after its own fence is cleared, but the family guard remains Fenced; only the later atomic guard flip can make it servable.
Failure returns the scope to Stale and preserves current/history for diagnosis.
A stale or incompatible scope is never served merely because its values look plausible.

History and provenance¶

Current revisions versus historical checkpoints¶

The project CommittedProjectionRevision advances atomically with each supported mutation. Its immutable delta record preserves the exact transition and provenance. User-visible history uses explicit checkpoint-set roots, bounded reference pages and immutable scope observations so API payload and storage growth remain bounded and timelines remain meaningful. A checkpoint set is a real observation of the complete catalogued bundle at its revision: reusing an older observation reference asserts that the scope did not change, not that a new past payload was synthesized.

The initial policy is:

retain each changed daily observation for 90 days;
retain one real changed observation per week through one year;
retain one real month-end observation through five years;
retain at most 256 history checkpoints per project unless measured payload growth requires a lower bound; and
keep current scopes independent from the history cap.

Compaction selects an existing observation. It does not average, interpolate, merge incompatible schema versions, or synthesize a value for a day on which nothing was observed.

Ordinary history has a named producer; the build and retention machinery above does not create observations on its own. The daily observation producer is a Project Management scheduled consumer on the existing Quartz-driven schedule, running once per UTC day, with a bounded catch-up window when a run is missed. For each project whose materialized statistics are enabled and whose checkpoint identity (SourceInvalidationRevision, CommittedProjectionRevision, mode epoch) has advanced in any component since its last published checkpoint root — source-only periods, when projection writes are disabled or capacity pressure routes mutations through the source-only path, advance only the source component and still change authorized statistics — it requests one daily-observed checkpoint through the singleton admission row described under the lifecycle section, so coalescing, the successor handoff and the build marker protocol apply unchanged and a request that arrives during a build is satisfied by that build. The request is idempotent on the key (project, observation day): a request whose day already has a published root at or above all three components of the current checkpoint identity (source revision, projection revision, mode epoch) is a no-op, and an admission failure or transient build failure is retried with backoff until the day window closes. A day on which the project did not change, or on which every retry failed, has no observation and is reported as a gap by the history API rather than filled from a neighbouring day. The producer never reads authoritative sources itself and never bypasses the fences that make a checkpoint build refuse to capture or publish. It is part of the Phase 1 dark foundation and runs only for allowlisted projects; durable user history is not delivered for a project until this producer has been observed publishing for it. Bootstrap, repair/rebuild and the explicit administrator request are the only other creation points.

Delta and checkpoint storage reclamation¶

Checkpoint retention selects surviving real checkpoint-set roots before deletion. Referenced scope observations remain reachable while any retained root uses them; unreferenced observations are then eligible for collection. Unpublished page/observation builds are lease-bound and collected separately. A root removed by retention enters the same bounded cleanup marker used for unpublished builds: after the reader grace boundary the marker deletes that root's reference pages in restartable, snapshot-safe batches, and only once the marker proves zero pages are the observations they referenced re-evaluated for collection.

Fine-grained revision rows and Delta signed moves may be compacted only after the retained-checkpoint, reconciliation, rebuild-lease, outbox and maximum-redelivery watermarks have all passed them, and only past every active staged operation's pinned cursor: each non-terminal publication-operation coordinator, family publication marker, per-scope operation fence and source-import rollback/abort operation contributes its captured/included replay boundary to the reclamation watermarks until it reaches its terminal state, because candidate catch-up must replay every delta after that captured boundary. Advancing the replay floor past a pinned cursor would force a typed terminal-too-old failure and abort or strand otherwise recoverable staged work, so the floors never pass it; ceiling pressure from pinned deltas takes the fail-closed path below rather than unpinning. Phase 1 starts with a 90-day/100,000-row exact-ledger ceiling per project and must replace those provisional values with measured limits before any production activation. Safe compaction advances durable DeltaReplayFloorRevision and DeltaIdentityFloorObservedAt values and replaces eligible contiguous revision ranges—including replacement barriers no surviving rebuild/checkpoint needs—with bounded, mergeable range seals containing revision/time bounds, kind counts and a digest; at most 64 seals are retained per project by merging adjacent seals. Individual operation IDs below the floor are not retained indefinitely. A projection replay/delta request older than either delta floor is never applied: it returns a typed terminal-too-old result, marks/fences the affected projection scope and requires reconciliation. Source-mutation redelivery is governed independently by the source-operation receipt and its floor.

If a row/age ceiling is reached before every safety watermark permits compaction, materialized writes stop and affected reads fall back to authoritative calculation; source writes may continue without creating detailed Delta rows only while their independent minimal source-operation receipt and required notification slot can commit. Source-operation receipts compact under their separate redelivery/audit floor and ceiling; exhaustion fails the source mutation before its authoritative write rather than weakening idempotency. Cleanup is restartable and rechecks all watermarks before deletion. Telemetry records retained checkpoint roots/pages/observations, exact and sealed delta rows and bytes by family, plus live/eligible source-operation receipt rows, bytes, floor pressure and rejected writes.

Bootstrap and rebuild truthfulness¶

Bootstrap records exactly one backfill-observed checkpoint at the time the authoritative calculation was made. It includes source high-water mark, definitions/configuration digests, versions, scope and initiating operation. It does not create points for earlier dates.

A rebuild records its captured and published source high-water marks. A parity repair retains the failed audit evidence and publishes a new current revision/checkpoint; it does not rewrite the mismatching historical record.

History survives Stale and Rebuilding state transitions. It remains queryable only through the normal current authorization boundary.

Definition changes and deletion¶

Question history preserves immutable definition/version identifiers and display metadata needed for interpretation.
Removing a membership prevents the former member from reading current or historical project data. Stored reviewer rows are filtered using current authorization.
Question deletion removes that scope from current state but does not mutate old history. Historical checkpoint references retain immutable project ownership and typed scope identity, so a currently authorized project reader can request a deleted question-version or systematic-search scope without requiring that scope to remain in the live Project graph.
Project deletion immediately makes current/history APIs unavailable, tombstones the control record and schedules projection/history purge under the approved project-retention policy. This programme does not create an independent right to retain a deleted project.

Event and invalidation contract¶

The complete implementation matrix is a Phase 0 deliverable and a gate for each family. The baseline matrix is:

Source event or operation	Affected families	Consistency strategy	Historical reason
Screening submit, correction or rescreen	Project and membership/reviewer screening; after Phase 3, every annotation project/stage/membership-stage profile whose authoritative population changes at the included/excluded boundary	Deterministic before/after classification; source + targeted signed bucket moves + unique delta record in one transaction; move dependent annotation population/profile buckets or fence all affected annotation scopes when the transition is not bounded	`screening-submitted` / `screening-corrected`
Screening decision deletion/reset	Project and membership/reviewer screening; the same dependent annotation families	Synchronous inverse/negative bucket moves, including annotation population reclassification, when bounded; otherwise fence and rebuild every dependent scope	`screening-reset`
Candidate annotation session start/save/complete/delete	Stage and membership-stage annotation profiles; approved question blocks	Synchronous signed transitions for catalogue-supported state changes	`annotation-session-*`
Durable slot-reservation allocate, release, disconnect-timeout or expiry	Membership-stage annotation availability/capacity and affected annotation-state profiles	Synchronous signed transition in the reservation transaction where a live formula depends on durable reservation state; presence alone stays operational	`annotation-reservation-*`
Effective active-reviewer tracking mode change: any transition of `FeatureFlags.ActiveReviewerTrackingAvailable` (`ActiveReviewerTrackingEnabled && SignalRActive`), whether from GitOps values or a runtime flag override	Reviewer annotation and membership-stage annotation availability/capacity profiles — every scope whose formula evaluates `SufficientlyAllocated` with the effective mode, as `GetInvestigatorAnnotationStats` does today	The effective mode is a fleet-wide compatibility input, not a per-project setting: the singleton control row records the current effective mode and a monotonic mode epoch, and every affected scope's configuration digest includes that epoch, so a row published under the old mode is `Incompatible` and falls back rather than serving as Fresh. The primary path is a durable transition owned by the Project Management consumer of the runtime feature-flag snapshot revision, and it is deliberately two-stage so that it never grows with fleet size. Stage one is one bounded transaction touching only the singleton control row: it advances the mode epoch, records the transition token, and advances the existing global invalidation slot that every family subscription observes. Because every affected scope's digest carries the mode epoch, all of them are `Incompatible` from that commit onward and fall back coherently; nothing else has to happen for correctness. Stage two is a restartable batch job under the transition token that walks enabled projects in bounded transactions, marks each project's affected scopes Stale or fenced, advances that project's ClientInvalidationRevision and affected-family slots, and schedules reclassification from retained reservation state and approved tallies where the new mode can be evaluated from them, otherwise rebuild. Stage two is progress bookkeeping and rebuild scheduling; a crash leaves the remaining projects `Incompatible`, not Fresh. A replica that observes the old flag after stage one commits cannot publish because its digest no longer matches the control row's epoch. Authoritative reads and authoritative checkpoint builds are pinned to the same durable mode: from the phase in which this row is live, the authoritative calculation evaluates `SufficientlyAllocated` with the mode captured from the singleton control row in the request's snapshot, not the process's `_featureFlags.ActiveReviewerTrackingAvailable` as `GetInvestigatorAnnotationStats` does today, and a process whose runtime flag disagrees with that durable mode fails closed with the typed recalculation-in-progress 503 rather than serving or recording the old formula under the new epoch. The same rule binds the source-write paths that branch on the flag today (`TryAtomicAssignStudyCoreAsync`, `SaveWithCapacityGuardAsync` and `SaveScreeningWithCapacityGuardAsync` in `StudyRepository`): from the same phase they evaluate reservation and capacity semantics against the durable mode read in the write's own snapshot and fail closed on disagreement, so a replica with a stale cached flag cannot assign or save Studies under superseded capacity semantics that a rebuild could not repair.	`active-reviewer-mode-changed`
Domain reconciliation start/save/complete/delete	Stage and membership-stage reconciliation profiles; approved question blocks	Synchronous signed transition for supported states; reconciliation is not parity repair	`domain-reconciliation-*`
Annotation content or suppression change	Question and affected annotation-state profile scopes	Stable definition/version identity; synchronous delta when bounded, otherwise fence/rebuild	`annotation-content-changed`
Screening threshold or agreement setting change	Screening families and every annotation family whose authoritative included/excluded population or inherited `SessionCountTarget` changes	`ActiveInclusionInfoCalculationJob` is a source-visibility fence: its start transaction fences every dependent family and invalidates clients; while its matching token is active, affected materialized and authoritative bundles return typed `StatisticsInclusionRecalculationInProgress`/503, and authoritative checkpoint builds reject both snapshot capture and publication, because the three Study update passes can expose a partial population. Only after all passes durably complete may a matching-token worker capture a coherent boundary and rebuild/publish. If materialized writes are disabled, verified source completion may instead clear the matching job, leave every family Stale and invalidate once, after which coherent authoritative fallback resumes. Failure/redelivery resumes the same idempotent job; timeout or a mismatched token never clears the fence.	`screening-configuration-changed` / `inclusion-info-recalculated`
Any statistics-affecting stage configuration change: `SessionCountTarget`, `SelfReconciliation`, selected questions/versions, filters, `MaxInProgress`, `HideExcludedStudiesFromReviewers`, `ExcludedSessionStatsGrouping`, `ReviewMode` or active state	Annotation and affected screening/derived families, including membership-stage availability/profile scopes	Reclassify from approved raw tallies when possible; otherwise configuration fence/rebuild every dependent scope; digest mismatch is a second fail-safe, not the primary invalidation path	`stage-configuration-changed`
Membership add/update/remove, excluding graph-permission edits	Membership/reviewer families and authorized response shape	Bounded membership-scope delta/invalidation in source transaction; fence only non-additive effects; current authorization always re-evaluated; advance the project-wide visibility slot when access is granted or revoked	`membership-changed`
Project/stage graph-permission change, including `UpdateProjectPermission` or `UpdateStagePermission`	Authorized current/history response shape across every screening, annotation, question, search and derived family the permission can expose	Re-evaluate current authorization without rewriting statistics values; in the source transaction advance ClientInvalidationRevision and the project-wide visibility slot observed by every family subscription (and any narrower affected-family slots required by the catalogue)	`statistics-authorization-changed`
Question create, version, reorder, detach or delete	Annotation/question families	Apply known session-reset bucket moves; retain old history identity. A change that rewrites Study annotations across the Project (today question deletion runs `RemoveAnnotationsFromStudiesInProjectForQuestionAsync`, a non-transactional `UpdateManyAsync`, and only afterwards saves the Project definition) is a source-visibility fence with the same semantics as the inclusion recalculation: its start transaction records a durable definition-rewrite token on the Project control row and fences every affected annotation/question scope before the first Study write; while the token is active, affected materialized and authoritative bundles return the typed recalculation-in-progress 503 and authoritative checkpoint builds reject both snapshot capture and publication, because the rewrite can expose a partially rewritten population under a still-live definition; the Project definition save that completes the change clears the matching token in the same transaction, after which Stale scopes rebuild or coherent authoritative fallback resumes. Failure or redelivery resumes the same idempotent rewrite under the same token; timeout or a mismatched token never clears the fence. Phase 3 reorders the current owner to fence first, rewrite second, save-and-release third.	`definition-changed` / `definition-rewrite-completed`
Study import or single Study creation	All study-population-dependent families	Single create uses transactional signed deltas; imports use two-phase staged invisibility plus bounded bulk delta or fenced rebuild	`studies-imported`
Terminal source-import failure before reveal	Every family fenced by the import	Keep hidden/admission tokens, roll back the exact hidden prefix from a paged manifest with stable inverse child IDs, verify zero rows/candidates, then atomically mark families Stale and release gates	`source-import-aborted` (provenance only; no invented checkpoint)
Post-source publication abort after complete source loading (`PublicationAbort`: capacity, incompatible mode/epoch, write-disable or retry exhaustion after `SourceComplete`)	Every family the operation fenced	Ownership-checked terminal protocol: purge candidates, mark families Stale, clear `ActiveHiddenImportJobId` and `ActiveImportOperationId`, advance `SourceInvalidationRevision` and the project-wide visibility slot in the same bounded transaction, so the committed source is revealed only through coherent authoritative fallback and no client cache can retain the pre-reveal population	`source-import-published-by-fallback` (provenance only; no invented checkpoint)
Bulk Study update or deletion	Catalogue-derived affected scopes	Two-phase operation with stable operation/child IDs, per-batch signed deltas and an Active fence until final reconciliation	`studies-bulk-changed`
Systematic search create/import/rollback/delete	Search, population, screening and annotation families	Natural search scope; one Project-level hidden-import token for atomic authoritative visibility, pending deletion, signed bulk deltas where bounded, otherwise fenced rebuild	`search-studies-changed`
Project deletion	All	Tombstone and deny reads, then policy-driven purge	No new user-visible checkpoint
Manual legacy question-tally refresh	Question family	Superseded after transactional materializer cutover; before cutover it remains authoritative	`legacy-question-tally-refresh`
Retry, broker redelivery or unknown commit result	Operation's original scopes	Reuse deterministic operation ID; verify source-operation receipt, source revision/digest and applicable delta before retry	No duplicate checkpoint

Exports, reports, DTO mapping, page reads and SignalR are consumers and do not mutate statistics.

Concurrency, retries, and rebuilds¶

Source ordering and measured contention¶

An ordinary transaction updates the source Study or Project, the bounded source-operation receipt, the small set of naturally affected statistics documents and the immutable delta record. The source before/after revision and stable operation ID define ordering; the receipt remains the idempotency authority in every write mode. Transactions that lose a write conflict reload the source and retry the deterministic transition under the same immutable operation envelope, subject to its staleness-floor admission guard; they do not apply a previously calculated delta to a newer source revision, and a compare-and-set-semantic mutation fails terminally with a new envelope required instead of rebasing.

Expected concurrency is a small set of authorized project reviewers, so generalized contention is not a STOP condition. Phase 0 and the first screening implementation benchmark simultaneous writes by 1, 2, 5 and 10 reviewers to both the same Study and different Studies. Measurements include p50/p95 transaction duration, retry/unknown-commit count, write-conflict rate, summary-document bytes and authoritative fallback frequency. A small shared summary remains the default until those results justify splitting or striped counters.

Transaction and idempotency rules¶

Allocate the operation ID outside the MongoDB driver's retryable callback and reuse it for redelivery.
Before any authoritative source write, canonicalize the exact affected scopes and signed moves and admit the ordinary point path only when it has at most 500 moves, touches at most 100 projection documents and produces at most 256 KiB of canonical delta-record BSON. Phase 0 transaction/BSON tests may lower these provisional limits. Exceeding any limit is not an authoritative-save error: acquire the bounded all-family operation ownership/fences first, then commit the source under that operation and finish through the staged publication/rebuild protocol. No oversized embedded Delta record or unbounded point transaction is attempted.
Inside the callback, read or guard the exact source revision, calculate deterministic before/after classifications, insert or verify the unique source-operation receipt, write the source, apply signed moves and insert the unique delta record.
A duplicate operation is a successful no-op only when its receipt matches the redelivered envelope's identity, expected source-before revision, source schema and content digest; the receipt's actual committed revisions may legitimately exceed the envelope's expected revision after a rebased retry. A conflicting duplicate fails closed and triggers a parity investigation.
An operation older than the durable source-operation idempotency floor is never executed even when its receipt and individual delta row have been compacted; it fails closed to reconciliation instead of risking a second apply.
Transient errors may rerun the entire callback. The callback contains no SignalR send, broker publish, email, object-storage mutation, remote API call or other irreversible external side effect.
Resolve an unknown commit by reading the unique source-operation receipt, associated delta when applicable, and authoritative source revision before deciding whether to retry.
A write conflict reloads and reclassifies; it never blindly reapplies an old delta.
SignalR/broker invalidations are inserted into the durable transactional notification outbox and carry the monotonic client-invalidation revision plus source and projection revisions. Clients discard stale or duplicate notifications by the family-aware or project-wide ClientInvalidationRevision identity defined above and refetch authoritatively when the projection revision has not advanced. The dispatcher is never invoked inside the retryable callback. Other non-critical external work may begin only after a confirmed commit but cannot be the sole delivery path for required invalidations.

Bulk and import operations¶

Multi-document import, bulk update and deletion use a durable staged-publication protocol:

allocate one operation/event identity outside retryable callbacks, then in one bounded transaction compare-and-set every affected family publication guard to Fenced with that retry-stable ActiveOperationId/kind. No scope fence or source write begins before this all-family ownership admission commits. A retry owned by the same ID resumes; a different owner receives typed-busy. Then persist an Active operation fence for every affected scope in bounded idempotent batches;
mark the exact affected metric scopes Stale; the family guard makes the whole affected family fall back while per-scope fence batches are being created or cleared;
commit source batches with restartable progress, stable child IDs and per-batch signed deltas in the same transaction when the transition is additive and bounded. Each child batch's transaction also inserts that batch's own namespaced source-operation receipt (parent operation ID plus direction and stable child batch ID) as defined above; the parent receipt is never the per-batch idempotency authority, and a child batch retry or unknown commit is resolved by reading exactly its child receipt and batch delta. Every child batch transaction additionally compare-and-sets a lease generation on the operation coordinator inside that same transaction — a snapshot read of the coordinator without this guarded write is not a fence. The two directions run under two distinct leases with distinct generation fields. A forward batch runs under the loader lease and asserts an unexpired loader generation and a lifecycle of exactly Preparing, the only loading lifecycle, so a stale loader cannot commit source rows after an abort owner has revoked its lease or after the SourceComplete seal has revoked the loader generation. An inverse (rollback) batch runs under the abort-owner lease and asserts an unexpired abort generation and a lifecycle of exactly Aborting; it is rejected in any other lifecycle, including Preparing and AllFamiliesAdmitted, because rollback before the abort decision would race the loader. The transition into Aborting revokes the loader lease and allocates the first abort-owner generation in the same compare-and-set, so no transaction can hold a valid lease of both kinds at once, and abort-owner renewal, expiry and takeover follow the same ownership-checked protocol as the loader lease;
for staged imports, first acquire Project.ActiveImportOperationId, write every Study with the stable PendingImportJobId while Project carries matching ActiveHiddenImportJobId, and have authoritative queries exclude only that hidden token. Never clear visibility per Study to reveal a batch. Update-in-place imports that replace an existing visible Study are not supported by this protocol and are outside the programme's scope: hidden staging of replacement versions, per-identity generation selection across disjoint imports and fencing of concurrent point edits to superseded Studies must all be designed and separately approved before any such import may use staged publication; until then a staged import creates only new Studies, and the inverse of a forward batch is deletion of the rows it created. For deletion, keep PendingDeletion semantics and apply inverse batches;
for non-additive or unsupported batches, calculate an authoritative result from a snapshot-isolated source read tied to a committed-delta boundary, replay later deltas or reject/restart when that boundary cannot be proven;
create one publication marker for every affected family at exactly its VisibleGeneration + 1, then write all immutable bounded manifest pages and candidate scope rows/tombstones in bounded, idempotent batches under their tokens; update completed batch/page counts, scope/tombstone counts, digests and included revisions without making candidates visible. Every candidate-artifact transaction (manifest page, candidate row, tombstone, count or digest update) compare-and-sets a revocable publisher generation on the coordinator inside that same transaction; the operation-wide admission in step 9, the final guard flip in step 10 and the transition into Aborting each revoke the publisher generation before verifying or exposing candidates, so an in-flight batch can neither commit after the zero-candidate proof nor alter an admitted manifest;
replay every complete ordered concurrent delta range into all affected candidates. A delta affecting a scope absent from the initial candidate set must clone that scope's selected published base. When no base exists, catch-up may initialize the catalogue's canonical empty scope immediately before the first creating delta only when the delta carries the stable scope-creation fact; otherwise it builds that scope from an authoritative snapshot at a proven source/replay boundary. It then incorporates only moves after the chosen boundary and adds the resulting row/tombstone plus manifest reference in a bounded idempotent batch. An unprovable boundary or capacity failure aborts/restarts the all-family publication. Repeat until every marker's included revision can compare-and-swap against the same project control revision;
while all family guards remain Fenced, verify and stamp every candidate row/tombstone Fresh, clear its per-scope fence, and record ready/fence counts and digests in bounded idempotent batches. The rows are still unservable because their family guards are fenced. A concurrent mutation advances a captured GuardVersion/control and invalidates the operation-wide admission, forcing catch-up and re-verification;
compare-and-set the bounded publication-operation coordinator from SourceComplete (the sealed state every bulk/import operation reaches after its last forward receipt) to AllFamiliesAdmitted only after every affected-family manifest is below its page/byte ceiling, complete, digest-valid and ready against one captured project revision plus all guard versions. Store those captured values and marker digests on the coordinator. No family may publish before this operation-wide admission succeeds; and
in one bounded transaction, compare-and-swap every captured guard/control value, allocate one new CommittedProjectionRevision, insert one bounded ReplacementPublication record per affected family under its deterministic operation + family PublicationId and referencing (not embedding) its manifest, point all affected guards to their exact candidate generation/token as Fresh, coalesce every family invalidation, compare-and-set the coordinator to Completed, clear every matching family ActiveOperationId, and clear its Project admission lock. For an import the same transaction clears ActiveHiddenImportJobId, advances SourceInvalidationRevision — the reveal changes every authoritative response that includes the imported Studies, so the source clock must move with it even though no further source row is written — advances ClientInvalidationRevision atomically (hidden rows changed no authorized response while loading, so without this advance the coalesced slot target could still equal the already-delivered revision and subscribed clients would stay on the pre-import population), and only then coalesces the family and project-wide visibility slots. The affected-family catalogue is fixed and bounded, so this transaction is constant-size with respect to scope/study count. It is all-or-nothing: a crash after admission but before commit resumes from the durable coordinator and leaves every candidate invisible; after commit it resumes only bounded cleanup. An abandoned candidate blocks a successor until every family candidate is resumed or all rows, tombstones and manifest pages for its token are proven and purged. If the final CAS loses to a supported concurrent mutation while every family is still owned by this operation, one bounded transaction compare-and-sets the coordinator from AllFamiliesAdmitted back to SourceComplete (never to Preparing, so no loader can be renewed), clears its obsolete captured revisions/guard versions/digests, allocates a fresh publisher generation so the catch-up candidate batches of steps 7–9 can CAS it (the generation revoked at admission stays revoked, so a batch from the lost attempt is still rejected), and advances its coordinator version. The guards remain fenced and candidates remain invisible while steps 7–9 catch up and re-admit. An incompatible durable mode/epoch or exhaustion of a measured retry bound takes the terminal post-source PublicationAbort path defined below instead of leaving an admitted coordinator stranded; an ownership mismatch fails closed for investigation and must not clear another owner's state.

An import that fails before source loading completes follows a separate restartable SourceImportAbortPending path; a terminal SearchImportJob error never strands the Project gates. The abort owner compare-and-sets its publication-operation coordinator to Aborting before cleanup; another owner cannot adopt it. That same transition revokes the loader lease by advancing its generation, so any still-running forward batch loses its guarded lease compare-and-set and cannot commit further hidden rows or manifest pages after the transition. The zero-row/zero-candidate verification runs only after the revocation has durably committed, so a stale loader cannot invalidate the proof between the scan and the terminal transaction that clears ActiveHiddenImportJobId. The loader maintains a bounded paged manifest of every newly inserted hidden Study identity (and a before image for any separately approved update-in-place import). Manifest capacity is reserved operation-wide, not merely per page: import admission reserves the import's total rollback-manifest row/byte budget under the per-project ceiling before any source batch commits, and every source batch first verifies that its identities/before-images fit the remaining reservation, committing the covering manifest rows atomically with, or durably before, the hidden source rows they cover. A batch that would exceed the reservation fails before any of its hidden source rows commit and enters the restartable SourceImportAbortPending path with a manifest that completely covers everything committed so far, so the manifest never grows past its reservation and no committed hidden row is ever absent from the abort proof. Abort keeps both Project tokens active, deletes those Studies or restores those before-images in bounded idempotent transactions — each inverse batch committing its own namespaced inverse child source-operation receipt atomically, exactly as forward batches do — emits stable inverse child records for any committed candidate/delta work, and purges all candidate publication artifacts. After a snapshot verifies zero source rows for the hidden token, zero live candidates and digest-complete terminal fences, one bounded transaction allocates and advances one new distinct CommittedProjectionRevision on project control — never reusing the current revision, because appending a late RecordId to an already-committed revision's sibling group after a replay or compaction cursor has passed that revision's final record would let the abort provenance be skipped permanently — inserts the immutable SourceImportAbort provenance at that revision, marks affected families Stale/non-servable, and clears every affected family guard whose ActiveOperationId matches the aborted operation with the same verified candidate-metadata reset the post-source PublicationAbort path requires — verify the matching candidate generation/token and marker identity, then clear that owner's candidate generation/token/marker, candidate lifecycle, ActiveOperationId and active-fence count, leave the guard Stale and advance GuardVersion — because candidate work may have begun before source loading failed, and a guard left with no owner must not retain an occupied sole-candidate slot or stale generation metadata for a later rebuild or import to trip over. The same transaction clears ActiveHiddenImportJobId and ActiveImportOperationId and advances the family plus project-wide visibility notifications while compare-and-setting the publication-operation coordinator from Aborting to Aborted. The guard-owner/candidate-metadata clears, Project-token clears and coordinator transition are one compare-and-set transaction; an owner mismatch aborts terminal publication for investigation rather than releasing another operation's guard. Because no partial source is revealed, clients use the unchanged authoritative dataset while reconciliation/backfill restores materialization. Crash or redelivery resumes from the abort manifest and stable child IDs; it cannot convert the job back to parsing, expose a prefix, or block the next import indefinitely.

Manifest page and byte limits are admission limits, not an indefinitely retryable state. Capacity exhaustion, an incompatible durable mode/epoch, a global or family write-disable transition, and terminal operational/CAS retry exhaustion after authoritative source completion (the sealed SourceComplete state) all enter the same typed, restartable post-source PublicationAbort protocol; the reason distinguishes PublicationCapacityExceeded from the other terminal causes. A non-import bulk update or deletion that reaches one of those terminal causes before its source loading completes enters the same ownership-checked protocol with typed reason BulkSourceIncomplete: no further source child batches may run under the operation, and the abort records the completed forward-batch boundary — proven by the per-batch child source-operation receipts — as its partial source watermark. Already-committed child batches remain authoritative committed changes served through coherent fallback; the abort purges the operation's candidates and manifest pages, moves its per-scope fences to terminal Aborted, marks the affected families Stale, and releases the family owners and operation locks through the same final compare-and-set transaction, so recovery or rebuild proceeds from the actual partial source state rather than leaving a permanently typed-busy coordinator. Crash or redelivery resumes the abort and cannot resume source loading. Only a staged import with incomplete source loading instead uses SourceImportAbortPending, because its hidden prefix must be removed rather than adopted. This safety cleanup remains permitted while materialized writes and serving are disabled. Operation-wide admission prevents any guard flip or ReplacementPublication before all family limits and final predicates pass. The matching owner first compare-and-sets its coordinator from Preparing, SourceComplete or AllFamiliesAdmitted to Aborting; the worker then purges every candidate row/tombstone/manifest page for the operation and changes all of its per-scope fences to terminal Aborted in bounded, digest-verified batches while the family guards stay fenced.

One final bounded transaction verifies those cleanup postconditions, inserts the immutable bounded PublicationAbort record with a distinct CommittedProjectionRevision, source watermark and typed reason, marks every affected family Stale/non-servable, coalesces their invalidations, compare-and-sets the publication-operation coordinator from Aborting to Aborted, and releases the overall operation/admission lock. For each guard it must verify the matching ActiveOperationId, candidate generation/token and marker identity, then clear that owner's candidate generation/token/marker and candidate lifecycle, clear its matching ActiveOperationId and active-fence count, leave the guard Stale, and advance GuardVersion. Coordinator, matching candidate/owner reset and overall-lock release are one compare-and-set transaction; any mismatch fails closed for investigation rather than clearing a successor's candidate or ownership. For a staged import whose source loading completed, that transaction also clears the matching ActiveHiddenImportJobId, advances ClientInvalidationRevision atomically for the same reason as the successful reveal, advances SourceInvalidationRevision (the reveal changes authoritative responses exactly as the completed-import reveal does, so a bundle captured before the abort can never advertise the post-reveal population under the same source revision) and advances the project-wide visibility notification, deliberately revealing the already committed source only through coherent authoritative fallback. An import whose source loading did not complete uses SourceImportAbortPending to remove the still-hidden prefix instead. A crash, callback retry or unknown commit resumes from the durable coordinator/abort identity and re-verifies the terminal state; it cannot duplicate provenance, leave a permanent typed-busy guard or report a materialized publication. Later rebuild is admitted only after the abort has released all matching guards and locks.

Only one import operation/admission lock and one hidden import gate may be active per project. Project stores both ActiveImportOperationId and ActiveHiddenImportJobId. The single all-family publication transaction clears both only while flipping every affected guard and advancing all family plus project-wide visibility notifications, so imported Studies and their materialized families become visible together or none do. A later import cannot enter candidate replay or change visibility out of order. Study-level PendingImportJobId values are provenance/cleanup markers after that flip and may be cleared in bounded batches without changing visibility. Authoritative query code reads the Project gate and Study source in one snapshot and excludes exactly the active hidden token; a missing/incompatible gate fails closed. Only the fenced import operation's internal candidate calculation may explicitly include its own hidden token, so candidates contain the post-import state before the atomic visibility flip; no user/export query can request that override.

A supported point mutation may commit while the bulk operation runs, but its transaction must observe the Active fence and cannot mark or expose that scope as Fresh. An abandoned or expired operation leaves the scope Stale and requires an explicit resumed operation or rebuild; timeout never clears the fence into Fresh. A failure after the fence therefore causes extra authoritative reads but cannot serve a partial materialized result.

Current-row reachability reclamation¶

After a successful generation flip, unchanged scopes may legitimately remain on their latest older published generation. For each scope, exactly the row/tombstone at VisibleGeneration whose token equals VisibleToken, when present, or otherwise the greatest published row below VisibleGeneration, is selected without compatibility filtering; rows referenced by the sole active candidate, rebuild/checkpoint build or lease are also temporarily reachable. Deletion tombstones stop older-row lookup and remain selected until catalogue/history retention floors permit their removal. Compatibility is evaluated only after selection, so an incompatible newest selected row forces authoritative fallback and is not skipped in favour of older payload. Unpublished candidate rows are never selected; a failed candidate is purged before its generation number can be reused. A restartable cleanup marker walks scopes in bounded batches and deletes only older non-selected rows after the MongoDB oldest-active-snapshot/read-transaction grace boundary has passed and all candidate, lease, manifest and build references are rechecked. Rollback never selects an old current generation; it uses the authoritative query, so unreachable payload rows are not retained as rollback state.

Published replacement-manifest pages and completed markers have their own restartable retention cleanup. They become eligible only after the referenced replacement revision/barrier is below every replay, idempotency, checkpoint, rebuild, audit and redelivery floor, no current/candidate/checkpoint build references them, and the snapshot-reader grace boundary has passed. Cleanup verifies the manifest root, page count and digest, deletes pages in bounded idempotent batches, then removes the root/marker. Orphaned and aborted manifests follow the same proof with their build/abort lease. Per-family page/byte ceilings stop further materialized publications and force authoritative fallback if safe cleanup cannot keep up.

Completed/aborted publication-operation coordinators have a restartable retention cleanup and remain until every named family marker/manifest, operation fence, source-operation receipt, revision/audit and redelivery reference is below its safety floor. Cleanup re-verifies the terminal state and all references before deleting the bounded coordinator; a Preparing, SourceComplete, AllFamiliesAdmitted or Aborting coordinator is never age/TTL-deleted because it is the recovery authority for that operation. Phase 0 sets measured per-project terminal-coordinator row/byte ceilings. Admission of a new staged operation first proves room for its coordinator; if safe cleanup cannot keep below the ceiling, it returns typed StatisticsPublicationOperationCapacityExceeded before acquiring family owners, Project locks or changing source data. It never evicts a live or still-referenced coordinator to make room.

Successful and aborted source-import identity/before-image rollback manifests have a separate restartable retention cleanup. A marker/pages set becomes eligible only after the import's publication or SourceImportAbort is terminal, its operation ID and source boundary are below every idempotency, audit, redelivery and reconciliation floor, no publication/abort/candidate/lease/cleanup references it, no hidden source row retains its token, and the snapshot-reader grace boundary has passed. Cleanup verifies the root, page count/digest and terminal source postcondition, deletes pages in bounded idempotent batches, then removes the marker. Per-project rollback-manifest row/byte ceilings prevent a new import and keep serving authoritative if safe reclamation cannot keep up; pages are never TTL-deleted or discarded while they could still be required to resume or undo an import.

Terminal operation-fence rows have a separate restartable retention cleanup; active fences are never TTL-deleted. A Cleared/Aborted fence becomes eligible only after its publication/abort revision and operation identity are below every replay, idempotency, audit and redelivery floor, no current candidate, lease, marker or cleanup references it, and the snapshot-reader grace boundary has passed. A cleanup marker walks terminal fences by project/family/operation in bounded idempotent batches, rechecks those proofs, then removes the rows and their scope-index entries. Per-family terminal-fence row/byte ceilings stop further materialized bulk work and force authoritative fallback if safe reclamation cannot keep up; they never clear an active safety boundary to relieve pressure.

Current storage has measured per-family row/byte ceilings. If cleanup cannot prove reachability safety or keep below a ceiling, materialized writes for that family stop and reads fall back authoritatively; no unsafe deletion or unbounded token accumulation is permitted. Metrics cover visible generation lag, reachable/superseded rows, cleanup age/failures and bytes reclaimed.

Rebuild leases¶

Leases are scoped by project, metric family and scope. They contain a random token, monotonic lease generation, owner, start/expiry times and captured source high-water mark. Expired owners cannot publish or release a successor's lease. Lease acquisition never blocks authoritative fallback.

The authoritative rebuild calculation must run in a snapshot-isolated source read whose database snapshot token/cluster time and visible committed-delta boundary are captured together. The rebuilder then replays every affected delta committed after that exact boundary and loses the final compare-and-swap if another mutation arrives before publication. If the datastore or repository path cannot provide and validate that snapshot boundary, the rebuild rejects and restarts whenever any affected source revision advances; it must not replay deltas over an indeterminate live calculation. It therefore cannot double-apply or silently omit a concurrent source mutation.

Query and authorization contracts¶

Canonical endpoints¶

The proposed domain-neutral contract is:

POST /api/projects/{projectId}/statistics/current/query
POST /api/projects/{projectId}/statistics/history/query
POST /api/projects/{projectId}/statistics/history/{checkpointId}/query
POST /api/projects/{projectId}/statistics/audits
POST /api/projects/{projectId}/statistics/rebuilds

The three read-only query routes accept a bounded JSON request body; history-list cursor and limit are body fields. They use POST because UUID-rich selector bundles cannot safely fit within Kestrel/ingress request-line limits. A small GET compatibility adapter may exist only with a separately measured encoded request-line cap below every deployed proxy/server limit; it cannot advertise the canonical body limit.

Each selection body item pairs exactly one catalogue metric key with one canonical typed scope; the server does not form a cross-product between independent metric and scope lists. This lets one coherent request contain, for example, project-screening@project and stage-annotation@stage:{stageId} while retaining one-snapshot fallback semantics. The typed scope uses the catalogue's canonical encoding: project, stage:{stageId}, membership:{membershipId}, membership-stage:{membershipId}:{stageId}, stage-question-version:{stageId}:{questionId}:{versionId}, or systematic-search:{searchId}. IDs are validated and canonicalized before repository access; each metric key declares its permitted scope types, and an incompatible explicit pair is a bounded validation error. Legacy metric/stage/membership parameters may exist only in wire-compatibility adapters that construct the intended pairs. The current, history-list and exact-history routes accept the same selections, so heterogeneous bundles stay coherent while question/search requests remain bounded and authorization can be applied before fetching stored scopes. Before full JSON materialization or canonicalization, the route applies a 128 KiB request-body ceiling and a streaming count ceiling of 100 raw selection items, including duplicates; the request-limit layer maps either overflow to typed StatisticsSelectionCapacityExceeded/413. After canonicalization and exact deduplication, every canonical body route also permits at most 100 selections and 64 KiB of encoded selection keys; Phase 0 capacity tests may only lower these limits before production. Exceeding either returns a typed StatisticsSelectionCapacityExceeded/413 before opening a MongoDB snapshot. Adapters request only scopes their page renders; a coherent bundle above the cap is explicitly unavailable rather than silently split across snapshots. For the history-list route, omitted limit normalizes to 20 checkpoint roots, the hard maximum is 50, and selectionCount * normalizedLimit must not exceed 1,000 returned metric blocks. A requested value above either bound returns typed StatisticsHistoryPageCapacityExceeded/413 before opening a MongoDB snapshot; it is never silently clamped. Phase 0 capacity tests may lower these bounds. The exact-checkpoint route remains bounded by the selection limits above.

Selection and block counts alone do not bound response bytes, because one authorized physical scope may be as large as its 8 MiB document admission ceiling. Every query route therefore also enforces a measured aggregate encoded-response byte ceiling — provisionally 16 MiB per response; Phase 0 capacity tests may only lower it before production — applied identically to materialized results, authoritative fallback results and historical observation loading. Response assembly tracks the running encoded size and stops before constructing or serializing an oversized payload: crossing the ceiling returns typed StatisticsResponseCapacityExceeded/413 for the whole requested bundle rather than a truncated, partial or memory-unbounded response.

Current and available-history responses carry:

requested metric catalogue keys and scopes;
lifecycle and read source (Materialized, AuthoritativeFallback or MaterializedCheckpoint);
fallback reason;
checkpoint ID, global client-invalidation revision, source-invalidation revision, materialized projection revision and project client-invalidation revision;
authoritative source and committed-delta watermark;
catalogue/schema/source versions;
configuration and content digests;
observed/calculated times and trigger; and
authorized typed metric blocks, including stable profile keys and counts for requested profile families.

The history cursor and exact selector use an opaque stable encoding of the complete ordered (CheckpointSourceRevision, CheckpointProjectionRevision, mode epoch) checkpoint identity, not a timestamp, so two checkpoints that share a revision pair across a mode transition are distinct and ordered. Ordering is lexicographic newest first and stable under retention compaction. The list and exact-checkpoint contracts accept the same metric-scope selections as current reads.

Authorization¶

Project/stage totals require the existing project-view boundary and metric-specific graph permissions.
Reviewer/member blocks are keyed by membership identity while carrying investigator/person identity. A reviewer's own row remains available where current behavior permits it; peer rows/leaderboards require the corresponding project permission.
Current stage-question-version and systematic-search selectors must resolve inside the requested live project and pass the same metric-specific graph/export permission used by their authoritative live calculation; unknown, cross-project or unauthorized typed keys are rejected before projection lookup.
Historical routes first authorize the requested project under current permissions, then resolve a deleted or live selector only against immutable project ownership and typed-scope metadata in the selected checkpoint root/reference. They never require the question/search to remain in the live graph and never perform a cross-project selector probe. Each historical block additionally requires the same metric-specific graph/export permission as its equivalent current selector, evaluated under current project state; project-level authorization alone never returns a metric block. When a deleted scope no longer has enough live authorization context to evaluate that metric permission, the request fails closed. Missing, foreign and permission-unevaluable selectors return the same bounded unavailable result so retained scope existence is not leaked.
Historical responses are filtered under current authorization, not authorization captured when the snapshot was written.
Audit/rebuild operations require project-edit or the narrower administrative permission approved in Phase 0.
Export/report callers use their existing explicit export permissions and the same query service.
The persistence document is a superset and is never serialized directly.

Coherent current fallback and historical availability¶

A current response reads ProjectStatisticsGlobalControl, ProjectStatisticsControl, every requested scope, its bounded family publication guards and matching operation fences — together with the authoritative Project document's membership, permission and configuration state used for authorization — in one MongoDB read-only transaction with snapshot read concern at one pinned cluster time. Separate majority reads are not sufficient. Authorization is evaluated against that same pinned snapshot for current, authoritative-fallback and historical responses, never against a separately read Project, so a concurrently committed membership or graph-permission revocation either wholly precedes the snapshot or does not affect it; revision notifications protect later requests and cached clients, not the in-flight read. Historical routes likewise read the Project authorization state in the same pinned snapshot used to resolve the checkpoint root, pages and observations. Materialization is allowed only when global/project mode is Enabled, every family guard is Fresh with zero active fences, and the query has first selected for each scope its exact published VisibleGeneration/VisibleToken row when present, or otherwise the greatest published row below that generation, without compatibility filtering. A selected tombstone is a conclusive deleted/omitted scope, not permission to fall through to an older row. Only after selection does the reader require every live scope to be present, Fresh, compatible, carry the active WriteEpoch, and have LastChangedRevision less than or equal to the control's CommittedProjectionRevision and has no matching Active operation fence. An older LastChangedRevision means the scope was unchanged, not stale, because the write contract requires every affecting mutation to update or fence it atomically before advancing the project commit clock. An incompatible newest selected row therefore rejects the complete materialized bundle; the reader must not descend to an older compatible row. Otherwise the query executes the authoritative current aggregate for the entire requested bundle. This prevents a response combining fresh screening with stale annotation or different configuration versions. The control, epoch, revision, scope and fence predicates are therefore evaluated against one true snapshot.

The complete authoritative fallback bundle must likewise execute every requested authoritative facet and mapper read inside one MongoDB source snapshot, including the Project hidden-import visibility gate. If a deployment cannot provide that source snapshot, it may instead capture the authoritative source revision and all visibility/configuration boundaries before the bundle, rerun/validate those same boundaries after the bundle, and retry only within a measured bounded limit. The captured boundary set must include the project's monotonic SourceInvalidationRevision (and the global and mode epochs): because every source commit that changes an authorized response advances that clock and it never returns to an earlier value, two compensating changes between capture and re-validation (a hidden-import token set then cleared, a threshold changed then restored) still leave the clock advanced and the bundle is rejected; comparing only the boundary values themselves is not a proof. If neither a single source snapshot nor this stable-boundary proof succeeds, the API returns typed StatisticsBundleSnapshotUnavailable bounded unavailability (mapped to 503), never a weaker multi-read result that can mix source revisions. An affected bundle also returns typed StatisticsInclusionRecalculationInProgress/503 while the matching Project inclusion-info calculation token is active: a snapshot cannot make the deliberately multi-pass source rewrite logically complete. Only a matching verified completion may clear that visibility fence; afterward either its atomic publication serves or the complete bundle falls back authoritatively from the post-rewrite boundary.

A historical response never uses the current authoritative aggregate as fallback because Project and Study cannot reconstruct an older checkpoint. A retained compatible checkpoint-set root returns MaterializedCheckpoint only when all requested references resolve and are compatible. An unchanged scope may resolve to an earlier immutable observation referenced by that same root; this is one logical checkpoint bundle, not a join across checkpoint roots. A checkpoint that was compacted, is incomplete, or has no compatible historical reader returns a typed HistoryCheckpointUnavailable result (and 410 Gone for an exact-checkpoint request) with checkpoint ID/revisions, observed time when known, and bounded reason. It never returns current values labelled as history or assembles a bundle without one complete checkpoint root.

Observability and acceptance metrics¶

Required telemetry¶

OpenTelemetry instruments must cover:

current/history requests and requested block counts;
materialized reads, authoritative current fallbacks and historical-checkpoint unavailability by bounded reason;
authoritative aggregation count and duration;
materialized read and transactional-delta duration;
delta committed, deduplicated, revision-gap and conflict counts;
source-operation receipt committed/deduplicated/conflicted/reclaimed/rejected counts, rows, bytes, floor age and capacity pressure;
transaction retry, unknown-commit and write-conflict counts;
rebuild requested/acquired/completed/failed/lost-race counts and duration;
parity audits, mismatches, invalidations and repairs;
checkpoint root/page/observation counts, page bytes, failed page-digest/Merkle/range validations and compaction;
checkpoint-build markers by lifecycle, active lease age/renewal/takeover/lost-owner counts, orphan pages/observations, cleanup lag/failures/bytes reclaimed, per-project active admission and reserved bytes, coalesced newer requests and terminal-marker capacity pressure;
replacement-manifest live/orphan/eligible page counts and bytes, aborts, cleanup lag/failures and bytes reclaimed;
publication-operation coordinator counts/bytes by bounded lifecycle, admission-to-completion latency, crash resumes, cleanup lag/failures, ceiling pressure/capacity rejections and rejected ownership transitions;
source-import rollback-manifest live/eligible page counts and bytes, cleanup lag/failures and bytes reclaimed;
active/terminal/eligible operation-fence rows and bytes, retention lag/failures and bytes reclaimed;
current-scope, exact/sealed revision-ledger and checkpoint byte sizes, floors/ceiling pressure and per-family growth;
durable global/project epoch transitions, stale-writer rejection and snapshot-read fallback;
notification-outbox backlog age/count, claim retries, delivery failures and deduplicated deliveries;
notification-slot coalesces, target/delivered client-invalidation revision lag and backlog-ceiling fallback transitions;
global/family/consumer flag decisions; and
SignalR invalidations and subsequent snapshot fetches.

Metrics must not label by project, membership, investigator or question ID. Those IDs may appear in structured, access-controlled diagnostic logs with operation/event correlation.

Histograms must support p50 and p95 reporting for authoritative aggregation, materialized reads, publication, rebuild and end-to-end consumer latency.

Common correctness gates¶

Every implementation phase requires:

100% exact parity for integer counters/tallies over its representative corpus and pilot observations, including query-level fixtures against every applicable current StudyStats facet;
zero materialized responses from Missing, Stale, Rebuilding, Incompatible, disabled or epoch-mismatched scopes, and no materialized multi-read path without a true MongoDB snapshot;
100% authoritative fallback success in injected lifecycle, flag and failure cases;
mutation/event coverage for every catalogue authority affecting that family;
no partial source/current/history state after injected transaction failure;
signed-delta inverse and old-bucket decrement/new-bucket increment tests;
idempotent callback retry, unknown commit and old redelivery tests;
deterministic ordering, gap/replay-floor rejection and bounded retention/compaction tests; and
permission parity for current and historical responses;
incompatible newest-row fixtures proving that readers never descend to an older compatible row;
concurrent source-mutation and injected snapshot-abort fixtures proving authoritative fallback is one coherent bundle or typed bounded unavailability; and
inclusion-info recalculation fixtures covering start fencing, each of the three update-pass crash boundaries, active-job read unavailability, same-token redelivery, mismatched completion, verified source-only release to Stale/fallback and atomic rebuild publication;
durable disable/re-enable transition tests covering invalidation delivery, callback retry, unknown commit and outbox deduplication.

Common performance gates¶

Phase 0 records current p50/p95 and query/facet counts using fixed datasets. Before a consumer cutover:

materialized response p95 must be at least 20% lower than its authoritative equivalent;
that consumer must execute at least 80% fewer authoritative statistics aggregations during a representative shadow/pilot run;
supported source-mutation p95 regression must remain below 10%;
½/5/10-reviewer concurrency must show bounded transaction retries/write conflicts and identify whether the small project summary is actually hot;
materialized block/storage growth must remain within the Phase 0 capacity model; and
parity or fallback instrumentation must add no unbounded-cardinality series.

If a family fails a performance gate, it remains dark or is revised; correctness gates are never relaxed to obtain the performance result.

The preserved screening candidate measured 1,000 studies over alternating samples with legacy p50/p95 of 75.29/99.06 ms and candidate p50/p95 of 69.63/77.47 ms. This is promising local evidence, not a Phase 2 pass: the generalized implementation must rerun the controlled proof and assert the agreed threshold.

Rollout, fallback, and rollback¶

Flags and allowlists¶

Later implementation uses the repository's generated feature-flag catalogue. The proposed controls are:

global materialized serving kill switch;
global materialized-write kill switch;
shadow parity mode;
per-family write/serve controls for screening, annotation and each later family;
independent consumer flags for Project Overview/API, Stage Overview, reviewer pages, SignalR, exports/reports and any history UI; and
an explicit project pilot allowlist, with no implicit wildcard outside reviewed configuration.

The exact names are finalized with the generated flag implementation. All default to off. A durable global/project control epoch—not the 30-second per-process runtime-flag cache—is the safety boundary. Every durable serving-mode transition uses a retry-stable transition identity and one bounded transaction. A global transition changes global mode, advances WriteEpoch where applicable, advances GlobalClientInvalidationRevision and coalesces the singleton global outbox slot observed by every statistics subscription; it never attempts an unbounded transaction over existing projects. A project or family transition advances that project's ClientInvalidationRevision and coalesces its reserved project-wide visibility slot plus any required family slot. This includes Enabled -> Disabling, Disabling -> Disabled, re-enable/new-epoch transitions and narrower family disable/re-enable. If the applicable fixed slot cannot be admitted, the transition fails closed before changing the durable serving mode.

Narrow gates are fenced by their own writer epochs, not only by the global one. The project control document carries a monotonic project WriteEpoch and each per-family lifecycle summary carries a monotonic family WriteEpoch; every projection row's AppliedWriteEpoch is the full (global, project, family) tuple captured in the writer's transaction snapshot, and a scope is servable only when all three components equal the current epochs for its project and family. A project or family disable advances its own epoch atomically with its Disabling transition and observes the same maximum-transaction-lifetime quarantine before Disabled, so a source transaction that observed the narrow gate as Enabled and commits afterwards produces rows that carry the superseded narrow epoch and are already non-servable. Narrow re-enable allocates a new narrow epoch and requires epoch-matched rebuilt scopes for that project/family exactly as global re-enable does; it never re-serves rows written under an earlier narrow epoch. Advancing the global WriteEpoch is reserved for global transitions because it invalidates every project. Callback retry and unknown-commit recovery compare the same transition identity and cannot publish two logical invalidations. Clients refetch after delivery, so a durable kill-switch transition invalidates a previous materialized response instead of leaving it cached until another source mutation.

Disabling first atomically advances WriteEpoch and changes durable mode to Disabling; the subsequent Disabled transition follows the same invalidation rule, and all snapshot reads use authoritative calculation immediately after either durable transition. Every source transaction must read that gate. Reading the gate inside the transaction selects behavior; it is not by itself a write fence, because a transaction whose snapshot predates the Disabling commit can still commit its projection writes afterward — reading a document does not conflict with a concurrent update to it. The durable barrier is epoch quarantine: Disabling -> Disabled may be claimed only after the deployment-verified maximum MongoDB transaction lifetime has elapsed since the Disabling commit, by which time every transaction that observed the previous Enabled epoch in its snapshot has committed or aborted. Projection rows committed inside that quarantine carry the superseded AppliedWriteEpoch and are already non-servable, and re-enable requires epoch-matched rebuilt scopes, so nothing a late old-epoch writer commits can ever serve. When it is not Enabled at the expected epoch, the transaction may commit the source mutation only while marking/fencing its affected scopes Stale, advancing SourceInvalidationRevision and ClientInvalidationRevision and coalescing the required ClientInvalidationRevision outbox slots, and inserting the bounded source-operation receipt; it cannot publish materialized deltas. If the receipt or outbox cannot be admitted, the source transaction fails before changing authoritative data. The same source-only notification rule applies when a family is non-servable because its dispatcher exceeded a backlog ceiling. Until the Phase 5 SignalR cutover, the existing legacy notification remains active; after cutover, no source mutation may commit without the durable invalidation slot update. Re-enabling uses a new epoch, compatible-writer readiness, rebuild/parity evidence and per-scope AppliedWriteEpoch checks before any scope can serve. A rolling deployment or rollback cannot activate serving until every active mutation owner advertises the required epoch-aware contract.

Rollback principles¶

Schema additions are additive until Phase 6 legacy retirement.
Legacy calculations and wire adapters remain available throughout pilot and soak.
A consumer flag rollback restores the legacy response without data migration.
Projection schema incompatibility marks affected scopes Stale and falls back.
A bad materializer release stops writes or advances the source version; it never rewrites compatible old history in place.
Rollback does not require deleting projection collections.
No production flag activation, data migration or legacy deletion is implicit in merging code.

Staged delivery plan¶

Phase 0: inventory, catalogue, baselines, and architecture¶

Outcome: one reviewed source of design truth before runtime changes.

PR boundary and waves

This documentation PR defines architecture, safety invariants, phase boundaries and the baseline catalogue. It does not claim the method-level Phase 0 inventory is complete.
A separate documentation-only Phase 0 completion PR must name every calculation/consumer, method-level mutation owner, stable profile formula, fixed benchmark dataset and executable benchmark command.
Parallel read-only analysis may refine metric equations/consumers, mutation authorities and benchmark datasets. The results join this document or a directly linked catalogue update; they do not create product code.

Migration, backfill and rollback: none.

History and event coverage: approve the canonical history semantics and complete method-level source/mutation ownership for every catalogue family. Unit and outcome materialization cannot enter this programme. Every profile family must document its bounded keys, formulas, authorized scopes, permitted overlaps and the application invariants that make any state impossible.

Acceptance and tests: this architecture PR passes documentation validation and link checks. The separate Phase 0 completion PR must prove every known current calculation and consumer has a catalogue disposition and must name fixed benchmark datasets and executable commands before Phase 1 is authorized.

Staging and production safety: no environment work.

User gate: approval of this PR authorizes only the separate documentation/read-only Phase 0 completion work. Phase 1 planning or implementation requires a second explicit user approval after that completion PR satisfies the catalogue, mutation-ownership and benchmark gates. No later merge, staging activation or production activation is authorized here.

Phase 1: shared versioned projection foundation, dark only¶

Outcome: reusable lifecycle, storage, history, event, rebuild and query infrastructure with no page cutover.

PR boundary and waves

1A — contracts and additive schema: global/project control, summary/token-addressed scoped-current, bounded source-operation receipt, immutable ordered revision record/range seal, checkpoint root/reference-page/observation, durable checkpoint-build marker/lease and per-project build admission, durable singleton-global plus project-scoped coalescing notification outboxes, rebuild-lease, publication-guard/operation-coordinator/marker/manifest-page, current/manifest/fence-cleanup and operation-fence types, indexes, serializers and repository contracts.
After 1A, two non-overlapping PRs may proceed in parallel:
1B — transactional delta foundation: deterministic signed moves, unique operation/delta idempotency through the all-mode source receipt, source-revision guards, safe driver callback retry/unknown-commit handling, transactional external-notification outbox, leased dispatcher and invalidation registry;
1C — read/control foundation: domain-neutral query envelopes, snapshot-read enforcement, current authorization filtering, durable write epochs/lifecycle, flags and telemetry.
1D — lifecycle integration: per-scope rebuild leases, backfill, bounded delta/checkpoint compaction, history query/unavailability and failure/repair evidence, integrating 1B and 1C.

Migration: create additive pmProjectStatisticsGlobalControl, pmProjectStatisticsControl, pmProjectStatisticsCurrent, pmProjectStatisticsSourceOperationReceipt, pmProjectStatisticsRevision, pmProjectStatisticsRevisionRangeSeal, pmProjectStatisticsCheckpointSet, pmProjectStatisticsCheckpointReferencePage, pmProjectStatisticsCheckpointObservation, pmProjectStatisticsCheckpointBuild, pmProjectStatisticsCheckpointBuildAdmission, pmProjectStatisticsGlobalNotificationOutbox, pmProjectStatisticsNotificationOutbox, pmProjectStatisticsRebuildLease, pmProjectStatisticsPublicationGuard, pmProjectStatisticsPublicationOperation, pmProjectStatisticsPublicationMarker, pmProjectStatisticsPublicationManifestPage, pmProjectStatisticsCurrentCleanup, pmProjectStatisticsPublicationManifestCleanup and pmProjectStatisticsPublicationOperationCleanup, pmProjectStatisticsOperationFence, pmProjectStatisticsOperationFenceCleanup collections/indexes. Outbox indexes enforce the bounded global singleton and one coalescing slot per project/family/kind, monotonic target revision, lease/undelivered scans and bounded delivery-receipt expiry; publication-manifest indexes enforce token/ordinal identity plus hard page/byte ceilings; checkpoint indexes enforce immutable root/page/reference identity, canonical metric/family/typed-scope full-reference-key range lookup, creator-BuildToken orphan scans, bounded pages and retention reachability; checkpoint-build indexes enforce token identity, one active unpublished builder across all CheckpointIds per project, lease expiry/recovery, cleanup cursors, active page/byte reservation and per-project terminal row/byte admission; receipt indexes enforce unique project/OperationId identity, observed-time floor scans and row/byte admission ceilings; revision indexes support exact operation/publication identity plus ordered project-revision/RecordId replay, complete sibling-group boundaries and barrier detection; publication- operation indexes enforce one coordinator per project/operation and bounded active/terminal-state scans. The durable MongoDB outbox is explicit programme storage and does not assume an existing in-memory middleware is durable. No Project or Study rewrite and no legacy deletion.

Backfill and rollback: allowlisted dry-run only; reads stay authoritative. Rollback disables writes and ignores the new collections.

History and event coverage: prove source + source-operation receipt + scoped current + unique delta atomicity, deterministic revision ordering, checkpoint selection and truthful current/history semantics with synthetic metric scopes before domain payloads arrive.

Acceptance and tests: domain tests, Mongo replica-set source/receipt/current/delta transaction tests, callback-retry/duplicate/digest/unknown-commit and below-floor rejection tests, point-delta move/document/BSON-cap admission, new/existing sparse-tally-key boundaries, over-cap source-only ineligibility and oversized-save fenced-routing tests, outbox crash-after-commit/redelivery/lease/dedup/concurrent-target/coalescing/prolonged-outage/ceiling-fallback tests, singleton-global transition broadcast/no-project-enumeration/reconnect/refetch/callback-retry/ unknown-commit/dedup/admission-fail-closed tests, same-source-revision projection-repair invalidation and same-client-revision multi-family dedup/refetch tests, source-only kill-switch/backlog-ceiling source/client revision advancement, source-operation receipt retry/unknown-commit/conflict/floor/reclamation/ceiling-fail-closed tests and post-cutover notification tests, checkpoint BuildToken/build-marker acquire-renew-expire-takeover/root-CAS/orphan-cleanup-resume/ terminal-marker-ceiling/fenced-scope/guard-CAS/ page-boundary/BSON-size/missing-page/page-digest/Merkle-proof/full-reference-key-range-scoped-read/ same-scope-cross-family-total-order/single-active-build-admission/newer-request-coalescing/ expired-build-clean-before-successor/project-active-byte-reservation/ unchanged-scope-reference/A-to-B-to-A-occurrence-identity/source-only-gap-forces-new-observation/ compacted-scope-proof-gap/atomic-root/retention tests, delta ceiling/range-seal tests, source-only and projection-only checkpoint-pair ordering/idempotent-repeat/digest-collision/gap tests, ½/5/10-reviewer concurrency tests, small-summary hotspot evidence, different-family concurrent lease and same-family typed-busy/wait/coordinated-candidate tests, true-snapshot control/scope/fence tests, gap-free ordered delta-replay and replay-floor restart tests, equal-revision sibling ordering/page-boundary/resume tests, concurrent delta introducing a scope absent from the initial candidate manifest with published-base, canonical-empty and authoritative-boundary initialization plus capacity-abort tests, durable epoch disable/rolling-writer/re-enable tests with project-wide client invalidation, transition callback-retry/unknown-commit/outbox-dedup and outbox-admission-fail-closed tests, bounded bulk candidate-batch/marker/guard-flip/ fence-cleanup and every crash-boundary test, all-family operation-owner admission/same-ID-resume/ different-ID-rejection/owner-release tests, operation-coordinator preparing/admitted/crash-resume/ final-CAS-race ownership-checked re-admission/crash-resume/retry-exhaustion/atomic-complete/abort/ matching-family-owner-release/terminal-retention/row-byte-ceiling/ pre-admission-rejection tests, visible/candidate token isolation, abandoned-token resume/purge/no-generation-skip/GuardVersion-independence tests and ordinary-write routing tests, bounded replacement-publication manifest/BSON-limit/missing-page/digest/candidate-ready tests, published-manifest-floor/retention/restartable-cleanup tests, capacity and non-capacity post-source abort/no-publication/operation-release/crash-resume tests, pre-completion BulkSourceIncomplete abort/partial-boundary/child-receipt-watermark/owner-release tests, non-terminal-coordinator child-receipt pinning/floor-hold/terminal-release tests, receipt-reservation-before-load, worst-case inverse-batch reservation growth (before-images larger than after-images) and terminal-cleanup-reserve abort-completion tests, question-deletion definition-rewrite fence tests (authoritative 503 and checkpoint rejection while active; idempotent resume; release only by the matching definition save), FullStats identity-envelope parity tests (MembershipIds under administrator and restricted views, derived StageIds), mode-transition stage-one singleton-only and stage-two restartable-batch tests (fleet of many projects; crash mid-batch leaves remaining scopes Incompatible), source-only-period daily observation tests (SourceInvalidationRevision advances alone and still produces the day's observation), inverse-reservation non-double-counting tests (one-forward/one-inverse extends by zero), fixed-session-threshold field tests (each of the ten minNumberSessions predicates asserted independently at SessionCountTarget 1 and 3), forward-versus-inverse lease-direction tests (inverse batches admitted only in Aborting under the abort-owner generation; forward batches rejected once Aborting; no transaction holds both leases), daily observation producer tests (changed-only request, per-day idempotency, coalescing through the admission row, gap reporting after exhausted retries), active-reviewer mode-epoch transition tests (digest Incompatible before the transition commits; Stale/fenced plus invalidation after it), mode-epoch checkpoint-identity tests (stage one alone makes the producer request a new observation and an authoritative build a distinct CheckpointId; same-day dedup and history cursors distinguish the epoch), screening capacity-fallback dependent-family fencing tests (no annotation row stays Fresh after a source-only screening commit), SourceComplete seal tests (crash before the seal recovers by rollback; after it by PublicationAbort; forward batches rejected after the seal; SourceComplete -> AllFamiliesAdmitted and the AllFamiliesAdmitted -> SourceComplete reverse are the only admission transitions), mode-pinned authoritative read tests (flag/durable-mode disagreement fails closed; observation identity distinct across epochs), candidate-artifact publisher-generation revocation tests (late candidate batch rejected after admission, after the guard flip and after abort entry), SourceComplete pin and retention tests, stable-boundary-proof compensating-change rejection tests, durable-mode source-write tests (assignment and capacity-guarded saves fail closed on flag/durable-mode disagreement), mode-epoch observation non-reuse tests, import-reveal ClientInvalidationRevision advancement tests (subscribed client receives the post-reveal population on both reveal paths), retention root/page cleanup-marker tests, successful-publication coalesced-successor handoff and successor-marker/orphaned-admission recovery tests, narrow project/family writer-epoch quarantine and re-enable tests, hidden-import reveal source-clock advancement tests, including incompatible mode/epoch, kill-switch and retry-exhaustion causes, matching candidate metadata reset, successor-ownership protection and completed-import authoritative reveal; checkpoint/client-dedup tests, explicit scope-deletion tombstone/no-old-row-resurrection/retention tests, atomic-all-family-admission/publication/no-partial-flip tests, terminal-fence-floor/reference/ snapshot-grace/restartable-cleanup/ceiling tests, typed paired project/stage/membership/ membership-stage/stage-question-version/search selection validation, heterogeneous-bundle coherence, raw-body/raw-item/repeated-selection cap/413, exact-dedup/count/encoded-byte-cap/413, aggregate-encoded-response-ceiling/413 on materialized, authoritative-fallback and history paths, history default/hard-limit/selection-product-cap/413, canonical-body/request-line-independent bounded-fetch, current-versus-deleted-history selector authorization/no-existence-leak tests, per-scope older-generation selection and snapshot-grace/ reachability cleanup/ceiling tests, point-mutation-during-bulk-fence tests, retention tests, migration/index tests, authorization tests and failure-injection fallback and historical-unavailability tests.

Staging proof: after separate approval, deploy disabled, verify additive indexes and startup, enable only a synthetic/non-serving allowlisted projection, exercise kill switches and rollback. No page reads may use it.

Production safety and user gate: flags remain off; no production backfill. User reviews every PR and approves completion of Phase 1 before Phase 2 merges.

Phase 2: project screening statistics¶

Outcome: reconcile the useful pull request #2985 screening work onto the shared foundation.

PR boundary and waves

2A — authoritative definition and transaction materializer: project screening profile payload, before/after classifier, signed delta, authoritative query and history checkpoint. Profiles preserve valid conflicting include/exclude decisions across reviewers and mirror all approved live facets.
2B — mutation adapters: screening submit/correct/rescreen/reset, threshold/stage/membership paths, imports, bulk updates and deletion fencing. This can proceed alongside 2A tests only where file ownership is disjoint.
2C — integration proof: API query-family adapter, parity audit, backfill/rebuild and controlled performance/regression harness. No ordinary page or SignalR cutover.

Migration: no source schema rewrite; create screening current scopes/checkpoints through pilot backfill. Add an import-operation rollback marker plus BSON-bounded identity/before-image pages and the PendingImportJobId lookup needed to resume or remove a hidden source prefix after terminal load failure; these are import-lifecycle records, not statistics history.

Backfill and rollback: project allowlist, restartable and idempotent. A flag rollback returns every read to the authoritative screening facets and preserves history.

History and event coverage: screening checkpoints use the shared policy; source, current signed deltas and immutable operation history share one transaction; configuration changes preserve old history and reclassify from settings-independent tallies when bounded, otherwise rebuild current.

Acceptance and tests: exact profile and total parity across zero/one/many-screening cases, mixed include/exclude decisions from different reviewers, threshold boundaries, rescreening stages, duplicate delivery, concurrent reviewers, transaction rollback, import/bulk/delete fences, authorization, history ordering/retention, terminal source-import error after each batch boundary, bounded rollback/redelivery, Project hidden-import gate/partial-load/zero-hidden-row proof, all-family manifest admission, atomic all-family publication/reveal, capacity-abort fallback, project-wide invalidation, source-import-abort matching-family-owner release and admission-lock release, sparse-screening-tally new/existing-key and per-document capacity fallback, and the common performance gates. Successful/aborted source-import rollback-manifest floor/reference/postcondition/ restartable-cleanup/ceiling tests belong to this phase because Phase 2 introduces the import rollback marker and pages. Tests exclude a state only when an enforced domain invariant proves it impossible.

Staging proof: after explicit approval, dark-write an allowlisted corpus, run shadow parity and load, force Stale/Rebuilding/Incompatible and kill-switch paths, and prove legacy response equivalence.

Production safety and user gate: no production serving. User accepts Phase 2 evidence before any consumer or wider-family work is promoted.

Phase 3: annotation, question, and domain reconciliation¶

Outcome: materialize the catalogue-approved annotation graph without reviving broad kappa work.

PR boundary and waves

3A — stage candidate/reconciliation statistics: bounded Study distributions over approved inclusion, no-session, incomplete/in-progress, completed and domain-reconciliation state profiles.
3B — membership/reviewer-stage statistics: authorized per-reviewer distributions over availability, incomplete/in-progress/completed and reconciliation profiles, plus approved capacity-derived metrics.
3C — question family: version-aware question tallies with no unit- or outcome-level materialization.
3D — mutation/invalidation integration: serialized wiring across shared Study/Project and durable SlotReservation repository/controller mutation paths, followed by the combined parity/load proof.

3A–3C may develop scoped payloads, authoritative queries and tests in parallel after Phase 1 contracts. Shared source mutation files remain single-writer and join only in 3D.

Definition-rewrite recovery: disabling statistics or removing a project from the allowlist prevents new fence admission, but does not cancel a durable token left by a failed question delete. The delete owner always attempts admission/resumption before rewriting Studies. When admission is disabled, this requires one indexed project-control read; with no held token the legacy two-write sequence remains unchanged, with no statistics writes or transaction. A matching token resumes and releases atomically with the Project definition save, while a foreign token remains typed-busy. This bounded read is an intentional exception to the flags-down no-control-read rule: skipping it can delete the question and make its earlier fence impossible to resume. Acceptance is a failed rewrite followed by disabling or de-allowlisting, retry, and re-enabling: the question deletion completes, the token and scope fences clear, and statistics reads recover. The owner regression and real replica-set recovery test cover these separate boundaries.

Project question selector boundary (#3197): current ProjectQuestion selectors resolve against Project.AnnotationQuestions from the bundle's already-pinned authorization snapshot, including synthesized system definitions. Missing and foreign question IDs are unavailable before any scope-row lookup. Historical selectors instead require membership in the checkpoint's retained scope set, so a question deleted from the current Project can still be read historically. Both routes retain Project.View; this project-wide grain adds no stage permission or database query. Contexts unable to resolve project question membership fail closed.

Definition rewrite event boundary (#3197): the fenced Project save uses an explicit SaveWithoutDispatchAsync unit-of-work path. It preserves ordinary version/audit persistence and invalidates the repository cache after every attempted save, but does not consume the domain-event graph. The owner evicts any Project cached during the transaction, then dispatches once after the shared commit helper confirms success. Reissuing an unknown-result commit does not repeat the save or event dispatch; save/release failure and exhausted unknown-result commits dispatch nothing. The flags-off ordinary save retains its existing behavior. This is the existing best-effort domain-event delivery model moved beyond the transaction boundary, not a durable outbox: a process crash after commit may lose events, and handler failures are not given exactly-once delivery guarantees. Durable domain-event delivery is separate follow-up work.

Migration: additive scoped-current documents only. The manual Project question tally remains until its replacement has passed parity and consumer cutover.

Backfill and rollback: family/project allowlists; each family can rebuild independently. Disabling one family causes coherent authoritative fallback for bundles requesting it.

History and event coverage: candidate and domain reconciliation transitions, annotation content changes, question-definition versions, suppression, membership/stage configuration, imports, correlated persistence, bulk update and deletion.

Acceptance and tests: profile classification truth tables, stage/inclusion splits, stage/membership-stage fixed inclusion-class payload identity and bucket-move tests, membership availability and unavailability subtraction formulas with unequal all/sessioned/investigator inputs, candidate/reconciliation lifecycle including every mapper output with AllowSelfReconciliation both enabled and disabled, outer merged membership-annotation formulas with non-zero unequal excluded inputs under WithUnexcluded, Unavailable and Separate grouping (including the separate excluded block), incomplete/completed session combinations, current-schema and legacy answer cases, reservation allocate/release/disconnect-timeout/expiry, concurrent writers, out-of-order events, definition deletion/version changes, screening-to-annotation dependency and every statistics-affecting stage-setting invalidation, parity, authorization, history and storage growth. Live connection presence remains outside the programme; only its durable reservation side effects are authoritative inputs. Unit-level statistics remain outside the programme. Outcome scopes remain unconditionally outside this programme and require a separately approved future feature; Phase 0 cannot enable them.

Staging proof: separately approved family-by-family dark backfill, representative annotation projects, controlled concurrent saves/reconciliation/import/delete operations, shadow parity and rollback drills.

Production safety and user gate: all serve flags off. User accepts each family and its metric catalogue before Phase 4 or consumer migration.

Phase 4: remaining valid project, stage, membership and derived summaries¶

Outcome: complete the approved catalogue without duplicating source counters.

PR boundary and waves

4A — membership/reviewer screening: reconcile reviewer and membership aliases into canonical scoped blocks.
4B — systematic-search population: implement the approved project + systematic-search payload, authoritative query, signed-mutation or fenced-operation adapters for search add/remove/import/delete, authorized current/history selectors and parity fixtures against the live population facets.
4C — approved project/stage derived summaries: derive from revision-consistent scopes where possible; materialize only when the performance case requires it.
4D — report/export projection adapters: prove the shared contract can satisfy approved reporting statistics without importing job-progress semantics.

These PRs may proceed in parallel when they consume stable Phase 1–3 blocks and do not share mutation adapters.

Migration/backfill/rollback: additive family blocks and independent allowlists. Derived summaries are rebuildable and revert to authoritative calculation on any dependency mismatch.

History and event coverage: search creation/removal, staged import reveal/abort, search deletion and population changes publish or fence the project + search scopes together. A derived historical response references one checkpoint-set root and may follow that root's immutable paged references for unchanged observations; it cannot assemble scopes from different roots. Dependency changes invalidate the derived scope.

Acceptance and tests: exact equality with current reviewer queries, including membership/reviewer fixtures for agreement exactly at its threshold and screening count above 3 but below a higher configured sufficiency threshold, unequal project/investigator sufficiency and overscreening counts proving the current project-wide membership mappings, plus unequal included-agree/excluded-agree source counts proving the duplicate excluded-agree output mapping, and unequal project/investigator values proving Available = allInfo.All.InsufficientlyScreened - investigator.InsufficientlyScreened and Unavailable = allInfo.All.SufficientlyScreened - investigator.SufficientlyScreened; approved report/export fixtures; project/search population parity for add/remove/import/delete and mixed-search projects; signed-delta/fenced routing, selector authorization, history, dependency invalidation, retention and common performance gates.

Staging proof: dark family backfill, parity across admin and ordinary reviewer views, report/export fixture comparison, storage/load measurement and rollback.

Production safety and user gate: no consumer activation. Agreement/kappa remains excluded unless an independent statistical specification has been approved.

Phase 5: page-by-page and consumer migration¶

Outcome: remove authoritative aggregation from hot consumers one reversible surface at a time.

PR boundary and order

5.1 — screening-only API and Project Overview screening surface. Request only the Phase 2 family so this first runtime slice can actually use materialization. Keep broad Project DTO/FullStats mapping authoritative until all families it requests are available.
5.2 — Stage Overview and stage charts. Consume catalogue keys rather than a page-specific cache.
5.3 — reviewer/stage review surfaces. Preserve own-row/peer-row authorization and availability semantics.
5.4 — SignalR. Publish client-invalidation revision/family invalidations plus source/projection provenance; authorized clients fetch a snapshot and discard older client-invalidation revisions.
5.5 — exports and reports. Use the same query service and projection-revision provenance.
A visible history UI is optional and follows the shared history endpoint; it is not required to prove durable backend history.

Each consumer is its own PR and independently reversible flag. The active-reviewer work delivered by pull request #2467 is now on main; SignalR begins only after the shared statistics contract is approved and integrates against that published boundary.

Migration/backfill/rollback: no destructive schema migration. A consumer may activate only for projects whose requested families are Fresh; its flag rollback immediately restores the unchanged legacy path.

History and event coverage: consumer adapters cannot invent additional counters or historical formats. SignalR carries client-invalidation revision, source/projection revisions and family identity only, not unauthorized raw projection documents.

Acceptance and tests: API/DTO and ngrx contract tests, page selectors, SignalR ordering/reconnect, authorization, export fixture parity, shadow comparison and common performance gates.

Staging proof: one page/consumer at a time, independently flagged, with before/after p50/p95, authoritative aggregation count, parity, fallback and rollback evidence.

Production safety and user gate: no automatic activation after merge. User approves each consumer's staging evidence and any later production pilot separately.

Phase 6: fleet backfill, soak, audit, and legacy retirement¶

Outcome: prove the programme under sustained load before removing legacy calculations.

PR boundary and order

6A — bounded fleet backfill/repair operations: admission, rate limits, leases, progress and safe cancellation.
6B — soak/audit/retention evidence: parity sampling, rollback drill, load/cost and storage proof.
6C — legacy deprecation: separate PRs after every consumer has passed and the rollback window is complete.

Migration/backfill/rollback: bounded batches, project/family admission, resume tokens and global stop. Backfill never invents history. Legacy code and indexes remain through the agreed rollback window.

History and event coverage: audit every catalogue family; compaction and schema migration retain interpretability and provenance. Repairs append evidence and a new projection revision/checkpoint.

Acceptance and tests: at least seven days of staging soak plus controlled 10,000 reads and 1,000 relevant mutations; 100% exact counter parity; zero stale materialized serves; 100% injected fallback success; no unresolved rebuild failures; bounded delta-ledger/storage growth; and the common performance gates.

Staging proof: full backfill, failure injection, rollback to authoritative reads, restart/resume, retention compaction and consumer rollback.

Production safety and user gate: production pilot, wider production rollout, legacy calculation removal and old collection/index cleanup are four separate explicit approvals. This plan grants none of them.

Reuse and supersession¶

Pull request #2985: reusable screening foundation¶

The preserved screening-only implementation contributes:

explicit Fresh/Stale/Rebuilding and authoritative fallback;
schema/source versions, calculation time, reason, stage scope, content/configuration watermarks;
rebuild lease expiry and publication-race rejection;
atomic Study + screening statistics mutation, optimistic retry and redelivery handling;
pilot allowlist, global kill switch, parity audit and OpenTelemetry instruments;
current/history authorization precedent; and
domain, repository, API, SignalR and controlled Mongo transaction tests.

It must be revised before reuse:

replace screening-specific document/service/controller/flag terminology with shared contracts;
split its payload into natural current scopes, a small control/summary and immutable delta/checkpoint history;
keep its source revision but bind it to the authoritative aggregate revision and stable operation ID;
stop appending a complete payload snapshot after every mutation;
replace the bounded in-document recent-operation list with immutable unique delta records and a compacted watermark;
add per-family lifecycle and coherent whole-bundle fallback;
add shadow mode and independent consumer flags; and
replace its final mixed FullStats/SignalR splice with the Phase 5 consumer contract.

The local #2985 branch remains evidence only until Phase 1 is approved. Useful commits may be cherry-picked or reimplemented into Phase 2 after comparison with current main; the stale branch is not rebased or continued as the programme branch.

Pull request #2534: design evidence, not implementation base¶

Retain:

exact screening and annotation tally schemas and derived-value rules, subject to query parity;
Study-owned before/after classifiers, signed deltas, inverse properties and targeted MongoDB increment/decrement updates;
same-transaction source + statistics writes for ordinary mutations;
method-level event/invalidation evidence and two-phase staged import/pending deletion mechanics;
three-source parity, query-level fixtures, sparse history and realtime goals; and
benchmark, test, risk and rollout evidence from M008-M011.

Supersede:

one payload-heavy current document and magic tracking/sentinel identities;
daily snapshots without immutable operation provenance or authoritative revision;
Legacy/Backfilling/Live serving ambiguity, while retaining its clear one-baseline purpose;
mandatory write-freeze/bootstrap coupling where scoped fences, fallback and replay suffice;
broad admin rollout UI in the foundation;
project-wide pairwise/Fleiss kappa and stage chart claims; and
its stale base branch and unrelated large diff.

The complete retained/revised/superseded/rejected comparison is the recovered-design reconciliation, including all current facets, formulas, mutation paths, bulk mechanics, authorization, tests and unresolved semantic boundaries.

Once this plan and successor implementation PRs provide traceable links, #2534 can be closed as superseded without deleting its preserved worktree until the history is safely recorded.

Validation strategy¶

Layer	Required evidence
Metric domain	Classification/delta truth tables, zero/empty cases, configuration boundaries, exact canonical serialization
Projection domain	lifecycle, version compatibility, natural scope keys, signed moves, dedupe, ordering, retention and history provenance
Mongo repository	unique/index migration, atomic source/receipt/current/delta writes, leases, concurrency, callback retry, rollback and unknown commit
Authoritative parity	fixed representative datasets, admin and reviewer authorization views, exact per-family and whole-bundle comparison
Mutation paths	every matrix row, supported signed deltas, old/new bucket moves, two-phase fences, imports/bulk/delete, retries and redelivery
API	current/history/audit/rebuild contracts, cursor ordering, coherent fallback, permissions and no raw-superset disclosure
SignalR	revision ordering, duplicate/old notification rejection, reconnect snapshot, fallback and authorization
Web consumers	DTO/ngrx/selector/chart parity, independent flags, history adapter and rollback
Export/report	approved fixture equality, permissions, provenance and no operational job-stat contamination
Performance	fixed data sizes, alternating warm/cold samples, ½/5/10-reviewer concurrency, p50/p95, retries/conflicts, query/facet count, transaction overhead and storage growth
End-to-end	controlled local/preview project through screen, annotate, reconcile, import, bulk update, delete, audit, rebuild and rollback

Risks and controls¶

Risk	Consequence	Control
Missed mutation path	Fresh-labelled stale data	Complete catalogue/method matrix, configuration digests, Stale fencing, parity audits and fallback
Projection becomes source of truth	Irrecoverable semantic drift	Project/Study remain authoritative; rebuildable blocks; no domain invariant depends on projection
One large project document	Contention, write amplification, document-size failure	Natural mutable scope partitions plus a measured small summary; stripes only if benchmarked
Mixed revision response	Internally inconsistent page/export	Whole-request revision check and coherent authoritative fallback
Retry/redelivery or out-of-order source revision	Double counts or regression	Stable operation ID, guarded source revision, bounded all-mode source-operation receipt, unique immutable delta record when materializing, digest conflict and watermark; source fails closed if receipt capacity is unavailable
Rebuild races with writers	Rebuild silently omits changes	Captured committed-delta watermark, replay/CAS and expiring scope-fenced lease
False history	Misleading research record	Observed-at timestamps, immutable definitions, no retroactive synthesis or averaging
Unbounded history/delta growth	Atlas storage/cost pressure	Tiered 90-day/one-year/five-year checkpoints, 256 cap, compact delta details and size telemetry
Permission leakage through history	Former member or reviewer sees restricted rows	Current authorization on every query; stored superset never serialized directly
Arbitrary profile cross-product	Unbounded payload/cardinality and undefined meaning	Only versioned catalogue profiles matching approved live formulas; no client-defined dimensions
Valid conflicting reviewer decisions dropped	Project profile undercounts real Study states	Preserve mixed include/exclude combinations; omit only states forbidden by enforced application invariants
Invalid cross-stage agreement	Misleading quality claims	Exclude #2534 kappa; require separate same-stage statistical specification
Rollout couples every page	Large blast radius and difficult rollback	Global/family/consumer flags, project allowlist, shadow parity and one-consumer PRs
Source-only or projection-only change leaves a live client stale	Source and projection clocks can advance independently	Independent transactional ClientInvalidationRevision and fixed family/project-wide coalescing outbox slots advance for either change; global mode changes use one GlobalClientInvalidationRevision/broadcast slot without enumerating projects
Source write latency or conflicts	Review workflow slows	Under-10% p95 gate plus realistic concurrency/retry evidence; split/stripe only when measured, fenced async rebuild for unsupported work

Review gates¶

Approval of this document should answer:

Are the metric families and exclusions correct?
Is the natural scoped-current + immutable delta/checkpoint model preferred over both one large project document and copy-on-write generations?
Are the history checkpoints and five-year bound acceptable?
Are the exact-parity, 20% read-p95, 80% aggregation-reduction and 10% mutation-regression gates appropriate?
Are Phase 1 dark foundation, Phase 2 screening, Phase 3 annotation, Phase 4 remaining families, Phase 5 consumer cutovers and Phase 6 retirement the correct order?
Is a separate Phase 0 completion PR required before any Phase 1 implementation approval? (Proposed: yes.)

Until those questions are accepted, broad implementation remains paused.

Epic #1831 — pre-calculated statistics programme
Pull request #2534 — broad original implementation/design evidence
Pull request #2985 — preserved screening-only implementation evidence
Issues #1836–#1850 — original screening, annotation, audit and history decomposition
FEAT-006 — domain reconciliation workflow
FEAT-009 — screening annotations
FEAT-013 — exports and reporting consumers
StudyStatsQuery and FullStats on current main — authoritative legacy calculation

FEAT-006 and FEAT-009 are catalogue compatibility inputs, not delivery prerequisites: this programme mirrors whatever versions of their live calculations are approved when a metric family enters Phase 0. Accordingly, depends-on remains empty and the catalogue does not claim hard dependency edges.

Materialized Project Statistics Programme¶

Status and authority¶

2534. That implementation can compare screening decisions originating in different stages. Any future¶

Contents¶

Decision summary¶

Programme MVP and progressive enhancement¶

Problem and current architecture¶

Scope and authoritative metric catalogue¶

Catalogue rules¶

Catalogue baseline¶

Bounded study-state profile distributions¶

Target architecture¶

Source-of-truth boundary¶

Logical data model¶

ProjectStatisticsGlobalControl and ProjectStatisticsControl¶

ProjectStatisticsSummary and ProjectStatisticsScope¶

ProjectStatisticsSourceOperationReceipt¶

ProjectStatisticsRevisionRecord¶

ProjectStatisticsCheckpointSet, reference pages and observations¶

ProjectStatisticsCheckpointBuild¶

Global and project ProjectStatisticsNotificationOutbox¶

ProjectStatisticsRebuildLease¶

ProjectStatisticsPublicationGuard, ProjectStatisticsPublicationOperation and¶

ProjectStatisticsOperationFence¶

Lifecycle¶

History and provenance¶

Current revisions versus historical checkpoints¶

Delta and checkpoint storage reclamation¶

Bootstrap and rebuild truthfulness¶

Definition changes and deletion¶

Event and invalidation contract¶

Concurrency, retries, and rebuilds¶

Source ordering and measured contention¶

Transaction and idempotency rules¶

Bulk and import operations¶

Current-row reachability reclamation¶

Rebuild leases¶

Query and authorization contracts¶

Canonical endpoints¶

Authorization¶

Coherent current fallback and historical availability¶

Observability and acceptance metrics¶

Required telemetry¶

Common correctness gates¶

Common performance gates¶

Rollout, fallback, and rollback¶

Flags and allowlists¶

Rollback principles¶

Staged delivery plan¶

Phase 0: inventory, catalogue, baselines, and architecture¶

Phase 1: shared versioned projection foundation, dark only¶

Phase 2: project screening statistics¶

Phase 3: annotation, question, and domain reconciliation¶

Phase 4: remaining valid project, stage, membership and derived summaries¶

Phase 5: page-by-page and consumer migration¶

Phase 6: fleet backfill, soak, audit, and legacy retirement¶

Reuse and supersession¶

Pull request #2985: reusable screening foundation¶

Pull request #2534: design evidence, not implementation base¶

Validation strategy¶

Risks and controls¶

Review gates¶

Related work¶

`ProjectStatisticsGlobalControl` and `ProjectStatisticsControl`¶

`ProjectStatisticsSummary` and `ProjectStatisticsScope`¶

`ProjectStatisticsSourceOperationReceipt`¶

`ProjectStatisticsRevisionRecord`¶

`ProjectStatisticsCheckpointSet`, reference pages and observations¶

`ProjectStatisticsCheckpointBuild`¶

Global and project `ProjectStatisticsNotificationOutbox`¶

`ProjectStatisticsRebuildLease`¶

`ProjectStatisticsPublicationGuard`, `ProjectStatisticsPublicationOperation` and¶

`ProjectStatisticsOperationFence`¶