FEAT-024 Phase 0: Benchmark Datasets, Executable Commands and Capacity Baseline

Purpose

The materialized project statistics technical plan requires a separate documentation-only Phase 0 completion review that names "every calculation/consumer, method-level mutation owner, stable profile formula, fixed benchmark dataset and executable benchmark command" before Phase 1 may be authorized (see Phase 0). This document supplies the last two of those five: the fixed benchmark datasets and the executable benchmark commands, together with the capacity model that the plan's provisional bounds must be replaced by or confirmed against.

It is scoped to measurement design only. It defines no product code, authorizes no implementation, and records no environment work. Every measurement whose subject is projection code that does not exist yet is explicitly marked as executing in Phase 1 against dark code; only the authoritative-baseline and capacity-arithmetic measurements are Phase 0 work.

The bounds and gates restated here are drawn from the plan's bounded study-state profile distributions, ProjectStatisticsSummary and ProjectStatisticsScope, ProjectStatisticsCheckpointSet, reference pages and observations, source ordering and measured contention, transaction and idempotency rules, delta and checkpoint storage reclamation, required telemetry and common performance gates sections. Where this document proposes a different number, the plan value is retained until the proposal is measured and approved.

The calculations being benchmarked are the ones already catalogued by the parallel Phase 0 read-only passes; this document does not re-derive that catalogue and refers to each calculation by its entry point.

Inventory of existing harnesses

Assembled read-only against /home/chris/workspace/syrf/main on 2026-09-01. Every path below was verified to exist.

Asset	Path	What it gives Phase 0	Gap
Testcontainers MongoDB fixture (standalone)	src/libs/testing/SyRF.Testing.Common/Fixtures/MongoDbTestFixture.cs	A disposable mongo:8.0 container per xUnit collection, CSUUID serializer already registered, CreateContext() returning a wired MongoContext, GetCleanCollection<T>()	Standalone mongod: cannot serve multi-document transactions, so it cannot host the concurrency benchmark
Testcontainers MongoDB fixture (replica set)	src/libs/testing/SyRF.Testing.Common/Fixtures/MongoDbReplicaSetTestFixture.cs	A mongo:8.0 single-node replica set (WithReplicaSet("rs0")) that can start sessions and transactions; MongoDbReplicaSetTestCollection defined in the same file	Aligned to mongo:8.0 to match the standalone fixture, so B1 and B2 cannot straddle a server version; its only consumer, src/services/api/SyRF.API.Endpoint.Tests/SubmitAnnotationSessionServiceTests.cs, passes 15/15 on the new image
Shared-runner container policy	src/testing/DockerTestContainerPolicy.cs	Owner labelling and CPU/memory/PID caps applied by both fixtures via WithCreateParameterModifier; keys CI ownership off SYRF_TEST_JOB_KEY	Caps are fixed (2 vCPU, 2 GiB, 256 PIDs) and are applied whenever SYRF_TEST_JOB_KEY is set, so CI timings are not comparable with unlimited local timings
Integration lane	.github/workflows/ci-cd.yml (test-dotnet-integration, --filter "Category=Integration", --maxcpucount:1, DOTNET_PROCESSOR_COUNT: "4")	An existing, reviewed, self-hosted lane that already runs Testcontainers MongoDB work on main	Runs the whole solution's Category=Integration set; a benchmark tagged Integration would run on every main push inside a pinned 25-minute budget
PR unit lanes	.github/workflows/pr-tests.yml (--filter "Category!=Integration")	Proof that any trait other than Category=Integration is executed on every PR	A benchmark tagged Category=Benchmark would be picked up by the PR unit lanes unless those filters are amended
Domain test builders	src/libs/testing/SyRF.Testing.Common/Builders/StudyBuilder.cs, ProjectBuilder.cs, InvestigatorBuilder.cs	Fluent, deterministic construction of Study, Project and Investigator with agreement thresholds and schema version	No builder covers screening decisions, annotation sessions, reconciliation sessions, stages or memberships; StudyBuilder seeds _randomId from Random.Shared, which is not reproducible
Deterministic seed generator	src/services/project-management/SyRF.ProjectManagement.Endpoint/Seeding/DatabaseSeeder.cs with DataSources/BogusDataSource.cs (Randomizer.Seed = new Random(seed), default 42) and Builders/AnnotationHierarchyBuilder.cs	The only existing generator that produces screening decisions with deliberate reviewer disagreement and reconciliation, plus annotation sessions through ReviewSubmissionService.AddScreening / AddSessionData; fixed GUIDs in src/libs/project-management/SyRF.ProjectManagement.Core/Seeding/SeedDataConstants.cs	Fixed scale (five canonical projects, 15-30 studies each), hard-coded reviewer counts, runs only as an IRunAtInit inside the Project Management host behind SYRF_SEED_DATA_ENABLED / SYRF_SEED_DATA_MODE; not parameterisable and not callable from a test
Realistic study text	src/libs/project-management/SyRF.ProjectManagement.Core/Seeding/SampleStudies.json (embedded resource)	Synthetic, non-clinical bibliographic records for realistic document sizes	Only a handful of records; needs cycling plus a deterministic suffix to reach dataset scale
Repository integration test project	src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/ with MongoDbTestCollection.cs	Already references SyRF.Testing.Common and SyRF.ProjectManagement.Mongo.Data, already binds MongoDbTestFixture, already has Category=Integration siblings elsewhere in the solution	Contains no test of StudyStatsQuery or the reviewer-stats fan-out; nothing measures duration or round trips
Existing stats unit tests	src/libs/project-management/SyRF.ProjectManagement.Core.Tests/Services/ReviewStatsQueryServiceTests.cs, src/libs/project-management/SyRF.ProjectManagement.Core.Tests/ProjectManagementServiceTests.cs	Service-level behaviour (including the CalculatingInclusionInfo guard)	Mock-based; no MongoDB is exercised, so they establish no timing or facet baseline
Embedded golden-fixture precedent	src/libs/project-management/SyRF.ProjectManagement.Core.Tests/SyRF.ProjectManagement.Core.Tests.csproj (<EmbeddedResource Include="Fixtures/Ris/*.ris" />)	An accepted in-repo convention for checked-in expectation files loaded by tests	No JSON golden-file loader or update mode exists
E2E fixtures	e2e/fixtures/factory.fixture.ts, e2e/fixtures/db.fixture.ts, e2e/helpers/csuuid.ts, e2e/fixtures/data/search-10-references.csv	CSUUID encoding in TypeScript and a direct database fixture, useful if a consumer-level shadow run is later wanted	Playwright-scoped, per-test unique names, 10-reference dataset; unsuitable as a statistics benchmark corpus
Local dev MongoDB	docker-compose.dev.yml (mongo:8, --replSet rs0, database syrf_local)	A transaction-capable local MongoDB for manual mongosh timing and explain work	Empty unless separately seeded
Prod-shaped snapshot	scripts/mongo-seed-from-atlas-backup.sh; snapshot described in CLAUDE.md and docs/architecture/mongodb-reference.md	Would give prod-shaped document sizes and per-project study distributions for read-only explain and timing	Verified unavailable on 2026-09-01: the mongodb-syrf-local MCP server returned "The configured connection string is not valid", and docker ps -a lists no snapshot container. No production-shaped distribution could be measured for this document
BenchmarkDotNet	not present	—	A repository-wide search for BenchmarkDotNet returned no matches; there is no benchmark project, no [Benchmark] attribute and no benchmark configuration anywhere
Load/perf tooling	not present	—	Searches for k6, NBomber, artillery and autocannon returned no matching project, package or configuration
Timed Mongo scripts	not present	—	scripts/ and docs/scripts/ contain no timed-query or explain script; the only mongosh uses are the dev-compose health check and e2e/scripts/setup.sh

The net position: the container, serialization, resource-policy and CI-lane plumbing all exist and are reusable unchanged. Everything that turns them into a measurement — data generation at scale, timing, round-trip counting, expectation freezing and BSON sizing — does not exist and is enumerated in Harness gaps.

Fixed benchmark datasets

All datasets are synthetic and deterministic. No dataset is derived from, seeded by, or compared against production content: titles and abstracts come from the checked-in SampleStudies.json records cycled with a deterministic index suffix, investigator names are the Alpha/Beta/Gamma/Delta style placeholders already used by SeedDataConstants, and no real project, person or record identifier appears in any dataset, golden file or result artifact.

Each dataset is fully described by its ID, its seed and the parameters in the table below. Given the same generator version, ID and seed, the generated corpus must be byte-identical; the generator must therefore derive every GUID from a deterministic function of (datasetId, seed, entityKind, ordinal) rather than from Guid.NewGuid(), and must not use Random.Shared.

The existing builders and mutation methods cannot satisfy that requirement. Three concrete non-determinism sources were verified against main, and each defeats byte-identical regeneration:

• StudyBuilder.Build() calls the public Study constructor, which delegates to the private constructor with Guid.NewGuid() (src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/Study.cs:29), and StudyBuilder itself seeds _systematicSearchId, _referenceFileId and _projectId from Guid.NewGuid() (src/libs/testing/SyRF.Testing.Common/Builders/StudyBuilder.cs:23-25).
• ReviewSubmissionService.AddScreening reaches ScreeningInfo.ScreenStudy, which constructs a Screening whose parameterless base call is Guid.NewGuid() (src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/Screening.cs:19). Screening exposes no identifier-taking overload.
• AddSessionData reaches ExtractionInfo.AddAnnotations, which constructs an AnnotationSession stamping CreatedAtUtc, ReservedAtUtc and — for a completed session — CompletedAtUtc from DateTime.UtcNow (src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/AnnotationSession.cs:19-23). The session identifier is caller-supplied and can be made deterministic; the timestamps cannot.

Only Study already has a deterministic escape hatch: the public factory Study.CreateWithId (Study.cs:89), and H-1 uses it rather than StudyBuilder.Build(). Avoiding the two mutation methods is not, however, sufficient, and is not what H-1 does. The non-determinism they carry is not theirs: ReviewSubmissionService.AddScreening (ReviewSubmissionService.cs:54) is a three-line wrapper that guards active membership and delegates to Study.AddScreening (Study.cs:194), which delegates to ScreeningInfo.ScreenStudy (ScreeningInfo.cs:111) — and that is where the Screening is constructed. The same holds for sessions, where the AnnotationSession is constructed inside ExtractionInfo.AddAnnotations (ExtractionInfo.cs:181). Bypassing the service reaches the same constructors; only bypassing the aggregate methods themselves would avoid them, and those methods are what maintain ScreeningInfo.IncludedCount, ScreeningInfo.AgreementMeasure and ExtractionInfo.SessionTallies — precisely the derived fields every measured calculation reads. A harness that reimplemented them would produce a corpus that no longer matched production, which is a worse failure than a non-deterministic one because it would be invisible in the measurements.

H-1 therefore builds through the real domain methods and settles what they inject afterwards, on the freshly constructed, unpersisted corpus: identifiers to a value derived from (datasetId, seed, entityKind, key), with every reference to a rewritten identifier rewritten with it, and timestamps to one fixed instant, since no measured calculation reads a document's age and a per-entity derived time would only add noise. An absent timestamp stays absent, because whether a session has a completion time is state rather than a clock reading. Both normalizations are recorded in every results artifact. The result is not argued from that reasoning but verified: a unit test serializes two independently generated corpora with the production BsonClassMaps and compares the persisted bytes, and two separate process runs of the full baseline recorded the same corpus fingerprint. This is a harness-design constraint, not a product change: no production constructor is altered by it.

Dataset parameters

Parameter	stats-small-v1	stats-medium-v1	stats-large-v1	stats-tally-natural-v1	stats-concurrency-v1
Dataset ID	PS-DS-01	PS-DS-02	PS-DS-03	PS-DS-04a	PS-DS-05
Seed	20260901	20260902	20260903	20260904	20260905
Projects (target)	1	1	1	1	1
Projects (unrelated siblings)	1	2	4	0	1
Studies in target project	300	5,000	25,000	7,381	400
Studies per sibling project	50	200	500	—	50
Stages	3 (1 screening, 2 annotation)	5 (2 screening, 3 annotation)	8 (3 screening, 5 annotation)	1 screening	2 (1 screening, 1 annotation)
Stages with reconciliation enabled	1	2	3	0	1
Reviewers (project members)	4	10	25	120	10
Project administrators	1	1	2	1	1
Systematic searches	1	3	8	1	1
Reference files per search	1	2	3	1	1
Screening decisions per study	0-3	0-5	0-6	0-120, by construction	0-2 pre-seeded
Studies with conflicting include/exclude across reviewers	12%	15%	15%	96.7% (7,140/7,381, fixed by construction — see below)	10%
Studies with zero screening decisions	20%	15%	12%	0%	30%
Annotation sessions per study per annotation stage	0-2	0-3	0-3	0	0-3
Completed vs in-progress session split	60/40	55/45	50/50	—	50/50
Studies reaching the reconciliation eligibility threshold (NCCS >= 2) per reconciliation stage	at least 20%	at least 20%	at least 20%	—	at least 20%
Reconciliation sessions	started on 8% of eligible studies, completed on 5%	started 10%, completed 6%	started 10%, completed 6%	0	started 5%, completed 2%
Annotation questions	14	40	120	0	8
Distinct (NumberScreened, IncludedCount) tally keys	10	21	28	7,381	6
Approximate generated pmStudy documents	350	5,400	27,000	7,381	450
Approximate embedded screening decisions	450	12,500	75,000	590,480	400
Intended use	smoke, golden-file authoring, developer loop	primary gate baseline	worst supported read scale	tally-cardinality and capacity	write concurrency (Phase 1)

Three constraints on the table above follow from the authoritative code rather than from choice.

Reconciliation eligibility is NCCS >= 2, so a dataset needs at least two completed candidate sessions on any study it expects to reconcile. The threshold is minNumberSessions, hardcoded to 2 in src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/StudyStats.cs:313, and every reconciliation counter in SessionedGroupStage (StudyStats.cs:287) is gated on NumberOfCompletedCandidateSessions >= minNumberSessions. An earlier revision of this table gave PS-DS-05 a maximum of one annotation session per study per stage, which makes the eligible set empty: no study can reach NCCS >= 2, so "started on 5% of eligible studies" describes 5% of nothing and the dataset cannot produce the partial reconciliation availability, started and completed coverage the invariants table claims for it. PS-DS-05 therefore now generates 0-3 sessions per study per annotation stage, and every dataset with a reconciliation stage guarantees an explicit floor of eligible studies. The generator implements that floor by position rather than by a random draw — a declared share of studies is given two completed, non-reconciliation sessions outright on every reconciliation stage — so the realized share is exactly the declared one at any dataset size and stays stable across runs. The floor is a floor: studies below the threshold still exist, or the "available" and "full" classifications would have nothing to separate. MinimumReconciliationEligibleShare in ProjectStatisticsDatasetSpec carries the value, and a unit test asserts both halves.

Question versions are not a parameter, because the current model has no question version. An earlier revision specified two versions per question for PS-DS-02 and three for PS-DS-03. No generator can produce them: AnnotationQuestion.Update (src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectAggregate/AnnotationQuestion.cs:218) mutates a definition in place with no history, AQVersion/QuestionSetVersion/SessionVersion have zero matches in src, and the authoritative tally groups by ExtractionInfo.Annotations.QuestionId alone (src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Repositories/StudyRepository.cs:951-956). That also contradicted D12, which requires the version component to stay reserved-but-unpopulated until a versioning model exists. The row has been removed: every dataset is defined at the current (ProjectId, QuestionId) grain, and version coverage is deferred to a future harness revision that follows a question-versioning model.

stats-tally-natural-v1's conflict rate is not a free parameter either. An earlier revision of this table gave PS-DS-04a a 22% conflicting-decision rate, which the generator ignored. The dataset's defining property is that it covers every reachable (NumberScreened, IncludedCount) key at least once, and with 7,381 studies and 7,381 reachable keys that means exactly one study per key. A key is conflicting whenever 0 < IncludedCount < NumberScreened, which is true of 7,140 of them, so full key coverage fixes the rate at 96.7%. Lowering it to a naturalistic figure would need many more studies than the dataset's stated size and would change what the dataset is for. The table now publishes the measured value; ProjectStatisticsDatasetSpec carries the same figure and a unit test pins the count at 7,140.

stats-large-v1 additionally accepts an opt-in --scale 4 multiplier producing 100,000 studies in the target project. That variant is not part of any CI lane; it exists for manual, out-of-lane runs, because the integration lane's timeout is pinned at 25 minutes with a four-processor cap in .github/workflows/ci-cd.yml and a 100,000-study generation plus aggregation does not fit inside it.

The adversarial tally datasets

The screening tally is the plan's only explicitly data-dependent dynamic-key structure (screeningTallyCounts[(NumberScreened, IncludedCount)]). Its natural key space is exactly determined by the maximum number of screening decisions any one study carries: with 0 <= IncludedCount <= NumberScreened <= K, the number of distinct keys is (K + 1)(K + 2) / 2.

Maximum decisions on one study (K)	Distinct reachable tally keys
10	66
25	351
50	1,326
89	4,095
120	7,381
126	8,128
127	8,256
315	50,086

Two consequences follow, and both must be validated rather than assumed:

• PS-DS-04a (stats-tally-natural-v1) is the largest key set reachable from real screening decisions with a bounded reviewer population. It sets K = 120 and generates at least one study for every one of the 7,381 reachable keys, which requires roughly 590,480 embedded screening decisions. It is the dataset that exercises the tally guard against real source data. On its own it does not exercise the proposed 8,192-key screening soft limit: 7,381 keys sits below that limit, so a run over PS-DS-04a alone can neither admit nor refuse at the boundary and cannot support calling 8,192 a measured limit.
• PS-DS-04c (stats-tally-natural-under-soft-limit-v1, seed 20260907) and PS-DS-04d (stats-tally-natural-over-soft-limit-v1, seed 20260908) straddle that boundary with natural source data. The reachable key count is monotonic in K, and 8,192 falls between K = 126 (8,128 keys) and K = 127 (8,256 keys), so those are the only two adjacent natural cases. PS-DS-04c sets K = 126 and generates one study for every one of the 8,128 reachable keys, which requires 126 reviewers and roughly 683,000 embedded screening decisions; PS-DS-04d sets K = 127 for 8,256 keys, 127 reviewers and roughly 699,000 decisions. C2 runs all three and asserts that the soft-limit admission decision flips exactly once, between PS-DS-04c and PS-DS-04d.
• PS-DS-04b (stats-tally-synthetic-ceiling-v1, seed 20260906) has no source studies at all. It is a pure encoding fixture that constructs candidate scope documents holding 1,000, 10,000, 25,000, 50,000 and 50,001 dynamic tally keys directly, so the encoded-BSON admission guard and the key ceiling can be exercised at and just beyond the plan's limit without generating an unrepresentable corpus. Reaching 50,000 keys from genuine screening decisions would require a single study screened by more than 315 reviewers and on the order of 10.5 million decisions, which no supported deployment produces.

Invariants each dataset must exercise

Invariant (plan source)	PS-DS-01	PS-DS-02	PS-DS-03	PS-DS-04a	PS-DS-04b	PS-DS-05
Valid mixed include/exclude decisions from different reviewers are preserved and tallied, never labelled impossible	yes	yes	yes	yes	n/a	yes
Sufficiency, started and overscreening classifications all populated	yes	yes	yes	yes	n/a	partial
Studies with no screening decision at all	yes	yes	yes	no	n/a	yes
Fixed included/unexcluded and excluded annotation class buckets both non-empty	yes	yes	yes	n/a	n/a	yes
No-session, in-progress and completed annotation states all present per stage	yes	yes	yes	n/a	n/a	yes
Domain reconciliation availability, started, in-progress and completed states all present	yes	yes	yes	n/a	n/a	yes
Per-reviewer membership scope non-empty for every reviewer	yes	yes	yes	yes	n/a	yes
Question scope populated at the current (ProjectId, QuestionId) grain, version component reserved and unpopulated	yes	yes	yes	n/a	n/a	yes
Multiple systematic searches contributing to project totals	no	yes	yes	no	n/a	no
Sibling projects present, to prove per-project scoping of every count	yes	yes	yes	no	n/a	yes
Dynamic tally-key cardinality at or beyond the plan ceiling	no	no	no	approach	at and beyond	no

PS-DS-04c and PS-DS-04d are omitted from the table above because they exercise exactly one invariant between them: natural tally-key cardinality immediately below and immediately above the proposed 8,192-key screening soft limit. They carry no annotation, reconciliation or question coverage.

Executable benchmark commands

Commands are given for a checkout root of /home/chris/workspace/syrf/main or an equivalent worktree. The Status column states honestly whether the command runs against the repository as it stands today or depends on a harness that does not yet exist; harness IDs resolve in Harness gaps.

(a) Current authoritative baselines: p50/p95 and query/facet counts

The plan requires that "Phase 0 records current p50/p95 and query/facet counts using fixed datasets" (common performance gates). The subjects are the authoritative calculations already catalogued: the 15-facet $facet aggregation in StudyStatsQuery.GetFullProjectStatsAsync (src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/StudyStats.cs), the CountDocumentsAsync fan-outs StudyRepository.GetReviewerStatsForStageAsync and GetReviewerStatsForProjectAsync, the assignment-path fan-out StudyRepository.GetStudyReviewStatus, the question tally StudyRepository.GetAnnotationQuestionAnswerTally, and the systematic-search population sum reached through SystematicSearchRepository.GetAllSystematicSearchesLinkedToProject. All five are measured; none is declared in scope and left unmeasured.

Two of them are stage-scoped, and both read the stage before they query: GetReviewerStatsForStageAsync branches on review mode and GetStudyReviewStatus reads the stage's selection mode and session target. A single number taken from whichever stage came first would therefore be a baseline for one review mode only, so each is measured once on a screening stage and once on an annotation stage, recorded as separate series. PS-DS-04a has no annotation stage and records the screening series alone.

#	Measurement	Command	Status
A1	Confirm the existing Testcontainers MongoDB path still works before any measurement	dotnet test src/libs/testing/SyRF.Testing.Common.Tests/SyRF.Testing.Common.Tests.csproj --filter "Category=Integration"	Runnable today
A2	Generate a dataset into a disposable container and report the corpus it wrote	SYRF_STATS_DATASET=PS-DS-02 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsDatasetGenerator"	Needs harness (H-1)
A3	p50/p95 of GetFullProjectStatsAsync for one dataset	SYRF_STATS_DATASET=PS-DS-02 SYRF_STATS_ITERATIONS=100 SYRF_STATS_WARMUP=10 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline.FullProjectStats"	Needs harness (H-1, H-2, H-7)
A4	p50/p95 of the CountDocumentsAsync fan-outs: GetReviewerStatsForStageAsync (measured once per review mode, because its screening and annotation branches issue different count queries), GetReviewerStatsForProjectAsync, and the assignment-path GetStudyReviewStatus (also per review mode)	as A3 with --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline.ReviewerStats", which selects both the reviewer-stats and the assignment-path measurement	Needs harness (H-1, H-2, H-7)
A5	p50/p95 of the question tally and the search-population sum	Two invocations. Question tally: SYRF_STATS_DATASET=PS-DS-02 SYRF_STATS_ITERATIONS=100 SYRF_STATS_WARMUP=10 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline.QuestionTally". Search population: the same command with --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline.SearchPopulation"	Needs harness (H-1, H-2, H-7)
A6	Round-trip and facet counts per calculation per dataset	SYRF_STATS_DATASET=PS-DS-02 SYRF_STATS_COUNT_ROUNDTRIPS=1 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline" (counts are taken from a CommandStartedEvent subscriber on the driver's ClusterConfigurator, and the facet count from the number of $facet sub-pipelines in the emitted command)	Needs harness (H-3)
A7	Full baseline sweep across every dataset, emitting the results artifact	for d in PS-DS-01 PS-DS-02 PS-DS-03 PS-DS-04a; do SYRF_STATS_DATASET=$d SYRF_STATS_COUNT_ROUNDTRIPS=1 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsAuthoritativeBaseline"; done	Needs harness (H-1, H-2, H-3, H-7)
A8	Server-side plan and execution shape for the $facet pipeline, independent of the .NET client	Start docker compose -f docker-compose.dev.yml up -d mongodb, seed the dev database, then run a script of the form below	Runnable today with manual transcription

Command A8 has no pipeline artifact to consume: the 15-facet pipeline is constructed in C# and is not exported anywhere, so it must be transcribed by hand from StudyStats.cs into the script. That makes A8 usable today for an explain sanity check, but it is not a substitute for A3, and its number must never be recorded as the gate baseline.

// phase0-facet-explain.js — read-only. Run with:
//   mongosh "mongodb://localhost:27017/syrf_local" --quiet --file phase0-facet-explain.js
const pipeline = [
  { $match: { ProjectId: BinData(3, "<dataset-project-guid-base64>") } },
  // ... transcribed $facet stage from StudyStats.cs ...
];
const explained = db.pmStudy.explain("executionStats").aggregate(pipeline);
printjson({
  totalDocsExamined: explained.executionStats && explained.executionStats.totalDocsExamined,
  executionTimeMillis: explained.executionStats && explained.executionStats.executionTimeMillis
});
const samples = [];
for (let i = 0; i < 30; i++) {
  const t0 = Date.now();
  db.pmStudy.aggregate(pipeline).toArray();
  samples.push(Date.now() - t0);
}
samples.sort((a, b) => a - b);
printjson({ p50: samples[Math.floor(samples.length * 0.5)], p95: samples[Math.floor(samples.length * 0.95)] });

The deterministic in-repo seed data can be produced today, at its own fixed scale, without any new harness:

SYRF_SEED_DATA_ENABLED=true \
ASPNETCORE_ENVIRONMENT=Development \
dotnet run --project src/services/project-management/SyRF.ProjectManagement.Endpoint/SyRF.ProjectManagement.Endpoint.csproj

That produces the five canonical seed projects, including deliberate reviewer disagreement and reconciliation, but at 15-30 studies each. It is a useful shape check and a reference implementation for the generator; it is not one of the fixed benchmark datasets.

(b) Write concurrency: ½/5/10 reviewers — protocol defined now, executes in Phase 1

The plan requires simultaneous-write benchmarks by 1, 2, 5 and 10 reviewers "to both the same Study and different Studies", measuring "p50/p95 transaction duration, retry/unknown-commit count, write-conflict rate, summary-document bytes and authoritative fallback frequency" (source ordering and measured contention). Four of those five metrics have no subject until the transactional projection path exists, so the protocol is fixed here and executes in Phase 1 against dark code. Only the source-mutation arm — today's screening submit with no projection writes — can be measured in Phase 0, and it is worth measuring then, because it is the denominator of the plan's "supported source-mutation p95 regression must remain below 10%" gate.

Protocol:

1. Fixture: MongoDbReplicaSetTestFixture (transactions are required; the standalone fixture cannot serve them), with its image aligned to mongo:8.0 before B1 is run so that B1 and B2 share a server version — see D5. Dataset PS-DS-05.
2. Arms: the cross product of writer count N in {1, 2, 5, 10} and target in {same-Study, distinct-Studies} — eight arms.
3. Each writer performs the supported screening-submit mutation in a loop; writers start on a barrier so the overlap is real rather than staggered.
4. Each arm runs 20 discarded warm-up iterations per writer, then 200 recorded iterations per writer.
5. Recorded per arm: transaction duration p50 and p95; retry count; unknown-commit count; write-conflict count and rate (conflicts divided by committed transactions); the project summary document size measured as the encoded length of the persisted document; and the count of authoritative fallbacks observed during the arm.
6. Phase 0 baseline arm: the same eight arms run against today's source mutation with no projection writes, giving the pre-change source-mutation p50/p95 for the regression gate.

Proposed pass criteria, to be confirmed rather than assumed in Phase 1: bounded retries means p95 retries per committed transaction at most 1 and maximum at most 5 in every arm.

Hotness of the shared project summary is decided by comparing each B2 arm against its matching B1 arm, never by comparing two B2 arms against each other. Comparing B2's distinct-Studies arm against B2's same-Study arm cannot detect a hot summary: the same-Study arm contends on both the source Study and the shared summary, while the distinct-Studies arm contends only on the summary, so the same-Study conflict rate is normally at least as high even when the summary is the bottleneck — a criterion that requires distinct-Studies to exceed same-Study would approve the shared document in exactly the case it is meant to catch. The correct signal is the arm-for-arm delta introduced by the projection write: the summary is declared hot, and striping or splitting is justified, when the 10-writer distinct-Studies B2 arm shows a materially higher write-conflict rate than the 10-writer distinct-Studies B1 arm, since B1 has no shared summary to contend on and the two arms are otherwise identical. The same-Study pair is read as a control: a comparable B2-over-B1 rise there is consistent with per-Study contention rather than with the summary.

#	Measurement	Command	Status
B1	Source-mutation-only concurrency baseline, all eight arms	SYRF_STATS_DATASET=PS-DS-05 SYRF_STATS_WRITERS=1,2,5,10 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsConcurrency.SourceMutationBaseline"	Needs harness (H-1, H-6)
B2	Full projection concurrency benchmark, all eight arms	SYRF_STATS_DATASET=PS-DS-05 SYRF_STATS_WRITERS=1,2,5,10 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsConcurrency.TransactionalProjection"	Executes in Phase 1 (needs H-6 plus the Phase 1 projection path)

© Capacity tests

#	Measurement	Command	Status
C1	Conservative encoded-BSON upper bound versus actual encoded size, per key count, at 1,000 / 10,000 / 25,000 / 50,000 / 50,001 keys	SYRF_STATS_DATASET=PS-DS-04b dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsCapacity.ScopeDocumentCeiling"	Needs harness (H-5)
C2	Natural tally-key cardinality reached from real screening decisions, at and across the proposed soft limit	SYRF_STATS_DATASET=PS-DS-04a,PS-DS-04c,PS-DS-04d dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsCapacity.NaturalTallyCardinality"	Needs harness (H-1, H-5)
C3	Point-path admission bound: canonical delta-record BSON size and document/move counts for the worst supported single mutation	dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsCapacity.PointPathAdmission"	Needs harness (H-5); the move and document counts become directly measurable only in Phase 1
C4	Checkpoint reference-page bound: encoded size of a 500-reference page and the reference count that first reaches 256 KiB	dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsCapacity.ReferencePageBound"	Needs harness (H-5)
C5	Checkpoint publication bound: pages and bytes for a full checkpoint over each dataset's scope inventory	SYRF_STATS_DATASET=PS-DS-03 dotnet test src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/SyRF.ProjectManagement.Mongo.Data.Tests.csproj --filter "FullyQualifiedName~ProjectStatisticsCapacity.CheckpointPublicationBound"	Needs harness (H-5); executes in Phase 1 for real observation payloads

How the conservative encoded-BSON upper bound is computed and validated. The bound is computed statically from the document's declared shape, never from a sampled document, so it can be evaluated before the write is attempted:

upperBound(doc) = 5                                   // BSON int32 length prefix + terminator
                + fixedFieldsUpperBound                // catalogue-declared fixed fields, max encodings
                + 5 + len("screeningTallyCounts") + 2  // embedded sub-document header and name
                + keyCount * (1 + maxKeyNameLength + 1 + 8)

with every counter treated as an Int64 (8 bytes) even when it would currently encode as an Int32, and maxKeyNameLength taken from the catalogue's declared key grammar rather than from the keys present. The key grammar is s{NumberScreened}i{IncludedCount}, and maxKeyNameLength must be sized for the widest legal value of each component, not for the widest value a fixture happens to produce. No domain invariant bounds either component to three digits: AgreementMeasure.NumberScreened is an int (src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectAggregate/AgreementMeasure.cs:23) and IncludedCount is derived from the same non-negative int domain, so the widest legal representation is ten digits per component. The 50,000-key ceiling bounds only the number of observed cells, not their numeric values — a sparse tally holding s10000i5000 is 11 characters wide while sitting far below the ceiling. Sizing for the legal maximum, maxKeyNameLength = 22 (s + 10 digits + i + 10 digits), so each key costs at most 32 bytes and 50,000 keys cost at most 1,600,000 bytes, roughly 1.53 MiB. C1's fixtures use short keys and therefore cannot prove the bound; they can only show that a real encoding stays under it, which is why the bound is computed from the grammar and asserted against actual encoded length rather than derived from the fixtures.

Validation is empirical and two-sided. For each key count in C1 the test builds the real BsonDocument, takes its actual encoded length, asserts actual <= upperBound, and records the ratio so that an over-loose bound is visible rather than merely safe. It then asserts the guard's decision matches the plan: at 50,000 keys the write is admitted; at 50,001 it is refused, no MongoDB update is attempted, the screening family is marked Stale with typed StatisticsScopeCapacityExceeded, and the source commits through the source-only receipt path. The arithmetic above predicts that the key ceiling binds roughly five times sooner than the byte ceiling for this key shape — at 32 bytes per key the byte ceiling would not be reached until around 262,000 keys — and C1 exists to confirm or refute that prediction rather than to assume it.

Capacity model

Bound	Plan value	Phase 0 measured or proposed value	How measured	Consequence when exceeded (per the plan)	Phase 0 recommends lowering?
Current scope document encoded BSON	no greater than 8 MiB	retain 8 MiB pending C1	C1: conservative static upper bound versus actual BsonDocument encoded length	Marks the screening family Stale with typed StatisticsScopeCapacityExceeded, commits the source through the source-only receipt/invalidation path, and makes the project ineligible for materialized screening serving until an approved bounded representation or a lower-cardinality state permits rebuild	No
Dynamic keys per current scope document	no greater than 50,000	retain 50,000 as the hard ceiling; add a proposed per-family soft limit of 8,192 for the screening tally, to be confirmed by C2	C1 and C2: reachable key count is (K + 1)(K + 2) / 2 for maximum decisions K; C2 measures what real data reaches at K = 120 (PS-DS-04a, 7,381 keys), K = 126 (PS-DS-04c, 8,128) and K = 127 (PS-DS-04d, 8,256), so the boundary is crossed rather than approached	Same typed StatisticsScopeCapacityExceeded path as above	Not the hard ceiling; yes for a family-level soft limit, because 50,000 keys need a study screened by more than 315 reviewers and are unreachable in practice
Immutable observation document encoded BSON	no greater than 8 MiB, counted against the build's reserved byte budget	retain 8 MiB pending C5	C5: encode real observation payloads for each dataset	Records typed CheckpointCapacityExceeded unavailability for that build; no insert is attempted; current serving is unaffected	No
Point-path signed moves per transaction	at most 500	propose 128 after C3 confirms the worst supported mutation	C3: enumerate moves analytically from the mutation-ownership matrix in Phase 0; measure directly from the canonicalizer in Phase 1	Not an authoritative-save error: acquire the bounded all-family operation ownership and fences first, commit the source under that operation, and finish through the staged publication/rebuild protocol	Proposed, but only on Phase 1 measurement; retain 500 until then
Point-path projection documents per transaction	at most 100	propose 32 after C3	as C3	as above	Proposed, subject to the same Phase 1 measurement
Point-path canonical delta-record BSON	at most 256 KiB	retain 256 KiB pending C3	C3: encode a synthetic worst-case delta record at the proposed move and document counts	as above	No
Checkpoint reference-page references	initially 500, Phase 0 capacity tests may lower	retain 500 pending C4	C4: encode a 500-reference page and find the reference count that first reaches 256 KiB	Records typed CheckpointCapacityExceeded unavailability; current serving unaffected	No
Checkpoint reference-page bytes	initially 256 KiB, Phase 0 capacity tests may lower	retain 256 KiB pending C4	as C4	as above	No
Checkpoint publication pages	initial hard ceiling 256; Phase 0 must replace with a measured equal-or-lower limit before production	must be set by C5 before production; no value is proposed without that measurement	C5: pages produced for each dataset's full scope inventory, extrapolated to the largest supported project	as above	Undetermined — the plan obliges a measured equal-or-lower value, and C5 is the blocking measurement
Checkpoint publication bytes	initial hard ceiling 64 MiB, same obligation	as above	as C5	as above	Undetermined, same obligation
Exact delta ledger age	90 days per project (Phase 1 provisional, must be replaced with measured limits before production activation)	no Phase 0 value; the arrival-rate input requires the timed workload W-1 below, which does not yet exist	W-1: a timed mutation workload with a declared duration and operation mix, run against a fixed dataset, counting statistics-affecting mutations per project per elapsed day. Not derivable from a dataset generation profile or a restored snapshot — see the note below	Materialized writes stop and affected reads fall back to authoritative calculation; source writes may continue without detailed delta rows only while the minimal source-operation receipt and the required notification slot can commit	Not in Phase 0; Phase 0's obligation is to define W-1, not to supply a number
Exact delta ledger rows	100,000 rows per project, same provisional status	as above	as above	as above	Not in Phase 0
Retained range seals	at most 64 per project, by merging adjacent seals	retain 64	Not a Phase 0 measurement; it is a fixed structural bound, not data-dependent	Adjacent seals merge	No

W-1: the timed workload the ledger-retention bounds require

The 90-day and 100,000-row delta-ledger limits are expressed per project per unit of time, so replacing them needs a mutation arrival rate, and an arrival rate needs a duration. Neither source this document previously named can supply one:

• The fixed datasets describe a final corpus, not a schedule. PS-DS-01 through PS-DS-05 specify how many studies, decisions and sessions exist when generation finishes; they do not specify over what elapsed period those mutations arrived, nor in what order or mix. Counting the rows a generation profile produces yields a total, and a total divided by an undefined interval is not a rate.
• A restored snapshot exposes current documents, not history. The aggregates carry Audit.LastModified and the audit stamps written by OnSaving, which record only the most recent write. An in-place overwrite (Screening.ChangeScreeningDecision, AnnotationQuestion.Update) leaves no prior value and no prior timestamp, so a snapshot cannot be counted to recover how many statistics-affecting mutations a project received per day.

Phase 0's honest position is therefore that it cannot replace the provisional 90-day/100,000-row limits, and this document does not claim to. What Phase 0 owes instead is the definition of the measurement that can, recorded here as harness gap W-1 and scheduled for Phase 1:

Element	Requirement
Duration	A declared wall-clock or simulated-clock window, minimum 24 simulated hours per arm, with the simulated-to-wall-clock ratio recorded in the results artifact
Operation mix	Explicit per-operation weights over the mutation families of the mutation-ownership matrix — at minimum screening submit/rescreen, annotation session save and complete, reconciliation submit, slot claim and release, membership change and stage-settings change
Arrival model	A deterministic arrival schedule derived from (datasetId, seed), not a live load generator, so a rerun reproduces the same schedule
Base corpus	One of the fixed datasets, unchanged, as the starting state; W-1 mutates it rather than generating it
Output	Statistics-affecting mutations per project per simulated day, split by family, plus the resulting delta-row count and the age at which the oldest retained row falls out

Until W-1 executes, the 90-day and 100,000-row values stand as the plan's provisional limits, and any statement that Phase 0 has measured them would be false. An alternative admissible input would be a durable historical mutation-event trace — an append-only source that records each statistics-affecting mutation with its timestamp — but no such trace exists on main, and creating one is product work outside this programme.

Performance gate baselines

The gates are restated verbatim in intent from common performance gates. Each row names the dataset and command that produce its baseline number and the telemetry instrument that will record it in Phase 1 and later. Instrument names are proposals consistent with required telemetry and must be confirmed when the instrumentation is written; every one of them is free of project, membership, investigator and question identifiers, as the plan requires.

Gate	Baseline dataset(s)	Baseline command(s)	Metric that records it
Materialized response p95 at least 20% lower than its authoritative equivalent	PS-DS-02 primary, PS-DS-03 confirming	A3, A4, A5 for the authoritative side; the materialized side executes in Phase 1	syrf.projectstatistics.authoritative.aggregation.duration and syrf.projectstatistics.materialized.read.duration, both histograms supporting p50 and p95
The consumer executes at least 80% fewer authoritative statistics aggregations during a representative shadow or pilot run	PS-DS-02	A6 for the pre-change aggregation count per consumer interaction	syrf.projectstatistics.authoritative.aggregation.count
Supported source-mutation p95 regression below 10%	PS-DS-05	B1 gives the pre-change source-mutation p50/p95; B2 gives the post-change value	syrf.projectstatistics.source.mutation.duration
½/5/10-reviewer concurrency shows bounded transaction retries and write conflicts, and identifies whether the small project summary is actually hot	PS-DS-05	B1 then B2, all eight arms	syrf.projectstatistics.transaction.retry.count, syrf.projectstatistics.transaction.unknown_commit.count, syrf.projectstatistics.transaction.write_conflict.count, syrf.projectstatistics.scope.bytes
Materialized block and storage growth remains within the Phase 0 capacity model	PS-DS-03, PS-DS-04a, PS-DS-04b	C1, C2, C4, C5	syrf.projectstatistics.scope.bytes, syrf.projectstatistics.checkpoint.page.bytes, syrf.projectstatistics.delta.rows
Parity or fallback instrumentation adds no unbounded-cardinality series	all datasets	A7 plus a label-cardinality assertion in the results emitter (H-7) that fails if any recorded series carries a project, membership, investigator or question identifier	syrf.projectstatistics.fallback.count, labelled only by bounded reason

The plan also preserves an earlier screening-candidate measurement — 1,000 studies, legacy p50/p95 of 75.29/99.06 ms against candidate p50/p95 of 69.63/77.47 ms — explicitly as promising local evidence rather than a pass. That measurement used neither of the datasets defined here and is not comparable with them; it must not be reused as a gate baseline, and the controlled proof must be rerun on PS-DS-02.

Parity fixtures

Parity fixtures are golden files: the authoritative calculation's output for a fixed dataset, frozen once and thereafter compared exactly. They are the corpus for the plan's correctness gate of "100% exact parity for integer counters/tallies over its representative corpus", including "query-level fixtures against every applicable current StudyStats facet".

Location convention, following the existing embedded-fixture precedent in src/libs/project-management/SyRF.ProjectManagement.Core.Tests/SyRF.ProjectManagement.Core.Tests.csproj (<EmbeddedResource Include="Fixtures/Ris/*.ris" />):

src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/
  Fixtures/
    ProjectStatistics/
      PS-DS-01/
        PF-SCR-PROJ-01.json
        PF-SCR-MEMB-01.json
        ...
      PS-DS-02/
        ...

Freezing rule. On the first run for a (fixtureId, datasetId) pair, the runner executes the authoritative calculation and writes the golden file only when SYRF_STATS_FREEZE_GOLDEN=1 is set. CI lanes never set it. Any later run compares and fails on mismatch; a golden file is regenerated only by an explicit, reviewed commit that states which behaviour changed. A missing golden file in a run without the variable is a failure, not an invitation to create one.

Fixture ID	Family	Drawn from	Frozen expectation	Comparison rule
PF-SCR-PROJ-01	Project screening	PS-DS-01, PS-DS-02	ProjectScreeningStats including the full ScreeningGroupedCount tally, serialised with keys in ascending (NumberScreened, IncludedCount) order	Exact integer equality on every counter and every tally key; the key set must match exactly, with no key omitted for being zero
PF-SCR-PROJ-02	Project screening, conflict preservation	PS-DS-02	The subset of tally keys produced by studies carrying both include and exclude decisions from different reviewers	Exact; the fixture exists specifically to fail if a conflicting combination is ever collapsed or dropped
PF-SCR-MEMB-01	Membership screening	PS-DS-01, PS-DS-02	One MembershipScreeningStats per membership, ordered by membership ordinal within the dataset	Exact integers; the fixture must also assert that no peer reviewer's decisions appear in another reviewer's payload
PF-ANN-STAGE-01	Stage annotation	PS-DS-01, PS-DS-02	One StageAnnotationStats per stage, both included/unexcluded and excluded class buckets present	Exact integers
PF-ANN-MEMB-01	Membership-stage annotation	PS-DS-02	One membership-stage payload per membership and stage, covering all three ExcludedSessionStatsGrouping behaviours	Exact integers
PF-RVW-SCR-01	Reviewer screening	PS-DS-01, PS-DS-02	ReviewerScreeningStats screened, available, unavailable and total for every reviewer	Exact integers; recorded against the membership-screening fixture in the same run so that the plan's aliasing question is answered with evidence
PF-RVW-ANN-01	Reviewer annotation	PS-DS-02	ReviewerAnnotationStats per reviewer and stage, including the derived unavailable sum and maximum-in-progress flag	Exact integers; the boolean flag compared exactly
PF-Q-TALLY-01	Question answers	PS-DS-02	Per-question distinct-study count and annotation count, ordered by question identity ordinal, carrying the question definition identifier only; the version component is frozen as explicitly unpopulated, because no question version exists to freeze (D12)	Exact integers
PF-SRCH-POP-01	Search and population	PS-DS-02	Per-search recorded study counts and their project-level sum	Exact integers
PF-DOMREC-01	Domain reconciliation	PS-DS-01, PS-DS-02	Reconciliation availability, started, in-progress and completed counts per stage	Exact integers
PF-TALLY-CARD-01	Capacity	PS-DS-04a	The exact set of reachable tally keys and the resulting key count	Exact set equality, so a change in the tally key grammar cannot pass silently

Tolerance and rounding. Every value in every fixture above is an integer counter or tally and is compared with exact equality; no tolerance is permitted, matching the plan's catalogue rule. The only non-integer value in the current calculation surface is the per-study absolute agreement ratio. Where a ratio must be frozen, it is stored as an explicit numerator and denominator pair of integers and compared exactly; a decimal is stored only where the consumer genuinely receives a decimal, in which case it is rounded half away from zero to six decimal places and compared as a decimal string, and the rounding rule is recorded in the fixture file itself. No implicit tolerance is ever introduced.

Harness gaps

Everything below is read-only measurement code. None of it is product code, none of it ships in a service, and none of it may write to any environment other than a disposable Testcontainers instance.

ID	Missing capability	Minimal read-only harness	Consumed by
H-1	Parameterised deterministic dataset generator	A class in SyRF.ProjectManagement.Mongo.Data.Tests that takes a dataset ID and seed and derives every GUID and every timestamp from (datasetId, seed, entityKind, ordinal). Study roots are materialized via Study.CreateWithId rather than StudyBuilder.Build(); screenings and sessions go through the real domain methods, because the identifiers and timestamps they inject are minted inside the aggregate rather than in the service wrapper, and the aggregate is what maintains the derived fields the calculations read. What those methods inject is then normalized on the unpersisted corpus and the normalization recorded in the results artifact (see the determinism note above), with a unit test comparing two generated corpora byte for byte. It then inserts them into a fixture-provided database	A2-A7, B1, B2, C2
H-2	Duration sampling and percentile reporting	A small recorder that runs a delegate for a configured warm-up and iteration count, collects elapsed ticks, and reports p50, p95, min, max and count. No new package is needed	A3-A5, A7, B1, B2
H-3	Round-trip and facet counting	A driver ClusterConfigurator subscribing to CommandStartedEvent, counting commands per measured operation and extracting the number of $facet sub-pipelines from the emitted aggregate command	A6, A7
H-4	Golden-file parity runner	A loader for embedded Fixtures/ProjectStatistics/** JSON, a canonical deterministic serializer with stable ordering, an exact comparer, and the SYRF_STATS_FREEZE_GOLDEN write mode	all PF-* fixtures
H-5	Encoded-BSON capacity sizer	A helper computing the conservative static upper bound described above, comparing it against BsonDocument actual encoded length, and asserting the admission decision at and just past each ceiling	C1-C5
H-6	Concurrency driver	A barrier-synchronised multi-writer runner over MongoDbReplicaSetTestFixture, classifying MongoDB command exceptions into write conflict, transient transaction error and unknown commit, and recording per-arm summaries	B1, B2
W-1	Timed mutation workload for ledger-retention sizing	The duration, operation mix, arrival model, base corpus and outputs specified in W-1. Executes in Phase 1; without it the 90-day and 100,000-row delta-ledger limits cannot be replaced	delta ledger age and rows
H-7	Results emitter	A writer producing one machine-readable JSON artifact per run under a run-scoped output directory, containing dataset ID, seed, host and container image, iteration counts, and every recorded metric, plus a label-cardinality assertion that fails if any series carries a project, membership, investigator or question identifier	A3-A7, B1, B2, C1-C5

Counting the commands in this document: three are runnable against the repository as it stands today (the existing integration-fixture check A1, the manual explain and timing script A8, and the in-repo deterministic seeder invocation), and thirteen depend on one or more of the seven harness gaps above. Two of the thirteen — B2 and the observation-payload portion of C5 — additionally require the Phase 1 projection path and are marked as executing in Phase 1.

Unresolved evidence-backed decisions

D1. Where the harness lives

Evidence. A repository-wide search for BenchmarkDotNet returns no matches, and there is no benchmark project of any kind. src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests already references both SyRF.ProjectManagement.Mongo.Data and SyRF.Testing.Common and already binds the shared MongoDB fixture through its own MongoDbTestCollection.cs.

Impact. A new project changes syrf.sln, the solution filters, the dependency map and every CI filter that enumerates projects; an existing project does not.

Options. (a) Add a SyRF.ProjectStatistics.Benchmarks project with BenchmarkDotNet. (b) Add the harness as additional test classes inside the existing SyRF.ProjectManagement.Mongo.Data.Tests. © Write standalone shell and mongosh scripts under scripts/.

Recommended default: (b). It reuses the fixture, container policy, CSUUID configuration and lane plumbing unchanged, and needs no solution or dependency-map change. BenchmarkDotNet's strengths are in-process microbenchmarking; these measurements are dominated by MongoDB round trips, so a simple percentile recorder (H-2) is sufficient and avoids a new dependency.

D2. Trait and CI-filter collision

Evidence. The PR unit lanes run dotnet test syrf.sln --filter "Category!=Integration" (.github/workflows/pr-tests.yml), and the main integration lane runs dotnet test syrf.sln --filter "Category=Integration" with a pinned 25-minute timeout (.github/workflows/ci-cd.yml). A benchmark tagged Category=Benchmark is therefore executed by every PR; a benchmark tagged Category=Integration is executed on every push to main.

Impact. Either tagging silently adds many minutes of container and aggregation work to a lane sized for something else, and the integration lane's budget is pinned exactly by .github/scripts/test-main-dotnet-integration-runner-routing.sh, so an overrun is a CI failure rather than a slow run.

Options. (a) Category=Benchmark plus amending both lane filters to exclude it, which requires updating the pinned routing contracts. (b) Category=Integration plus a second trait and an amended main filter. © No trait; gate execution on an environment variable so the tests skip by default.

Recommended default: © now, (a) later. An environment-variable skip (SYRF_STATS_DATASET unset means skip) needs no workflow change at all, which keeps Phase 0 documentation-only and touches no pinned contract. If the benchmarks later need to run on a schedule, promote to (a) in the same PR that updates the routing contract.

D3. Whether local prod-snapshot timings are admissible as baselines

Evidence. The snapshot is described in CLAUDE.md and docs/architecture/mongodb-reference.md as a 2026-02-21 restore held in a standalone local container. On 2026-09-01 it was verified unavailable: the mongodb-syrf-local MCP server returned "The configured connection string is not valid", and docker ps -a listed no snapshot container. No production-shaped study-count distribution could therefore be measured for this document, which is why stats-large-v1 is sized by judgement rather than by evidence.

Impact. Snapshot timings would be prod-shaped but are single-host, single-restore, uncontrolled for cache state, and cannot be regenerated identically by a reviewer. Using them as gate baselines would make the 20% and 10% thresholds unreproducible.

Options. (a) Admit snapshot timings as gate baselines. (b) Exclude them entirely. © Admit them as advisory shape checks recorded separately from gate baselines.

Recommended default: ©. The synthetic datasets are the normative baseline for every gate. If the snapshot container is restored, run A8 against it read-only to check that the synthetic datasets' document sizes and per-project study counts are in the right range, and record the result in a clearly separate advisory section that no gate cites. Any such run stays read-only and records only aggregate shape — never a project identifier, title, or person.

D4. Whether any plan bound should be lowered now

Evidence. The reachable screening tally key count is exactly (K + 1)(K + 2) / 2 for a maximum of K decisions on one study, so 50,000 keys require K >= 315. The conservative encoding arithmetic puts 50,000 keys at roughly 1.53 MiB, about one fifth of the 8 MiB document ceiling. Neither figure has yet been validated by an executed test, because C1 needs harness H-5.

Impact. Lowering a hard ceiling on unvalidated arithmetic risks rejecting writes that the real encoding would have admitted; leaving an unreachable ceiling in place risks the guard never firing in any test that uses real data.

Options. (a) Lower the dynamic-key ceiling now. (b) Change nothing in Phase 0. © Keep the 50,000 hard ceiling and add a measured per-family soft limit that is reachable in tests.

Recommended default: ©. Retain 8 MiB and 50,000 as the hard admission ceiling, and propose a screening-family soft limit of 8,192 keys — comfortably above the 7,381 that 120 reviewers can produce, and low enough to be reached by natural source data. PS-DS-04a alone cannot exercise it, because 7,381 keys never reaches the limit; the soft limit is therefore proposed rather than measured until C2 runs PS-DS-04c (K = 126, 8,128 keys, admitted) and PS-DS-04d (K = 127, 8,256 keys, refused) either side of it. Confirm or revise both after C1 and C2 execute. Make no change to the 500-move, 100-document, 256 KiB point-path bounds or the 500-reference, 256 KiB page bounds in Phase 0: the proposals of 128 moves and 32 documents in the capacity model are recorded as candidates and must not be adopted until Phase 1 can measure the real canonicalizer.

D5. Which fixture hosts which measurement

Evidence. MongoDbTestFixture runs mongo:8.0 standalone and cannot serve transactions; MongoDbReplicaSetTestFixture ran mongo:7.0 as a single-node replica set and can. The repository test project that owns the aggregation under test binds the standalone fixture.

Impact. Running read baselines on one server version and write benchmarks on another makes the source-mutation regression gate compare across MongoDB versions.

Options. (a) Move everything to the replica-set fixture. (b) Split by need and accept the version skew. © Split by need and align the replica-set fixture's image to mongo:8.0.

Recommended default: ©. Phase 0's authoritative read baselines need no transactions and should run on the standalone mongo:8.0 fixture that already hosts the code under test. The concurrency protocol needs the replica set — and that is exactly where deferring the image alignment to Phase 1 breaks the gate. B1 runs on the replica-set fixture in Phase 0; if that fixture is still mongo:7.0 and is upgraded to mongo:8.0 before B2 runs in Phase 1, the <10% source-mutation regression gate compares a MongoDB 7 baseline against a MongoDB 8 measurement, which is the comparison this decision exists to prevent. Recording the tags makes the mismatch visible; it does not make the two numbers comparable.

Resolved, and done. MongoDbReplicaSetTestFixture is aligned to mongo:8.0 in this branch, ahead of any B1 run, so both arms of every gate share a server version and the contingency this decision described — re-running B1 in Phase 1 and discarding the 7.0 numbers — does not arise. It was a one-line image change; the fixture's only consumer, src/services/api/SyRF.API.Endpoint.Tests/SubmitAnnotationSessionServiceTests.cs, was run against the new image and passes 15/15 with the replica set started and transactions exercised, so no dependency on 7.0 behaviour exists. Every results artifact still records the image tag and the reported server version, so a cross-version comparison cannot be made by accident even after a future bump. This alignment is a fixture change, not a product change.

D6. How results are recorded and reviewed

Evidence. No results artifact convention exists; the integration lane already uploads .trx and coverage artifacts, but neither carries custom metrics.

Impact. Without a machine-readable artifact, gate evidence exists only in scrollback, and the plan's requirement to compare a post-change p95 against a recorded baseline cannot be audited.

Options. (a) Prose only, appended to this document. (b) A JSON artifact per run plus a summary table appended here. © Push metrics to an observability backend.

Recommended default: (b). H-7 emits one JSON file per run capturing dataset ID, seed, container image, server version, host CPU count, iteration counts and every metric; a short human-readable summary table is appended to this document when each baseline is executed. Option © belongs with the Phase 1 telemetry work, not with the Phase 0 baseline.

Results

Harness status

Harnesses H-1 (deterministic dataset generator), H-2 (duration sampling and percentile reporting), H-3 (round-trip and facet counting) and H-7 (results emitter) are implemented, in src/libs/project-management/SyRF.ProjectManagement.Mongo.Data.Tests/ProjectStatistics/, per decision D1's recommended default (b). Commands A2-A7 therefore run today. H-4 (golden-file parity runner), H-5 (encoded-BSON capacity sizer) and H-6 (concurrency driver) are not implemented, so the PF-* fixtures, C1-C5 and B1/B2 remain unexecuted; PS-DS-04b has no generator because it is a pure encoding fixture consumed only by H-5.

Following decision D2's recommended default ©, the harness carries no xUnit trait and skips unless SYRF_STATS_DATASET is set. No workflow, filter or pinned routing contract was changed, and an ordinary dotnet test run reports these six tests as skipped. The generator's own unit tests do run in the ordinary lanes.

The byte-identity contract stated under Fixed benchmark datasets is enforced rather than asserted. Deriving the generator's own identifiers from (datasetId, seed, entityKind, ordinal) is not sufficient on its own: the domain aggregates mint identifiers of their own (a join request, a screening decision) and stamp wall-clock timestamps (Entity.DateTimeCreated, Project.CreationDate, Audit.Created/LastModified, a session's CreatedAtUtc/ReservedAtUtc/CompletedAtUtc), so two runs persisted different BSON while reporting the same fingerprint. DeterministicCorpusNormalizer now rewrites those identifiers from the dataset seed, repoints every reference to a rewritten one, and freezes every timestamp to one fixed instant while leaving absent timestamps absent — a session's completion time is state, not a clock reading. A unit test serializes two independently generated corpora with the production BsonClassMaps and compares the persisted bytes, so the contract is proved on the corpus that is actually written rather than on the subset the fingerprint happens to cover.

PS-DS-04c and PS-DS-04d are generatable: both specs exist, and unit tests generate each corpus in full and assert it reaches 8,128 and 8,256 distinct tally keys respectively, which is what makes them the pair that straddles the proposed 8,192-key soft limit. C2 itself still cannot run — it needs H-5 (the encoded-BSON capacity sizer), which is not implemented, and its command passes the three dataset IDs as one comma-separated value, which the single-valued SYRF_STATS_DATASET contract does not yet parse. Both belong with H-5 rather than here; the datasets they consume are ready.

There is no option to leave a seeded container resident after a run. MongoDbTestFixture owns the container, disposes it with the collection, and enables neither Testcontainers reuse nor an opt-out from the resource reaper, so the earlier SYRF_STATS_KEEP_CONTAINER flag could only record a request it was unable to honour; it has been removed from the harness and from command A2. To inspect a corpus, point a client at the connection string command A2 prints while the run is still in progress, or generate it into a container you started yourself.

Measured authoritative baselines

One real run of A3-A6, executed on 2026-09-02 against the generator as it stands at the head of this branch — that is, after the reconciliation eligibility floor, which raises the annotation-session counts below relative to any earlier recorded run. Every number below is measured, not estimated, and every one of them is reproduced by the JSON artifacts checked in beside this document at evidence/phase0-baselines/ — one file per dataset per measurement:

Dataset	Artifacts
PS-DS-01	PS-DS-01_FullProjectStats.json, PS-DS-01_ReviewerStats.json, PS-DS-01_ReviewerStatsAssignmentPath.json, PS-DS-01_QuestionTally.json, PS-DS-01_SearchPopulation.json
PS-DS-02	PS-DS-02_FullProjectStats.json, PS-DS-02_ReviewerStats.json, PS-DS-02_ReviewerStatsAssignmentPath.json, PS-DS-02_QuestionTally.json, PS-DS-02_SearchPopulation.json

The artifacts, not the tables below, are the auditable record; the tables are the human-readable summary of the same run. Each file carries the container image, server version, host CPU count, whether the container resource caps were applied, the iteration counts, the generated corpus's row counts, the corpus fingerprint, and every recorded duration and round-trip series. They contain no identifiers and no clinical content: the corpus is synthetic, and the results emitter refuses to write a metric name carrying an entity identifier. A run emits them under the test project's output directory (bin/<configuration>/net10.0/benchmark-results/, or SYRF_STATS_RESULTS_DIR when set); these copies are that run's output, renamed to a stable file name so a later run can be diffed against them.

Environment. Local developer workstation, Ubuntu 22.04 (kernel 6.8.0-124), .NET 10.0.0, 48 host CPUs, Docker 29.6.0. MongoDB mongo:8.0, reported server version 8.0.28, standalone, started by MongoDbTestFixture (decision D5's Phase 0 placement). SYRF_TEST_JOB_KEY was unset, so DockerTestContainerPolicy's 2 vCPU / 2 GiB / 256 PID caps were not applied. These are therefore unconstrained local numbers and are not comparable with a capped CI run; the inventory table already records that caveat.

Corpus actually generated.

Dataset	Studies (total / target project)	Embedded screening decisions	Annotation sessions	Distinct tally keys
PS-DS-01	350 / 300	568	682	10
PS-DS-02	5,400 / 5,000	14,119	24,509	21

Both tally-key counts equal the value the dataset table declares. The screening-decision counts are higher than the table's approximations (450 and 12,500) because the generator guarantees coverage of every reachable (NumberScreened, IncludedCount) key before it fills the zero-screening and conflicting-decision quotas; the table's figures were approximations, and these are the measured truth for this generator version.

Timings. 100 measured iterations after 10 warm-up iterations, nearest-rank percentiles, milliseconds.

Command	Measurement	Dataset	p50	p95	min	max
A3	GetFullProjectStatsAsync	PS-DS-01	69.11	85.73	44.20	100.27
A3	GetFullProjectStatsAsync	PS-DS-02	968.30	1413.97	880.14	1507.51
A4	GetReviewerStatsForStageAsync, screening stage	PS-DS-01	12.46	16.58	6.77	21.34
A4	GetReviewerStatsForStageAsync, screening stage	PS-DS-02	41.79	51.13	33.77	54.72
A4	GetReviewerStatsForStageAsync, annotation stage	PS-DS-01	26.46	32.94	17.46	36.64
A4	GetReviewerStatsForStageAsync, annotation stage	PS-DS-02	60.36	68.66	42.29	74.33
A4	GetReviewerStatsForProjectAsync	PS-DS-01	50.19	60.29	35.89	80.48
A4	GetReviewerStatsForProjectAsync	PS-DS-02	143.87	162.53	115.85	174.95
A4	GetStudyReviewStatus, screening stage	PS-DS-01	15.66	19.38	6.54	21.20
A4	GetStudyReviewStatus, screening stage	PS-DS-02	37.19	45.45	27.02	49.44
A4	GetStudyReviewStatus, annotation stage	PS-DS-01	14.35	18.99	7.03	25.24
A4	GetStudyReviewStatus, annotation stage	PS-DS-02	30.02	40.52	24.66	44.42
A5	GetAnnotationQuestionAnswerTally	PS-DS-01	5.79	8.05	4.12	16.01
A5	GetAnnotationQuestionAnswerTally	PS-DS-02	79.49	113.04	72.19	126.03
A5	search-population sum	PS-DS-01	1.27	1.89	0.91	1.98
A5	search-population sum	PS-DS-02	1.54	1.93	0.85	2.11

Round trips and facets (A6). Counted from a CommandStartedEvent subscriber on the driver's ClusterConfigurator, one un-recorded call first so connection establishment is not counted.

Measurement	Dataset	Driver round trips	$facet sub-pipelines
GetFullProjectStatsAsync	PS-DS-01, PS-DS-02	1	15
GetReviewerStatsForStageAsync, screening stage	PS-DS-01, PS-DS-02	4	0
GetReviewerStatsForStageAsync, annotation stage	PS-DS-01, PS-DS-02	8	0
GetReviewerStatsForProjectAsync	PS-DS-01 (3 stages)	25	0
GetReviewerStatsForProjectAsync	PS-DS-02 (5 stages)	39	0
GetStudyReviewStatus, either review mode	PS-DS-01, PS-DS-02	4	0
GetAnnotationQuestionAnswerTally	PS-DS-01, PS-DS-02	1	0
search-population sum	PS-DS-01, PS-DS-02	1	0

The measured 15 sub-pipelines confirm the catalogue's description of the full-project statistics query as a single 15-facet $facet. The reviewer-stats fan-outs are the opposite shape: one countDocuments per counted predicate, which the driver emits as an aggregate command, so the project-scoped call grows linearly with stage count — 25 round trips over three stages and 39 over five. That is the concrete pre-change figure the plan's "at least 80% fewer authoritative statistics aggregations" gate must be measured against.

Measuring the stage-scoped call once per review mode was not a formality: the annotation branch issues eight round trips where the screening branch issues four, and is correspondingly slower at both scales. A single stage-scoped number would have understated the annotation path by a factor of two. GetStudyReviewStatus issues four counts in both review modes, so its round-trip cost does not vary with review mode even though its duration does.

What these numbers do and do not establish

• They are the authoritative-side baseline for the first two performance gates on PS-DS-01 and PS-DS-02. PS-DS-03's confirming run, and every gate whose subject is projection code, remain unexecuted because that code does not exist.
• They were taken on an uncapped local host. Before any gate is judged, the same commands must be rerun in the environment that will host the comparison, and both runs' recorded containerImage, serverVersion, hostProcessorCount and containerResourceCapsApplied fields must match.
• Each recorded artifact carries the corpus fingerprint of the run that produced it. A later run whose fingerprint differs is measuring a different corpus, and its numbers are not comparable with these however similar they look. Every PS-DS-01 artifact above records F11E12925C4D4F512244F702F58A8BEA97185C558349BA8670FA7BD76406DB95 and every PS-DS-02 artifact D0D56518026AAE4659B141C95E6F7D5380FA4084E20A2A618F565BA41F4A56EA. Earlier runs of the same commands in separate processes agreed with each other on the value for the generator version they ran, which is the reproducibility contract holding across processes rather than only within one; the value changed here because the reconciliation eligibility floor changed the corpus, which is exactly what a fingerprint is for.