FEAT-024 Phase 0 Calculation and Consumer Catalogue¶
Purpose and Phase 0 gate¶
This document is one of the two method-level deliverables required by the separate documentation-only
Phase 0 completion review described in
Phase 0: inventory, catalogue, baselines, and architecture.
It names every known current project-statistics calculation on main, gives each exactly one
disposition against the family table in
Catalogue baseline, names every consumer of each calculation,
and records the exact current formula, bounded profile keys, permitted overlaps, invariants, rounding
rule and authorization rule for every approved family, as required by
Catalogue rules and
Bounded study-state profile distributions.
It records evidence only. It does not authorize Phase 1, does not re-decide anything the technical plan
fixes, and does not propose product code. Where this catalogue and the technical plan disagree, the plan
wins and the discrepancy is recorded in Section 5.
Evidence base. Every path:line citation below was re-verified by reading the file in the main
worktree at /home/chris/workspace/syrf/main, commit 7e0ed90c9 (ci: migrate high-value Actions jobs
to self-hosted runners (#3027)), on 2026-09-01. Paths are repo-relative. Anchors that the earlier
read-only research passes recorded incorrectly are listed in
Appendix A: corrections applied. No repository file was modified while
producing this document, and it contains no clinical data, report contents or participant identifiers.
1. Disposition table¶
One row per known current calculation. The Catalogue family column uses the exact family names from
the plan's Catalogue baseline table. Every row carries exactly
one disposition; rows marked Needs-decision are cross-referenced to
Section 5.
| Calculation | Owner method (file:line) | Consumers | Catalogue family | Phase 0 disposition | Evidence |
|---|---|---|---|---|---|
Project screening counts and screening-agreement matrix (ProjectScreeningStats) |
StudyStatsQuery.CreateProjectScreeningStats — src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/StudyStats.cs:252, fed by the All facet (:274) and the ScreeningGroupedCount facet (:429) |
GET api/projects/full-stats; GET api/projects/{projectId}/stages/full-stats; ProjectStatsNotification SignalR push; Angular project-screening-stats entity slice; ScreeningOverviewComponent; StageOverviewComponent |
Project screening | Approved-authoritative | §3.1 |
Per-membership screening counts (MembershipScreeningStats) |
StudyStatsQuery.CreateMembershipScreeningStats — StudyStats.cs:186, fed by the Investigator facet (:278) |
Same FullStats consumers; Angular membership-screening-stats entity slice; selectScreeningMembershipData named leaderboard |
Membership screening | Approved-authoritative | §3.2 |
Per-membership screening SufficientlyIncludedAndAgree argument |
StudyStats.cs:199-200 |
As above | Membership screening | Needs-decision (D3) | StudyStats.cs:199-202 passes SufficientlyExcludedAndAgree into both the sufficientlyIncludedAndAgree and sufficientlyExcludedAndAgree constructor positions of MembershipScreeningStats (src/libs/project-management/SyRF.ProjectManagement.Core/Model/ValueObjects/MembershipScreeningStats.cs:15-16) |
Stage-scoped annotation session distribution (StageAnnotationStats) |
StudyStatsQuery.CreateStageAnnotationStats — StudyStats.cs:235, via CreateAnnotationSessionStats (:221), fed by UnexcludedSessioned/ExcludedSessioned (SessionedGroupStage, :286) and the two CandidateSessionsStartedGroupedCount histograms |
Same FullStats consumers; Angular stage-annotation-stats entity slice; StageOverviewComponent; ProjectOverviewComponent |
Stage annotation | Approved-authoritative | §3.3 |
Membership × stage annotation distribution (MembershipStageStats) |
StudyStatsQuery.CreateMembershipStageAnnotationStats — StudyStats.cs:132, local function CreateMembershipAnnotationStats (:152-171), fed by InvestigatorSessionGroupStage (:283) |
Same FullStats consumers; Angular membership-annotation-stats entity slice; StageOverviewComponent |
Membership-stage annotation | Approved-authoritative | §3.4 |
Excluded-session regrouping views (MergedMembershipAnnotationStats, SeperatedExcludedMembershipAnnotationStats) |
src/libs/project-management/SyRF.ProjectManagement.Core/Model/ValueObjects/MembershipAnnotationSessionStats.cs:54-112 and :114-140, constructed eagerly at MembershipStageStats.cs:22 |
Same FullStats consumers |
Project/stage derived summaries | Derived | §3.9 |
Percent-of-total presentation values (PercentAvailable, PercentCompleted, PercentIncluded, …) |
MembershipAnnotationSessionStats.cs:39-51 and MembershipScreeningStats.cs:74-86 |
Same FullStats consumers |
Project/stage derived summaries | Derived | §3.9 |
| Reconciliation availability/started/in-progress/completed counters | SessionedGroupStage — StudyStats.cs:286; InvestigatorSessionGroupStage — StudyStats.cs:283 |
Same FullStats consumers |
Domain reconciliation | Approved-authoritative | §3.10 |
Dedicated reconciliation facets Unexcluded/ExcludedStartedReconciliationCount and …CompletedReconciliationCount |
StudyStats.cs:384-397, :414-427; assigned to AllInfo.cs:36-40 |
none — never read when constructing FullStats |
Domain reconciliation | Dead query work, excluded from the approved family | §3.10 |
Per-reviewer live screening counts (ReviewerScreeningStats) |
StudyRepository.GetInvestigatorScreeningStats — src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Repositories/StudyRepository.cs:287-306 |
GET api/projects/{projectId}/stages/reviewer-stats; GET api/projects/{projectId}/stages/{stageId}/reviewer-stats; Angular reviewer-screening-stats entity slice; ReviewCompletedComponent; ProjectOverviewComponent progress bars |
Reviewer screening | Alias-of-Membership screening (with a divergence, see D4) | §3.5 |
Per-reviewer live annotation counts (ReviewerAnnotationStats) |
StudyRepository.GetInvestigatorAnnotationStats — StudyRepository.cs:218-283 |
Same reviewer-stats consumers | Reviewer annotation | Alias-of-Membership-stage annotation (with a divergence, see D4) | §3.6 |
| Whole-project study count used as every reviewer-stats denominator | StudyRepository.GetReviewerStatsForStageAsync:189 and GetReviewerStatsForProjectAsync:205 (CountDocumentsAsync(Filters.InProject(project.Id))) |
Same reviewer-stats consumers | Project/stage derived summaries | Derived | StudyRepository.cs:185-216 |
| Annotation question answer tally (per question: distinct studies + total annotations) | StudyRepository.GetAnnotationQuestionAnswerTally — StudyRepository.cs:947-964; refreshed by ProjectManagementService.UpdateAnnotationQuestionAnswerTally — src/libs/project-management/SyRF.ProjectManagement.Core/Services/ProjectManagementService.cs:514-520; stored on Project.AnnotationQuestionAnswerTally — src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectAggregate/Project.cs:205 |
PUT api/projects/{projectId}/update-annotation-answer-tally; every AnnotationQuestionDto.AnsweredAnnotationCount on every ProjectDetailsDto; Angular design.store.ts isLocked gate; assign.store.ts |
Question answers | Approved-authoritative (replace the manual refresh only after catalogue approval) | §3.7 |
AnnotationQuestion.AnsweredAnnotationCount read-through |
src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectAggregate/AnnotationQuestion.cs:157-158, via Project.AnnotationAnswerTallyDictionary (Project.cs:207-209) |
AnnotationQuestionDto on every project-details response and the SignalR project push |
Question answers | Derived | Project.cs:205-216 |
Per-search imported-record count (SystematicSearch.NumberOfStudies) |
src/libs/project-management/SyRF.ProjectManagement.Core/Model/SystematicSearchAggregate/SystematicSearch.cs:42 (SyrfReferenceFiles.Sum(...)), loaded via SystematicSearchRepository.GetAllSystematicSearchesLinkedToProject — src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Repositories/SystematicSearchRepository.cs:30-33 |
GET api/projects/{projectId}/searches; SearchResolver on every ProjectDetailsDto; systematic-searches admin table |
Search/population | Approved-authoritative | §3.8 |
Project-level studyCount total |
Angular selector only — src/services/web/src/app/core/state/entities/project/project-detail.viewmodel.ts:99 (_.sumBy(searches, s => s.numberOfStudies)) |
project-overview.component.html; living-searches.component.html |
Search/population | Derived (client-side only; no server equivalent ships) | §3.8 |
| Server-side project study-count sum | SystematicSearchRepository.NumberOfStudiesInProject — SystematicSearchRepository.cs:23-28; declared at src/libs/project-management/SyRF.ProjectManagement.Core/Interfaces/ISystematicSearchRepository.cs:11 |
None | Search/population | Dead-code-retire (D6) | Only two non-test matches for NumberOfStudiesInProject in src, both the declaration and the implementation |
Per-study absolute agreement ratio (StudyAgreementMeasure) |
StudyAgreementMeasure.New(IEnumerable<Screening>) — src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/StudyAgreementMeasure.cs:22-35; surfaced as the computed, persisted ScreeningInfo.AgreementMeasure — src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/ScreeningInfo.cs:67 |
AllScreeningPipelineGroupStage/InvestigatorScreeningPipelineGroupStage read the persisted BSON fields directly (StudyStats.cs:274, :278); ProjectAgreementThreshold.GetScreeningCompleteness (ProjectAgreementThreshold.cs:53-61); reconciliation-pool sort keys in StudyRepository |
Project screening | Approved-authoritative (source input, not itself a materialized metric) | §3.1 |
| Screening completeness / inclusion classification | ProjectAgreementThreshold.GetScreeningCompleteness — ProjectAgreementThreshold.cs:53-61; ProjectAgreementThreshold.GetInclusionStatus — ProjectAgreementThreshold.cs:63 |
ScreeningInfo.InclusionInfo projection (ScreeningInfo.cs:52-58); StageReviewService; every NewStudyFilters predicate |
Project screening | Derived | §3.1 |
Persisted per-threshold inclusion partition (ScreeningInfo.InclusionInfo[]) |
Written by StudyRepository.UpdateStudyInclusionInfoForProjectAsync — StudyRepository.cs:1224-1249; read by NewStudyFilters.SufficientlyScreenedStudies/ExcludedStudies/IncludedStudies/InsufficientlyScreenedStudies — src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/Filters.cs:471, :490, :507, :524 |
Every annotation family's excluded/unexcluded partition; both reviewer-stats calculations; export inclusion counts | Project screening | Approved-authoritative (source input) | §3.1 |
Next-study assignment availability counts (ReviewStatusStats → ReviewStatusInfo) |
StudyRepository.GetStudyReviewStatus — StudyRepository.cs:322-360; collapsed to booleans at src/libs/project-management/SyRF.ProjectManagement.Core/Model/ReviewStatusStats.cs:26-44 |
StageReviewService.GetRandomStudyAsync only; raw counts never reach a response |
Operational progress/presence | Excluded-outside-programme | The record is consumed only to choose the next study; ReviewStatusInfo exposes booleans, not counts |
| In-progress-count helpers | StudyRepository.GetUnexcludedStudiesInProgressCountAsync/GetExcludedStudiesInProgressCountAsync — StudyRepository.cs:362-382 |
GetStudyReviewStatus; StageReviewService.HasStageReachedMaxInProgress |
Operational progress/presence | Excluded-outside-programme | Assignment/capacity gating, not a reported statistic |
| Export paging counts | StudyRepository.GetStudiesForProjectWithCountAsync / GetIncludedStudiesForProjectWithCountAsync — StudyRepository.cs:577-664 |
DataExportController.GetDataExport — src/services/api/SyRF.API.Endpoint/Controllers/DataExportController.cs:73-178 |
Operational progress/presence | Excluded-outside-programme | Sizes a paged cursor and a progress percentage; GetIncludedStudiesForProjectWithCountAsync does independently apply NewStudyFilters.IncludedStudies, noted in D7 |
| Export row-progress counter | OptimizedDataExportProgressReporter usage in CsvDataExportWriter (++studyCount) |
Export progress UI | Operational progress/presence | Excluded-outside-programme | Job progress, explicitly outside the programme per the plan |
| Investigator engagement counters | InvestigatorUsage.RecordSignIn/RecordProjectAccess/RecordStageAccess — src/libs/project-management/SyRF.ProjectManagement.Core/Model/InvestigatorUsageAggregate/InvestigatorUsage.cs:31, :48, :68 |
ApplicationService.RecordSignInAsync; ApplicationController |
Operational progress/presence | Excluded-outside-programme | Sign-in and page-visit counters; no screening/annotation input |
| Reconciliation-pool random selection | StudyRepository.NextRandomStudyAvailableForReconciliation — StudyRepository.cs:309-320; NextRandomStudyAvailableAndUnstartedForReconciliationAsync — StudyRepository.cs:384-396 |
StageReviewService |
Operational progress/presence | Excluded-outside-programme | .Sample(1).FirstOrDefault() selection; no count is produced |
Blocking FullStats value resolver |
ProjectStatsResolver.Resolve — src/services/api/SyRF.API.Endpoint/Models/ValueResolvers/ProjectStatsResolver.cs:11-29 (.Result at :27) |
None | Project screening | Dead-code-retire (D5) | ProjectDetailsDto.CreateMappings (src/services/api/SyRF.API.Endpoint/Models/ProjectDto/ProjectDetailsDto.cs:44-64) never calls MapFrom<ProjectStatsResolver>(), and ProjectDetailsDto has no FullStats member |
| Hardcoded annotation summary | AnnotationSummaryResolver.Resolve — src/services/api/SyRF.API.Endpoint/Models/ValueResolvers/AnnotationSummaryResolver.cs:9-27; body is return new(2, 0.333, 23, 5, 3, 24); at :25 |
None | Stage annotation | Stub (D5) | Never wired via ForMember; ProjectDetailsDto has no AnnotationSummaryDto member |
| Abandoned daily-history scaffold | ProjectDailyStat / StageDailyStat — src/libs/project-management/SyRF.ProjectManagement.Core/Model/ProjectDailyStatsAggregate/ProjectDailyStat.cs:6 |
None | Project/stage derived summaries | Dead-code-retire | The only three matches for ProjectDailyStat in src are all inside that one file; no repository, service, controller, interface or test references it |
| Abandoned membership-stats scaffold | StudyRepository.StageStatsQueryResult — StudyRepository.cs:555; OldMembershipAnnotationStats — src/libs/project-management/SyRF.ProjectManagement.Core/Model/ValueObjects/OldMembershipAnnotationStats.cs |
None | Membership-stage annotation | Dead-code-retire | StageStatsQueryResult has exactly one match in src — its own declaration; never constructed or consumed |
| Placeholder per-reviewer history selectors | selectScreeningDataForInvestigator — src/services/web/src/app/core/state/entities/membership-screening-stats/membership-screening-stats.selectors.ts:132; selectAnnotationDataForInvestigator — .../membership-annotation-stats/membership-annotation-stats.selector.ts:122 |
None (both unconditionally return null) |
Membership screening / Membership-stage annotation | Stub | Both sit behind a TODO: History time-series data … not yet implemented on the backend comment |
| Dead screening-summary action pair | projectDetailActions.getScreeningSummary / screeningSummaryReceived — src/services/web/src/app/core/actions/project-detail.actions.ts:299-300 |
Listed in all four FullStats entity reducers' on() clauses, but never dispatched or emitted |
Project screening | Dead-code-retire (D8) | No effect or component dispatches getScreeningSummary; nothing emits screeningSummaryReceived |
No-op receivedFullStats UI reducer handler |
src/services/web/src/app/core/state/ui/stage/stage-ui.reducer.ts:48-50 |
— | Project screening | Dead-code-retire (D8) | Handler body is (state, props) => ({...state}) |
| Inter-rater agreement / Cohen's kappa | — | — | Agreement/kappa | Excluded-outside-programme | grep -rn "Kappa\|Cohen" across all src/**/*.cs returns zero matches; see Section 4 |
| Outcome-level cross-study statistics | — | — | Outcome-level statistics | Excluded-outside-programme | No cross-study outcome aggregate exists; see Section 4 |
| PRISMA source-type counts | — | — | Search/population | Excluded-outside-programme | SourceType has no matches in src/libs/project-management, src/services/api or src/services/project-management; PRISMA remains [TARGET - Phase 12] |
2. Consumer table¶
One row per consumer. Post-programme role states whether the consumer becomes a materialized consumer behind its own flag, stays on the authoritative calculation, or is retired.
| Consumer | Type | File:line | Calculation(s) read | Authorization observed today | Post-programme role |
|---|---|---|---|---|---|
ReviewController.GetFullStats (GET api/projects/{projectId}/stages/full-stats, Name = "GetStats") |
REST endpoint | src/services/api/SyRF.API.Endpoint/Controllers/ReviewController.cs:458-477 |
Project screening, Membership screening, Stage annotation, Membership-stage annotation, Domain reconciliation | No [Authorize(Policy)] on the method or the class; only the global RequireAuthenticatedUser() filter (src/services/api/SyRF.API.Endpoint/Program.cs:126-132) |
Materialized consumer behind its own flag; the missing policy is D2 |
ProjectController.GetFullStats (GET api/projects/full-stats) |
REST endpoint | src/services/api/SyRF.API.Endpoint/Controllers/ProjectController.cs:779-787 |
Same, investigator-scoped for the membership blocks only | Same global filter only | Materialized consumer behind its own flag; first Phase 5 cutover candidate |
NotificationHub.SubscribeToProjectFullStats |
SignalR hub method | src/services/api/SyRF.API.Endpoint/SignalR/NotificationHub.cs:1027-1035 |
Same as ProjectController.GetFullStats, re-run on every project/study notification via AggregateRootEntitySubscriptionManager.SubscribeToFullStatsForProjectAndInvestigator |
Class-level [Authorize] only (NotificationHub.cs:31); the sibling SubscribeToProject does carry [Authorize(ProjectAuthorization.ProjectViewSignalRPolicy)] at NotificationHub.cs:934 |
Replaced by revision/scope invalidation publication; the client fetches an authorized snapshot |
ProjectStatsNotification client callback |
SignalR client push | src/services/web/src/app/core/services/signal-r/signal-r.service.ts:792-810 |
FullStats |
— | Retired in its current form; becomes an invalidation listener |
ReviewController.GetReviewerStatsForStage |
REST endpoint | ReviewController.cs:507-527 |
Reviewer screening, Reviewer annotation | No [Authorize(Policy)]; membership is enforced indirectly because project.GetMembershipFor(investigatorId) throws for a non-member (StudyRepository.cs:279, :305) |
Materialized consumer after the reviewer families are reconciled against the membership families |
ReviewController.GetReviewerStatsForProject |
REST endpoint | ReviewController.cs:529-542 |
Reviewer screening, Reviewer annotation | Same | Materialized consumer |
ProjectController.UpdateAnnotationAnswerTally (PUT .../update-annotation-answer-tally) |
REST endpoint (refresh trigger) | ProjectController.cs:652-658 |
Question answers (write) | [Authorize(ProjectAuthorization.ProjectDesignPolicy)] at :653 |
Retired once the transactional question materializer is cut over |
ProjectDetailsDto.AnnotationQuestions[].AnsweredAnnotationCount |
REST DTO field (wide fan-out) | src/services/api/SyRF.API.Endpoint/Models/ProjectDto/ProjectDetailsDto.cs:27; SignalR push at NotificationHub.cs:946 |
Question answers (read) | Per-endpoint project policies | Materialized consumer |
SearchController.GetProjectSearches (GET api/projects/{projectId}/searches) |
REST endpoint | src/services/api/SyRF.API.Endpoint/Controllers/SearchController.cs:66-75 |
Search/population | [Authorize(ProjectAuthorization.ProjectViewPolicy)] |
Stays authoritative until Phase 2/4 |
SearchResolver → ProjectDetailsDto.Searches |
AutoMapper resolver (wired) | src/services/api/SyRF.API.Endpoint/Models/ValueResolvers/SearchResolver.cs:11-28, wired at ProjectDetailsDto.cs:50 |
Search/population | Per-endpoint project policies | Stays authoritative until Phase 2/4 |
DataExportController.GetDataExport |
REST endpoint (export) | src/services/api/SyRF.API.Endpoint/Controllers/DataExportController.cs:73-178 |
Export paging counts only | Per-endpoint export policies | Stays authoritative; no statistics dependency (D7) |
projectCanMatchGuard |
Angular route guard | src/services/web/src/app/core/services/project/project-guard.service.ts:82-89 and :90-100 |
Triggers retrieveProjectReviewerStats and retrieveFullProjectStats |
Route guard plus backend auth | Unchanged trigger; the underlying fetch becomes a materialized consumer |
fullProjectStatsRecieved$ effect |
ngrx effect | src/services/web/src/app/core/services/project/project-detail.effects.ts:716-740 |
FullStats via GET api/projects/full-stats |
— | Materialized consumer |
projectReviewerStats$ effect |
ngrx effect | project-detail.effects.ts:1527-1550 |
ProjectReviewerStats |
— | Materialized consumer |
stageReviewerStats$ effect |
ngrx effect | src/services/web/src/app/core/services/stage/review-effects.ts:273-304 |
StageReviewerStats |
— | Materialized consumer |
getStatsAfterSessionSave$ effect |
ngrx effect (poll-on-mutation) | review-effects.ts:517-544 |
StageReviewerStats |
— | Replaced by revision/scope invalidation |
receiveStudyAndRequestStats$ helper |
ngrx effect | review-effects.ts:134-202 and its callers |
StageReviewerStats |
— | Replaced by revision/scope invalidation |
stageReviewerStatsRequestedAfterProjectUpdated$ / …AfterStageAdded$ |
ngrx backfill effects | project-detail.effects.ts:679-714 and :665-677 |
Reviewer screening/annotation | — | Replaced by revision/scope invalidation |
project-screening-stats / stage-annotation-stats / membership-screening-stats / membership-annotation-stats reducers |
ngrx entity reducers | src/services/web/src/app/core/state/entities/** |
FullStats graph |
— | Materialized consumers; the dual-producer merge issue is D8 |
reviewer-screening-stats / reviewer-annotation-stats reducers |
ngrx entity reducers | .../reviewer-screening-stats/reviewer-screening-stats.reducer.ts:9-17 and .../reviewer-annotation-stats/reviewer-annotation-stats.reducer.ts:9-17 |
Reviewer families, fed by both the stage- and project-scoped endpoints | — | Materialized consumers |
ProjectOverviewComponent |
Angular component | src/services/web/src/app/project/project-overview/project-overview.component.ts:48, :52, :114, :140-142 |
Stage annotation, Membership screening, Reviewer families, Search/population (studyCount) |
Route guard only | Materialized consumer (first Phase 5 page) |
ScreeningOverviewComponent |
Angular component | src/services/web/src/app/screening/screening-overview/screening-overview.component.ts:51, :53, :55 |
Project screening, Membership screening | Route guard only | Materialized consumer; leaderboard visibility is D1 |
StageOverviewComponent |
Angular component | src/services/web/src/app/stage/stage-overview/stage-overview.component.ts:87-100 |
Project screening, Stage annotation, Membership screening, Membership-stage annotation | Route guard only | Materialized consumer; leaderboard visibility is D1 |
ReviewCompletedComponent |
Angular component | src/services/web/src/app/stage/stage-review/review-completed/review-completed.component.ts:37-54 |
Reviewer screening, Reviewer annotation | Route guard only | Materialized consumer |
design.store.ts question edit-lock |
Angular store | src/services/web/src/app/project/project-admin/question-management/design/design.store.ts:295, :528-533 |
Question answers (isLocked derived from answeredAnnotationCount.annotations > 0) |
Route guard only | Materialized consumer; the staleness risk is D9 |
assign.store.ts |
Angular store | .../question-management/assign/assign.store.ts:220-221 |
Question answers | Route guard only | Materialized consumer |
NavComponent.updateAnnotationAnswerTally |
Angular component (manual refresh button) | src/services/web/src/app/core/nav/nav.component.ts:223-230, bound at nav.component.html:202 and :658 |
Question answers (write trigger) | Backend ProjectDesignPolicy |
Retired with the manual refresh path |
systematic-searches.component.html |
Angular template | src/services/web/src/app/project/project-admin/systematic-searches/systematic-searches.component.html:72-81 |
Search/population | Route guard only | Stays authoritative until Phase 2/4 |
project-overview.component.html study total |
Angular template | .../project-overview.component.html:220-229 |
Search/population studyCount |
Route guard only | Stays authoritative until Phase 2/4 |
stageActions.retrieveFullStats / review-effects.ts fullStats$, stageLoad$ |
Commented-out ngrx effects | review-effects.ts:499-515 and :546-570; action creators at src/services/web/src/app/core/actions/stage.actions.ts:58-69 |
Would have read ReviewController.GetFullStats |
— | Retired; ReviewController.GetFullStats currently has no live frontend caller |
| Export/report writers | — | src/libs/project-management/SyRF.ProjectManagement.Core/Services/DataExportServices/DataFormatWriters/** |
None of the six families | — | Unaffected; per-study row writers with no cross-study aggregation |
3. Per-family exact formulas¶
Notation used throughout: NS is ScreeningInfo.AgreementMeasure.NumberScreened, AAR is
ScreeningInfo.AgreementMeasure.AbsoluteAgreementRatio, INC is ScreeningInfo.Inclusion, T is
Project.AgreementThreshold.NumberScreened, R is Project.AgreementThreshold.AbsoluteAgreementRatio
(nullable; the pipeline substitutes -1 when it is null — StudyStats.cs:274, :278), OSB is the
hardcoded overscreening lower bound 3 (StudyStats.cs:294), and MNS is the hardcoded annotation
minimum session count 2 (StudyStats.cs:313).
Shared source-field definitions:
ScreeningInfo.IncludedCount—ScreeningInfo.cs:64— the number of that study's screenings whoseScreeningDecisionisIncluded.ScreeningInfo.Inclusion—ScreeningInfo.cs:70-72—IncludedCount / (double)NumberOfScreenings, andnullwhen the study has no screenings.StudyAgreementMeasure.New—StudyAgreementMeasure.cs:22-35— for a non-empty screening list of one study,numberScreened = count, andabsoluteAgreementRatio = 2 * |includedCount / (double)numberScreened − 0.5|; for an empty list it returns(ratio = 1, numberScreened = 0)(StudyAgreementMeasure.cs:17-20). It throws when the supplied screenings do not all belong to one study, so this is explicitly a per-study, never a cross-study or cross-stage, measure.
3.1 Project screening¶
Scope key. ProjectId.
Bounded profile keys. Count, SufficientlyScreened, InsufficientlyScreened,
SufficientlyIncluded, SufficientlyExcluded, StartedScreening, Overscreened,
OverscreenedYetInsufficientlyScreened, OverscreenedAndSufficientlyIncluded,
OverscreenedAndSufficientlyExcluded, plus the two-dimensional ScreeningGroupedCount distribution
keyed by (NumberScreened, IncludedCount). Constructor mapping at StudyStats.cs:252-272; value object
at src/libs/project-management/SyRF.ProjectManagement.Core/Model/ValueObjects/ProjectScreeningStats.cs.
Exact formulas (AllScreeningPipelineGroupStage, StudyStats.cs:274-276; one $group over the
whole matched project, _id: "$ProjectId"):
| Profile key | Formula |
|---|---|
Count |
$sum: 1 over every Study in the project |
SufficientlyScreened |
NS >= T ∧ AAR > R |
InsufficientlyScreened |
¬(NS >= T ∧ AAR > R) — the exact complement, so these two always sum to Count |
SufficientlyIncluded |
NS >= T ∧ AAR > R ∧ INC > 0.5 |
SufficientlyExcluded |
NS >= T ∧ AAR > R ∧ INC < 0.5 |
StartedScreening |
NS > 0 |
Overscreened |
NS > OSB (OSB = 3) |
OverscreenedYetInsufficientlyScreened |
NS > OSB ∧ AAR < R |
OverscreenedAndSufficientlyIncluded |
NS > OSB ∧ AAR > R ∧ INC > 0.5 |
OverscreenedAndSufficientlyExcluded |
NS > OSB ∧ AAR > R ∧ INC < 0.5 |
ScreeningGroupedCount[ns][inc] |
$group by (ScreeningInfo.AgreementMeasure.NumberScreened, ScreeningInfo.IncludedCount), value $count — StudyStats.cs:429-437; reshaped to a nested dictionary at StudyStats.cs:265-270 |
Configuration inputs. Project.AgreementThreshold.NumberScreened and .AbsoluteAgreementRatio
(read at StudyStats.cs:292-295). OSB and the strict comparison operators are hardcoded.
Exclusivity and permitted overlaps. SufficientlyScreened and InsufficientlyScreened are strict
complements and partition Count. SufficientlyIncluded and SufficientlyExcluded are mutually
exclusive but are not exhaustive over SufficientlyScreened: a study with INC exactly 0.5 —
which is precisely the valid mixed case in which different reviewers recorded include and exclude
decisions in equal numbers — satisfies neither, because both comparisons are strict. The plan requires
that combination to be preserved and tallied rather than labelled impossible
(Bounded study-state profile distributions),
and the ScreeningGroupedCount distribution already represents it exactly: it is the cell
ScreeningGroupedCount[ns][ns/2] for even ns. The Overscreened* family overlaps the
Sufficiently* family by construction — a study may be counted in both — so the materialized contract
must not treat them as a partition. StartedScreening overlaps everything with NS > 0.
Application invariants that make a state impossible. Only two are enforced in code, and both are
narrow. First, a screener can hold at most one Screening per study per project: ScreeningInfo.ScreenStudy
(ScreeningInfo.cs:111-142) branches on HasBeenScreenedByScreener and mutates the existing record
through screening.ChangeScreeningDecision(decision, stageId) (:126) rather than appending, and
throws InvalidOperationException if it ever finds two (:128-133). Therefore NS is also the number
of distinct screeners, and IncludedCount <= NS. Second, AAR is a pure function of IncludedCount
and NS, so the pair (NS, IncludedCount) fully determines AAR and INC; a profile whose AAR is
inconsistent with its (NS, IncludedCount) cell cannot exist. Nothing else is enforced: in particular
no invariant forbids a study being Overscreened while InsufficientlyScreened, and none forbids
NS > 0 with AAR = 0.
Tolerance and rounding. AAR is computed in double and then narrowed to float before storage
(StudyAgreementMeasure.cs:33-34, (float) absoluteAgreementRatio), while the threshold R is
compared as the stored float in a MongoDB $gt. Every metric in this family is an integer count, so
Catalogue rules requires exact equality for the counts themselves;
the float narrowing is a property of the input classification, not of a reported ratio, and any
materialized reclassification must reproduce the same double-computed, float-narrowed value before
comparing, or it will disagree at the boundary. No reported value in this family is a ratio, so no
tolerance is defined and none may be assumed.
Authorized scope and visibility. Served today with no membership check at all — see D2. The plan's target is the existing project-view boundary plus the metric-specific graph permission (Authorization).
Note on the partition used by the annotation families. The annotation families do not partition
studies with the live AAR/INC fields above; they use NewStudyFilters.ExcludedStudies
(Filters.cs:490-504), which matches the persisted ScreeningInfo.InclusionInfo[] element whose
ProjectAgreementThreshold.NumberScreened and .AbsoluteAgreementRatio equal the project's current
threshold and whose SufficientlyExcluded flag is set. That element is written only by
StudyRepository.UpdateStudyInclusionInfoForProjectAsync (StudyRepository.cs:1224-1249). This is the
mechanical reason the plan treats ActiveInclusionInfoCalculationJob as a source-visibility fence
(Event and invalidation contract): while those
three passes are mid-flight, the persisted partition is partial and every annotation family reads a
partial population.
3.2 Membership screening¶
Scope key. (ProjectId, MembershipId), carrying InvestigatorId
(MembershipScreeningStats.cs:52-54).
Bounded profile keys. Screened, Included, Excluded, SufficientlyIncluded,
SufficientlyExcluded, SufficientlyIncludedAndAgree, SufficientlyExcludedAndAgree,
SufficientlyIncludedAndDisagree, SufficientlyExcludedAndDisagree, SufficientlyScreened,
InsufficientlyScreened, Overscreened, OverscreenedYetInsufficientlyScreened,
OverscreenedAndSufficientlyIncluded, OverscreenedAndSufficientlyExcluded, Available,
Unavailable, TotalNumberStudies (MembershipScreeningStats.cs:55-72).
Exact formulas. The Investigator facet (InvestigatorScreeningPipelineGroupStage,
StudyStats.cs:278-281) unwinds ScreeningInfo.Screenings, optionally matches
Screenings.ScreenerId == investigatorId (StudyStats.cs:300-302), and groups by
ScreeningInfo.Screenings.ScreenerId. Let D be the unwound screening's ScreeningDecision
(1 = Included, 0 = Excluded). Per screener:
| Facet field | Formula |
|---|---|
ScreenedCount |
$sum: 1 over that screener's screening rows |
IncludedCount |
D = 1 |
ExcludedCount |
D = 0 |
SufficientlyIncluded |
NS >= T ∧ AAR > R ∧ INC > 0.5 |
SufficientlyExcluded |
NS >= T ∧ AAR > R ∧ INC < 0.5 |
SufficientlyIncludedAndAgree |
NS >= T ∧ AAR > R ∧ INC > 0.5 ∧ D = 1 |
SufficientlyExcludedAndAgree |
NS >= T ∧ AAR > R ∧ INC < 0.5 ∧ D = 0 |
SufficientlyIncludedAndDisagree |
NS >= T ∧ AAR > R ∧ INC > 0.5 ∧ D = 0 |
SufficientlyExcludedAndDisagree |
NS >= T ∧ AAR > R ∧ INC < 0.5 ∧ D = 1 |
SufficientlyScreened / InsufficientlyScreened / Overscreened* |
Identical predicates to §3.1, evaluated over that screener's rows |
The C# mapping (StudyStats.cs:186-219) then composes the wire row per membership:
Screened,Included,Excluded← that screener'sScreenedCount,IncludedCount,ExcludedCount(StudyStats.cs:194-196).SufficientlyIncludedAndDisagree,SufficientlyExcludedAndDisagree← that screener's equivalents (:203-206).SufficientlyIncluded,SufficientlyExcluded,SufficientlyScreened,InsufficientlyScreened,Overscreened,OverscreenedYetInsufficientlyScreened,OverscreenedAndSufficientlyIncluded,OverscreenedAndSufficientlyExcluded,TotalNumberStudies← the project-wideAllfacet, not the screener's own values (:197-198,:207-212,:217). These eight fields are therefore identical for every membership row in a response; the catalogue must record them as project projections carried on a membership row, not as per-reviewer metrics.Available=All.InsufficientlyScreened − thisScreener.InsufficientlyScreened(:213-214).Unavailable=All.SufficientlyScreened − thisScreener.SufficientlyScreened(:215-216).SufficientlyIncludedAndAgreeandSufficientlyExcludedAndAgreeare both fed the screener'sSufficientlyExcludedAndAgree(:199-202) — see D3.
Exclusivity and permitted overlaps. For one screener, IncludedCount and ExcludedCount partition
ScreenedCount when ScreeningDecision has only those two values. The four
Sufficiently{Included,Excluded}And{Agree,Disagree} cells are mutually exclusive and are jointly
contained in — not equal to — SufficientlyScreened, because a study with INC = 0.5 satisfies none of
them. The valid mixed include/exclude case is preserved: two reviewers of the same study can each
contribute to SufficientlyIncludedAndAgree and SufficientlyExcludedAndDisagree respectively (or the
mirror pair), and nothing collapses them.
Application invariants. The one-screening-per-screener-per-project rule (§3.1) means ScreenedCount
is a distinct-study count, so Screened + Available + Unavailable = TotalNumberStudies should hold. The
code states that invariant explicitly as MembershipScreeningStats.NumberVerification
(MembershipScreeningStats.cs:80-81) but never asserts it; it is a self-check exposed on the wire.
Nothing enforces that a membership row's project-wide fields agree with ProjectScreeningStats, even
though they are read from the same facet.
Tolerance and rounding. All wire counts are integers and require exact equality. The five
Percent* values (MembershipScreeningStats.cs:74-78) use AsPercentOfTotal (:83-86):
TotalNumberStudies == 0 ? 0 : Math.Truncate(10000 * (decimal)number / TotalNumberStudies) / 100 —
decimal arithmetic, truncated (never rounded) to exactly two decimal places, with an explicit
zero-denominator guard. Any materialized parity check on these values must reproduce that truncation
exactly; they are derived and must not be stored as independent counters.
Authorized scope and visibility. investigatorId restricts which membership rows are built
(StudyStats.cs:190), and it is nulled for project admins (StudyStats.cs:97), but the endpoints carry
no membership policy (D2)
and the named leaderboard built from these rows has no enforced gate
(D1).
3.3 Stage annotation¶
Scope key. (ProjectId, StageId). One payload per stage carrying two fixed inclusion-class buckets
— unexcluded and excluded — exactly as the plan requires; inclusion class is not a separate scope key
(StageAnnotationStats constructed at StudyStats.cs:235-249).
Bounded profile keys. Per inclusion class: TotalCount; the two-dimensional
AnnotationCountTally distribution keyed by (NumberOfCandidateSessions, NumberOfCompletedCandidateSessions);
CompletedAndStartedReconciliation; CompletedAndCompletedReconciliation. The stage-level Sessioned
rollup additionally supplies Available, Full, Completed, CompletedAndNotStartedReconciliation,
CompletedAndInProgressReconciliation and Count.
Population. The unexcluded bucket is ¬NewStudyFilters.ExcludedStudies(threshold) and the excluded
bucket is NewStudyFilters.ExcludedStudies(threshold) (StudyStats.cs:315-318), both evaluated against
the persisted ScreeningInfo.InclusionInfo[] element for the current threshold (Filters.cs:490-504).
Exact formulas (SessionedGroupStage, StudyStats.cs:286-287; $group by
ExtractionInfo.SessionTallies.StageId after unwinding SessionTallies). Let NCS be
SessionTallies.NumberOfCandidateSessions, NCCS be SessionTallies.NumberOfCompletedCandidateSessions,
RS be SessionTallies.ReconciliationStarted and RC be SessionTallies.ReconciliationCompleted:
| Profile key | Formula |
|---|---|
Available |
NCS < MNS |
Full |
NCS >= MNS |
Completed |
NCCS >= MNS |
CompletedAndNotStartedReconciliation |
NCCS >= MNS ∧ RS = false |
CompletedAndStartedReconciliation |
NCCS >= MNS ∧ RS = true |
CompletedAndInProgressReconciliation |
NCCS >= MNS ∧ RS = true ∧ RC = false |
CompletedAndCompletedReconciliation |
NCCS >= MNS ∧ RC = true |
Count |
$sum: 1 over studies that have a SessionTally for the stage |
The CandidateSessionsStartedGroupedCount facets (StudyStats.cs:385-399 unexcluded, :415-429
excluded) group the same population by (NCS, NCCS) and count studies.
CreateAnnotationSessionStats (StudyStats.cs:236-249) then reshapes that into a nested
Dictionary<NCS, Dictionary<NCCS, StudyCount>> and — critically — appends a synthetic
(0, 0, totalCount − sessioned.Count) cell (:242) so that studies with no SessionTally at all for
the stage are represented as the "no session" profile. That synthetic cell is the family's no session
profile key; it is derived, not read from Mongo, and a materialized implementation must reproduce it.
The synthetic cell shares its (0, 0) key with a real (0, 0) facet row whenever a study holds
only a durable slot reservation or only a reconciliation session for the stage, since neither is a
candidate session. The two describe disjoint sets of studies in the same cell, so the reshaping sums
the study counts sharing a pair (:245-246) rather than indexing them by key. Issue #3140 records the
prior behaviour, where ToDictionary threw ArgumentException and the whole calculation could not
return for such a project; a materialized implementation must reproduce the summed cell, not the
throw.
Configuration inputs. MNS = 2, hardcoded at StudyStats.cs:313 with an explicit
TODO: This should use Project.AgreementThreshold.NumberScreened (SessionCountTarget) instead of a
hardcoded value. See PR #2331 at :311-312. Project.AgreementThreshold supplies the excluded/unexcluded
partition. The stage list comes from project.Stages (StudyStats.cs:236), so a stage with no sessions
still yields a payload. Per the
recovered-design reconciliation,
materialization mirrors this hardcode until a separate correction is approved.
Exclusivity and permitted overlaps. Available and Full partition the sessioned population.
Completed ⊆ Full whenever NCCS <= NCS. CompletedAndNotStartedReconciliation and
CompletedAndStartedReconciliation partition Completed, because they are exact complements on RS.
The reconciliation pair below them is not a partition and must not be treated as one.
CompletedAndInProgressReconciliation tests RS = true ∧ RC = false, but
CompletedAndCompletedReconciliation tests RC = true without testing RS
(SessionedGroupStage, StudyStats.cs:287). On a document where RC = true ∧ RS = false — persisted
or legacy data, which no guard prevents, as the invariant note below records — the completed counter
increments while neither the started nor the in-progress counter does, so
CompletedAndInProgressReconciliation + CompletedAndCompletedReconciliation exceeds
CompletedAndStartedReconciliation. A materialized implementation must therefore compute all three
counters independently from their own predicates and must never derive one from the other two;
deriving would silently disagree with the authoritative result on exactly those records. The (NCS, NCCS)
distribution cells are mutually exclusive by construction and their sum, including the synthetic
(0, 0) cell combined with any real (0, 0) row as above, equals the inclusion class's total.
Application invariants that make a state impossible. NCCS <= NCS holds because a completed
candidate session is a candidate session — SessionTally is derived from the study's own Sessions
list and SlotReservations (ExtractionInfo.cs:30-40), which are mutually exclusive holders of a slot.
RC = true ∧ RS = false is not reachable through the ordinary lifecycle for the same reason, but no
guard asserts it, so a materialized family must fence rather than assume it. No invariant bounds NCS
by Stage.SessionCountTarget; enforcement is conditional on Stage.EnforceAnnotationTarget.
Tolerance and rounding. Integer counts only; exact equality required.
Authorized scope and visibility. As §3.1: currently the global authenticated-user filter only; the plan's target is project view plus the metric-specific graph permission.
3.4 Membership-stage annotation¶
Scope key. (ProjectId, MembershipId, StageId), carried in a
MembershipStageAnnotationStatsMap per membership holding one MembershipStageStats per stage
(StudyStats.cs:132-184). Both fixed inclusion classes live in the one membership-stage payload
(MembershipStageStats.cs:30-31), matching the plan's requirement that inclusion class is not a
caller-selected dimension.
Bounded profile keys. Per inclusion class: Available, InProgress, Completed, Unavailable,
Total, CandidateAnnotationSessionsGloballyCompleted, ReconcileAvailable, ReconcileInProgress,
ReconcileCompleted, ReconcileUnavailable, SelfReconciliationEnabled
(MembershipAnnotationSessionStats.cs:27-38); plus the stage-level Total, IncompleteThreshold,
HasReachedMaxInProgress and HasReachedMaxInProgressReconciliation (MembershipStageStats.cs:28-35).
Per-investigator inputs (InvestigatorSessionGroupStage, StudyStats.cs:283-284; unwinds
ExtractionInfo.Sessions, matches on investigator and stage, unwinds SessionTallies and re-matches
SessionTallies.StageId == Sessions.StageId, then groups by (InvestigatorId, StageId)). Let S be
the unwound session:
| Facet field | Formula |
|---|---|
NumberOfStudiesSessionInProgress |
S.Status = 0 ∧ S.Reconciliation = false |
NumberOfStudiesSessionCompleted |
S.Status = 1 ∧ S.Reconciliation = false |
NumberOfStudiesSessionAvailable |
NCS < MNS ∧ S.Reconciliation = false |
NumberOfStudiesSessionFull |
NCS >= MNS ∧ S.Reconciliation = false |
NumberOfStudiesReconcileSessionInProgress |
S.Status = 0 ∧ S.Reconciliation = true |
NumberOfStudiesReconcileSessionCompleted |
S.Status = 1 ∧ S.Reconciliation = true |
NumberOfStudiesReconcileSessionAvailable |
RS = false ∧ NCCS >= MNS |
Exact composition (local function CreateMembershipAnnotationStats, StudyStats.cs:152-171), where
allCount is the inclusion class's project total (AllUnexcludedCount or AllExcludedCount),
sessioned is the stage's §3.3 rollup for that class, and inv is the per-investigator facet row
(all null-coalesced to 0):
| Profile key | Formula |
|---|---|
Available |
allCount − sessioned.Count + sessioned.Available − inv.NumberOfStudiesSessionAvailable |
InProgress |
inv.NumberOfStudiesSessionInProgress |
Completed |
inv.NumberOfStudiesSessionCompleted |
Unavailable |
sessioned.Full − inv.NumberOfStudiesSessionFull |
Total |
allCount |
CandidateAnnotationSessionsGloballyCompleted |
sessioned.Completed |
ReconcileAvailable |
sessioned.CompletedAndNotStartedReconciliation − (stage.AllowSelfReconciliation ? 0 : inv.NumberOfStudiesReconcileSessionAvailable) |
ReconcileInProgress |
inv.NumberOfStudiesReconcileSessionInProgress |
ReconcileCompleted |
inv.NumberOfStudiesReconcileSessionCompleted |
ReconcileUnavailable |
stage.AllowSelfReconciliation ? 0 : inv.NumberOfStudiesReconcileSessionAvailable |
SelfReconciliationEnabled |
stage.AllowSelfReconciliation |
InsufficientlyCompletedCandidateSessions |
Total − CandidateAnnotationSessionsGloballyCompleted (MembershipAnnotationSessionStats.cs:37) |
Available is the family's "no session and the study is not yet full" profile: the
allCount − sessioned.Count term restores the studies with no SessionTally for the stage, exactly
mirroring §3.3's synthetic (0, 0) cell.
Configuration inputs. MNS = 2; Stage.AllowSelfReconciliation (Stage.cs:157);
Stage.MaxInProgress and Stage.HideExcludedStudiesFromReviewers for the HasReachedMaxInProgress
derivations (Stage.cs:130-146); Stage.ExcludedSessionStatsGrouping for the §3.9 merged view.
Project.AgreementThreshold supplies the inclusion partition.
Exclusivity and permitted overlaps. For one membership and inclusion class, Available,
InProgress, Completed and Unavailable are intended to be disjoint and to sum to Total, but
nothing enforces it and the formulas are subtractive: Available can go negative when
sessioned.Count exceeds allCount + sessioned.Available, which is reachable if a SessionTally
survives for a study that has since left the inclusion class. A materialized implementation must
therefore either reproduce the subtraction or fence, and may not assume a non-negative partition.
Reconciliation profiles overlap the ordinary ones: a study can be Completed for this reviewer and
simultaneously counted in ReconcileAvailable for the stage.
Application invariants that make a state impossible. SelfReconciliationEnabled is
Stage.AllowSelfReconciliation, which is assigned nowhere in production code — it has one
declaration (Stage.cs:157) and five read sites (StageReviewService.cs:93,
MembershipAnnotationSessionStats.cs:98, StudyStats.cs:163, :168, :171), and it is absent from
StageUpdateDto. Every stage is therefore permanently at its default, so the
AllowSelfReconciliation = true branches of the formulas above are currently unreachable. This is the
one place in this family where an enforced-by-absence invariant genuinely removes profiles; it must be
recorded as "unwritable on current main" rather than modelled as a mutable stage setting.
Tolerance and rounding. Integer counts require exact equality. The eight Percent* values
(MembershipAnnotationSessionStats.cs:39-46) use the same
Math.Truncate(10000 * (decimal)n / Total) / 100 rule as §3.2 (:48-51), with a zero-Total guard,
and are derived.
Authorized scope and visibility. Rows are built only for the requested investigator's membership
when investigatorId is non-null and the caller is not a project admin (StudyStats.cs:135, :97).
The plan's target keys these blocks by membership identity while carrying investigator identity, with
peer rows requiring the corresponding project permission
(Authorization).
3.5 Reviewer screening¶
Scope key. (ProjectId, InvestigatorId, MembershipId) (StageReviewerStats.cs:47-50).
Bounded profile keys. Screened, Available, Unavailable, Total
(StageReviewerStats.cs:51-54).
Exact formulas (StudyRepository.GetInvestigatorScreeningStats, StudyRepository.cs:287-306;
three concurrent CountDocumentsAsync calls, all conjoined with Filters.InProject(project.Id)):
| Profile key | Formula |
|---|---|
Screened |
NewStudyFilters.ScreenedByInvestigator(investigatorId) (Filters.cs:541) |
Available |
NewStudyFilters.InsufficientlyScreenedStudies(threshold) ∧ ¬ScreenedByInvestigator(investigatorId) |
Unavailable |
NewStudyFilters.SufficientlyScreenedStudies(threshold) ∧ ¬ScreenedByInvestigator(investigatorId) |
Total |
CountDocumentsAsync(Filters.InProject(project.Id)) — the whole-project study count (StudyRepository.cs:189, :205) |
Configuration inputs. Project.AgreementThreshold only. Note that
SufficientlyScreenedStudies (Filters.cs:471-487) matches the persisted InclusionInfo element whose
StudyScreenStatus is Included, Excluded or Disagree, and InsufficientlyScreenedStudies
(Filters.cs:524-540) matches StudyScreenStatus == InsufficientlyScreened.
Exclusivity and permitted overlaps. Available and Unavailable are disjoint by their
StudyScreenStatus predicates and both exclude Screened. A study whose InclusionInfo element for the
current threshold is missing falls into none of the three, so Screened + Available + Unavailable can be
strictly less than Total.
Alias analysis against Membership screening. MembershipScreeningStats.Available is
All.InsufficientlyScreened − thisScreener.InsufficientlyScreened and .Unavailable is
All.SufficientlyScreened − thisScreener.SufficientlyScreened (§3.2). Because a screener holds at most
one screening per study, those subtractions are set differences and are semantically the same quantities
as ReviewerScreeningStats.Available/.Unavailable. They are nevertheless not guaranteed to agree
numerically, because the two read different source fields: the membership form evaluates NS >= T ∧ AAR > R
live from ScreeningInfo.AgreementMeasure, while the reviewer form matches the persisted
ScreeningInfo.InclusionInfo[] element. See
D4.
Tolerance and rounding. Integer counts; exact equality.
Authorized scope and visibility. investigatorId is always CurrentUserId, never caller-supplied
(ReviewController.cs:507-527, :529-542). A non-member receives an unhandled exception rather than an
empty response, because project.GetMembershipFor(investigatorId) (StudyRepository.cs:305;
Project.cs:623-627) is called unconditionally to build the DTO. See
D10.
3.6 Reviewer annotation¶
Scope key. (ProjectId, StageId, InvestigatorId, MembershipId) (StageReviewerStats.cs:77-81).
Bounded profile keys. UnexcludedInProgress, UnexcludedCompleted, UnexcludedAvailable,
Unavailable, ExcludedInProgress, ExcludedCompleted, Total, HasReachedMaxInProgress
(StageReviewerStats.cs:82-89).
Exact formulas (StudyRepository.GetInvestigatorAnnotationStats, StudyRepository.cs:218-283;
seven concurrent CountDocumentsAsync calls, all conjoined with Filters.InProject(project.Id)). Write
EX for NewStudyFilters.ExcludedStudies(project.AgreementThreshold) and SA for
NewStudyFilters.SufficientlyAllocated(stageId, stage.SessionCountTarget, featureFlags.ActiveReviewerTrackingAvailable):
| Profile key | Formula |
|---|---|
UnexcludedInProgress |
¬EX ∧ ReviewerStageSessionInProgress(stageId, investigatorId, false) |
UnexcludedCompleted |
¬EX ∧ ReviewerStageSessionCompleted(stageId, investigatorId, false) |
UnexcludedAvailable |
¬EX ∧ ¬StageSessionByReviewer(stageId, investigatorId, false) ∧ ¬SA |
Unavailable |
count(EX ∧ ¬StageSessionByReviewer(…)) + count(SA ∧ ¬StageSessionByReviewer(…) ∧ ¬EX) — the sum of two separate counts (StudyRepository.cs:274-276) |
ExcludedInProgress |
EX ∧ ReviewerStageSessionInProgress(stageId, investigatorId, false) |
ExcludedCompleted |
EX ∧ ReviewerStageSessionCompleted(stageId, investigatorId, false) |
Total |
Whole-project study count |
HasReachedMaxInProgress |
stage.MaxInProgress != null ∧ ((stage.HideExcludedStudiesFromReviewers ? 0 : excludedInProgress) + unexcludedInProgress) >= stage.MaxInProgress (StudyRepository.cs:280-282) |
Configuration inputs. Stage.SessionCountTarget, Stage.MaxInProgress,
Stage.HideExcludedStudiesFromReviewers, Project.AgreementThreshold, and the
ActiveReviewerTrackingAvailable feature flag, which changes what SufficientlyAllocated counts as
allocated (StudyRepository.cs:243, :249) — durable SlotReservation state participates in the
availability formula only when that flag is on. This is exactly the coupling the plan's
annotation-reservation-* row anticipates
(Event and invalidation contract).
Exclusivity and permitted overlaps. The two Unavailable addends are disjoint (one requires EX,
the other ¬EX), so Unavailable is a genuine count, but it deliberately merges an
inclusion-class-driven reason with an allocation-driven reason and cannot be decomposed from the wire
value. UnexcludedAvailable excludes any study the reviewer already has a session on, so it does not
overlap UnexcludedInProgress/UnexcludedCompleted. Total is the whole-project count and is not the
sum of the other six.
Alias analysis against Membership-stage annotation. UnexcludedInProgress/UnexcludedCompleted
correspond to the membership-stage InProgress/Completed of the unexcluded bucket, and
ExcludedInProgress/ExcludedCompleted to the excluded bucket's. HasReachedMaxInProgress duplicates
Stage.HasReachedMaxInProgress(MembershipStageStats) (Stage.cs:140-143) with independent code and
identical semantics. UnexcludedAvailable and Unavailable are not aliases: the membership-stage
forms derive from stage-level Sessioned rollups and the MNS = 2 hardcode, whereas these derive from
SufficientlyAllocated against Stage.SessionCountTarget and the reservation flag. Recorded as a
divergence, not a rename, in D4.
Tolerance and rounding. Integer counts; exact equality. HasReachedMaxInProgress is a boolean and
must be recomputed, never stored as a counter.
Authorized scope and visibility. As §3.5, including the non-member throw
(D10). The
service additionally throws InvalidOperationException while project.CalculatingInclusionInfo is true
(src/libs/project-management/SyRF.ProjectManagement.Application/Services/ReviewStatsQueryService.cs:40-41;
the project-scoped variant guards identically at ProjectManagementService.cs:65-66) — the only existing
in-code precedent for the plan's StatisticsInclusionRecalculationInProgress fence, and it is applied to
the reviewer families but not to FullStats (D11).
3.7 Question answers¶
Scope key. (ProjectId, QuestionId). The stage dimension named in the plan's family table is not
present in the current calculation: the aggregation groups by ExtractionInfo.Annotations.QuestionId
only (StudyRepository.cs:951-956), with no stage or question-version key. Recorded as a discrepancy in
D12.
Bounded profile keys. Studies (distinct studies with at least one answer to the question) and
Annotations (total answer records), carried as the record struct AnnotationAnswerTally
(AnnotationQuestion.cs:21) inside AnnotationAnswerTallyStore(Id, Studies, Annotations)
(AnnotationQuestion.cs:23-39).
Exact formula (StudyRepository.GetAnnotationQuestionAnswerTally, StudyRepository.cs:947-964):
$match { ProjectId } → $unwind ExtractionInfo.Annotations → $group by
ExtractionInfo.Annotations.QuestionId, with AnnotationCount = $sum: 1 and
StudyCount = $size($addToSet($_id)). Results are materialized with a blocking synchronous .ToList()
inside the async method (StudyRepository.cs:962).
Configuration inputs. None. The tally is threshold-independent and stage-independent; it counts every annotation on every study in the project, including annotations belonging to reconciliation sessions and to sessions in any status.
Refresh semantics. The value is a stored projection on Project.AnnotationQuestionAnswerTally
(Project.cs:205, private set), mutated only by Project.UpdateAnnotationQuestionAnswerTally
(Project.cs:213-216), which is called only by ProjectManagementService.UpdateAnnotationQuestionAnswerTally
(ProjectManagementService.cs:514-520), which is called only by the single REST endpoint
PUT api/projects/{projectId}/update-annotation-answer-tally (ProjectController.cs:652-658). No event
handler, scheduled job or post-save hook refreshes it. Project.AnnotationAnswerTallyDictionary
(Project.cs:207-209) rebuilds a Dictionary<Guid, AnnotationAnswerTally> on every access, with no
caching beyond the stored list. This is the closest existing precedent in the codebase for a stored
statistics projection on Project, and it is exactly the legacy path the plan's
legacy-question-tally-refresh row supersedes.
Exclusivity and permitted overlaps. Studies <= Annotations always, because Studies is a distinct
count over the same unwound rows. Questions with no answers are absent from the result entirely rather
than present with zeroes, so a missing key and a genuine zero are indistinguishable on the wire.
Application invariants. None constrain this family. In particular, deleting a question does not
refresh the tally: ProjectManagementService.DeleteQuestionAsync (ProjectManagementService.cs:39-54)
strips the annotations from every study and replaces the Project, but never calls
UpdateAnnotationQuestionAnswerTally, so the stored tally retains counts for a deleted question until
the next manual refresh. The same holds for question upsert, copy, reposition and stage-question replace.
Tolerance and rounding. Integer counts; exact equality.
Authorized scope and visibility. The refresh endpoint requires
[Authorize(ProjectAuthorization.ProjectDesignPolicy)] (ProjectController.cs:653) — the only statistics
endpoint in this catalogue that carries an explicit project-role policy. The read path inherits whatever
policy the enclosing ProjectDetailsDto endpoint carries.
3.8 Search/population¶
Scope key. (ProjectId, SystematicSearchId).
Bounded profile keys. NumberOfStudies per search; and the client-derived project total
studyCount.
Exact formulas.
SystematicSearch.NumberOfStudies—SystematicSearch.cs:42—SyrfReferenceFiles.Sum(srf => srf.NumberOfStudies), a pure in-memory LINQ sum recomputed on every load. It is explicitly unmapped from BSON (SystematicSearchRepository.cs:55,cm.UnmapProperty(ss => ss.NumberOfStudies)), so it is never stored. EachStudyReferenceFile.NumberOfStudiesis set once at import time (src/libs/project-management/SyRF.ProjectManagement.Core/Model/SystematicSearchAggregate/StudyReferenceFile.cs:13,:22).- Project total —
project-detail.viewmodel.ts:99—_.sumBy(searches, s => s.numberOfStudies)over whatever per-search DTOs are currently in the ngrxsearchentity slice. No backend endpoint returns a pre-aggregated project study count.
Source and inclusion/exclusion filters. Neither formula reads pmStudy at all. StudyStats.cs has
zero references to SystematicSearch. This family answers "how many records were imported", not "how
many Study documents exist"; it is never reconciled against an actual study count after deduplication,
merge or deletion. Two consequences the plan's search-population row must carry: the numbers can diverge
permanently from pmStudy, and the divergence is not an error state.
Pending-import and pending-deletion exclusions. The plan's family row says pending import and
deletion rows "remain excluded exactly as authoritative queries define". On current main the
authoritative definition is narrower than that phrase suggests. The SystematicSearch document is not
created until ProjectManagementService.CompleteSearchImportJob (ProjectManagementService.cs:370-394),
so search-level listing cannot see an in-flight import — but Study.SystematicSearchId is populated as
each study is parsed and bulk-upserted, well before completion, and project-wide study queries
(StudyRepository.GetStudiesForProject, GetStudiesForProjectWithCountAsync) filter only on
ProjectId with no join against Project.SystematicSearchIds. Half-imported studies are therefore
already visible to project-level study counts. There is no pending-deletion state at all: both deletion
endpoints fail closed with 503 (§5, D13).
Exclusivity and permitted overlaps. A SystematicSearch may be linked to more than one project
(ProjectIds.Contains(projectId), SystematicSearchRepository.cs:30-33), so the project total is a sum
over the searches linked to that project, not a partition of a global set.
Application invariants. None enforce that NumberOfStudies matches the number of Study documents
carrying that SystematicSearchId.
Tolerance and rounding. Integer counts; exact equality against the reference-file values, not against
pmStudy.
Authorized scope and visibility. GET api/projects/{projectId}/searches carries
[Authorize(ProjectAuthorization.ProjectViewPolicy)] (SearchController.cs:66-75); the
ProjectDetailsDto path inherits its endpoint's policy.
3.9 Project/stage derived summaries¶
This family holds every value that is a pure function of the approved scopes above and must therefore be derived rather than stored as an independent counter (Catalogue baseline: "Derive without duplicating source counters").
MergedMembershipAnnotationStats (MembershipAnnotationSessionStats.cs:54-112), constructed
eagerly for every membership-stage cell (MembershipStageStats.cs:22). Let U and E be the unexcluded
and excluded MembershipAnnotationSessionStats, and G be Stage.ExcludedSessionStatsGrouping
(Stage.cs:345; values WithUnexcluded, Unavailable, Separate):
| Merged field | Formula |
|---|---|
Available |
U.Available |
InProgress |
U.InProgress + (G = WithUnexcluded ? E.InProgress : 0) |
Completed |
U.Completed + (G = WithUnexcluded ? E.Completed : 0) |
Unavailable |
U.Unavailable + E.Unavailable + E.Available + (G = Unavailable ? E.InProgress + E.Completed : 0) |
Total |
E.Total + U.Total |
CandidateAnnotationSessionsGloballyCompleted |
U.… + E.… |
ReconcileAvailable |
U.ReconcileAvailable |
ReconcileInProgress |
U.… + (G = WithUnexcluded ? E.… : 0) |
ReconcileCompleted |
U.… + (G = WithUnexcluded ? E.… : 0) |
ReconcileUnavailable |
U.… + E.… + E.ReconcileAvailable + (G = Unavailable ? E.ReconcileInProgress + E.ReconcileCompleted : 0) |
SelfReconciliationEnabled |
Stage.AllowSelfReconciliation |
SeperatedExcludedMembershipAnnotationStats (MembershipAnnotationSessionStats.cs:114-140) is
populated only when G = Separate (:101-108) and carries ExcludedInProgress, ExcludedCompleted,
ExcludedInProgressReconciliation, ExcludedCompletedReconciliation plus their Percent* views, all
computed against the merged total.
Configuration invariant. Stage rejects two (G, HideExcludedStudiesFromReviewers) combinations at
construction (Stage.cs:65-70), throwing for WithUnexcluded and for Unavailable under the stated
conditions. These are the only enforced application invariants that genuinely remove profiles from this
family, and a materialized implementation may rely on them only for stages created through that
constructor.
Other derived values. MembershipStageStats.HasReachedMaxInProgress and
HasReachedMaxInProgressReconciliation (MembershipStageStats.cs:20-21 → Stage.cs:140-146);
ReviewerAnnotationStats.HasReachedMaxInProgress (StudyRepository.cs:280-282);
MembershipAnnotationSessionStats.InsufficientlyCompletedCandidateSessions (:37);
MembershipScreeningStats.NumberVerification (:80-81); FullStats.MembershipIds and StageIds
(FullStats.cs:15-57); the whole-project Total denominators; and the client-side studyCount (§3.8).
Tolerance and rounding. Every Percent* value in this family uses the same
Math.Truncate(10000 * (decimal)n / Total) / 100 rule with a zero-denominator guard — the only two
implementations are MembershipAnnotationSessionStats.AsPercentOfTotal (:48-51) and
MembershipScreeningStats.AsPercentOfTotal (:83-86), and they are byte-identical. Per
Catalogue rules this is a floating-style metric and must carry its
rule explicitly: truncate to two decimal places using decimal arithmetic, never round, and return
exactly 0 when the denominator is zero. No implicit tolerance is permitted.
Authorized scope and visibility. Inherited from the scope each value is derived from; a derived summary never widens the authorization of its inputs.
3.10 Domain reconciliation¶
Scope key. (ProjectId, StageId) for the stage-level counters and
(ProjectId, MembershipId, StageId) for the per-reviewer counters. There is no separate reconciliation
scope in code.
Bounded profile keys. Stage level (from SessionedGroupStage, §3.3):
CompletedAndNotStartedReconciliation, CompletedAndStartedReconciliation,
CompletedAndInProgressReconciliation, CompletedAndCompletedReconciliation. Membership level (§3.4):
ReconcileAvailable, ReconcileInProgress, ReconcileCompleted, ReconcileUnavailable,
SelfReconciliationEnabled.
The four dedicated reconciliation facets are dead query work and are excluded from the family.
Unexcluded/ExcludedStartedReconciliationCount and
Unexcluded/ExcludedCompletedReconciliationCount (StudyStats.cs:384-397, :414-427) are computed by
the $facet stage, unwound (:440-449) and assigned to AllInfo
(src/libs/project-management/SyRF.ProjectManagement.Mongo.Data/AllInfo.cs:36-40, :71-75) — and then
never read. Nothing in the construction of FullStats consults them: the only consumer-visible
reconciliation counts, AnnotationSessionStats.StartedReconciliationCount and
.CompletedReconciliationCount, are taken from Sessioned.CompletedAndStartedReconciliation and
Sessioned.CompletedAndCompletedReconciliation instead (StudyStats.cs:231-232). A repository-wide
search for either property name returns only the AllInfo declaration and assignment plus the unrelated
StageAnnotationStats/SessionTallyCounts members.
They are also differently scoped from the keys they superficially resemble. GetFullProjectStatsAsync
runs with stageId == null (StudyStats.cs:99), and under that arm the facet predicate degrades to
st => st.ReconciliationStarted with no stage term (:387, :395, :417, :425), so the facets
count matching SessionTally rows across all stages rather than producing a per-stage value.
Listing them as stage-level profile keys would materialize an unconsumed calculation at the wrong grain.
They are recorded here as dead query work, retained as an efficiency observation for a separate change
— removing them shortens the $facet pipeline — and excluded from the approved family.
Source fields. Reconciliation is not a distinct aggregate. It is two booleans on existing
aggregates: AnnotationSession.Reconciliation and SessionTally.ReconciliationStarted /
.ReconciliationCompleted. There is no ReconciliationSession, ReconciliationSessionVersion or
DomainReconciliation type anywhere in src. The reconciliation family therefore shares every mutation
owner with the annotation family; there is nothing separate to instrument.
Exclusivity and permitted overlaps. As §3.3: CompletedAndNotStartedReconciliation and
CompletedAndStartedReconciliation partition Completed. The in-progress and completed reconciliation
counters do not partition the started ones — CompletedAndCompletedReconciliation tests RC alone
and so counts an RC = true ∧ RS = false record that the started counter misses (StudyStats.cs:287;
see §3.3). Each of the three must be computed from its own predicate. Reconciliation counters
overlap the ordinary annotation counters by design — the same study is counted in both.
Application invariants. Study.DeleteSessionAndRestoreSlotReservation explicitly excludes
reconciliation sessions from slot-reservation restore (Study.cs:282), and reconciliation submission
deliberately skips the capacity guard and the reservation removal (ReviewController.cs:365-367, :373;
ReviewSubmissionService.cs:63-64). So a reconciliation session never consumes an annotation
reservation, and profiles that assume it does are impossible.
Tolerance and rounding. Integer counts; exact equality.
Authorized scope and visibility. As the annotation families it travels with. Reconciliation materialization is explicitly not statistics parity repair (Event and invalidation contract).
4. Exclusions confirmed¶
Three families the plan places outside the programme were re-checked directly against main. In each
case the finding is that no live implementation exists, so there is nothing to catalogue, reconcile or
retire.
Agreement / kappa. grep -rn "Kappa\|Cohen" across all of src/**/*.cs, production and test code
alike, returns zero matches. The only agreement calculation with a live code path is the per-study
StudyAgreementMeasure (§3.1), a 2 * |inclusionRatio − 0.5| absolute-agreement ratio over one study's
own screenings; StudyAgreementMeasure.New throws if the supplied screenings do not all belong to the
same study (StudyAgreementMeasure.cs:28-30), so it is structurally incapable of the cross-stage
comparison the plan forbids reviving. CLAUDE.md's warning that "the invalid cross-stage agreement/kappa
calculation must not be revived" refers to design-time evidence in pull request #2534 and the M008-M011
plans, not to surviving code. Disposition: excluded until a separate statistical specification is
approved, per Catalogue baseline.
Outcome-level statistics. No cross-study outcome aggregate, shared unit definition or consumer
contract exists. OutcomeData is per-study state on ExtractionInfo and is deleted with its session
(ExtractionInfo.cs:306-308); the only outcome-shaped writer is
OutcomeDataFormatRowWriter under Services/DataExportServices/DataFormatWriters/, a per-study row
writer with no GroupBy, Sum or cross-study count. The plan states that the shared scope key supports
versioned question identity only, so outcome materialization cannot enter this programme and may return
only through a separately approved feature.
Operational progress and presence. Import, export and Bulk PDF progress are job state, and SignalR
presence is connection state. The concrete surfaces re-checked are: the export progress counter in
CsvDataExportWriter (a ++studyCount fed to OptimizedDataExportProgressReporter) and the paging
counts at DataExportController.cs:87-97; InvestigatorUsage's three engagement recorders
(InvestigatorUsage.cs:31, :48, :68), whose only writers are ApplicationService.RecordSignInAsync
and ApplicationController, with no screening, annotation or reconciliation event calling into them;
and the assignment-decision counts in GetStudyReviewStatus (StudyRepository.cs:322-360), which are
collapsed to booleans by the ReviewStatusInfo constructor (ReviewStatusStats.cs:26-44) before any
controller sees them. None of these is project statistical history, and each stays in its existing
operational model.
Not-yet-implemented, therefore not excluded but absent. PRISMA source-type counts have no
implementation: SourceType has no matches in src/libs/project-management, src/services/api or
src/services/project-management, consistent with the [TARGET - Phase 12] status of the
Publication/Citation model. There is nothing to catalogue and nothing to exclude.
5. Unresolved evidence-backed decisions¶
Each item below is a decision the Phase 0 completion review must record. None is a licence to change product code in this PR. Recommended defaults follow the plan's scope-discipline posture: mirror current behaviour, record the divergence, and route any correction to a separate approved change.
D1: Named per-reviewer screening leaderboard has no enforced visibility gate¶
Evidence. StageOverviewComponent hardcodes
screeningSettingLabel = 'This graph is only visible to admin' and
screeningTableSettingLabel = 'This graph is hidden from all project members'
(stage-overview.component.ts:108, :110) and offers an openVisibilitySettings() action
(:124-132) that configures the ProjectActivity.ViewScreeningProgressGraph /
ViewScreeningProgressGraphDecisions permissions. The template nevertheless renders the leaderboard
unconditionally (stage-overview.component.html:55-63), gated only on stageHasScreening$ and data
presence. selectScreeningMembershipData (membership-screening-stats.selectors.ts:94-119) returns
fully named rows — investigatorId, investigatorName, screened, available, unavailable,
included, excluded — for every membership. ScreeningOverviewComponent renders the same data the
same way (screening-overview.component.ts:53, screening-overview.component.html:36-42). The leaf
chart and table components perform no permission check. Server-side there is no filter either:
MembershipScreeningStats is a plain member of FullStats and neither full-stats endpoint carries a
ViewScreeningProgressGraph check anywhere in the chain. By contrast the annotation-side label
(annotationSettingLabel = 'This graph is visible to all project members', :109) matches its own
unrestricted rendering, so this is specifically a screening finding, not a general pattern.
Impact. The plan requires peer rows and leaderboards to sit behind the corresponding project permission (Authorization). Materializing this family without first settling the gate would either freeze the current unrestricted behaviour into the projection contract or silently change behaviour at cutover. It also decides whether reviewer names may appear in a materialized row at all, or only membership identifiers resolved at read time.
Options. (a) Treat the labels as aspirational and materialize current behaviour unchanged, deferring enforcement. (b) Treat this as a pre-existing authorization defect and fix it in a separate PR before Phase 4. © Materialize membership identifiers only, resolving investigator names at query time under the metric-specific permission.
Recommended default: ©, with (b) tracked separately. It satisfies the plan's authorization rule without this programme taking ownership of a pre-existing gap, and it keeps person identity out of the stored projection.
D2: full-stats and reviewer-stats endpoints carry no project-membership policy¶
Evidence. ReviewController.GetFullStats (ReviewController.cs:458-477),
ProjectController.GetFullStats (ProjectController.cs:779-787),
ReviewController.GetReviewerStatsForStage (:507-527) and GetReviewerStatsForProject (:529-542)
have no [Authorize(Policy)] attribute on the method, and neither controller class carries one. Only the
global RequireAuthenticatedUser() MVC filter (Program.cs:126-132) applies.
NotificationHub.SubscribeToProjectFullStats (NotificationHub.cs:1027-1035) likewise has only the
class-level [Authorize] (:31), while its sibling SubscribeToProject does carry
[Authorize(ProjectAuthorization.ProjectViewSignalRPolicy)] (:934). For ProjectController.GetFullStats
the investigator filter narrows only the membership blocks: ProjectScreening and StageAnnotation are
built from the unfiltered All, AllUnexcluded, AllExcluded, UnexcludedSessioned,
ExcludedSessioned, ScreeningGroupedCount and the two histogram facets, which are never filtered by
investigatorId.
Impact. Any authenticated SyRF user who knows a project GUID can read that project's aggregate
screening and annotation tallies. The reviewer-stats endpoints are incidentally protected only because
GetMembershipFor throws (see D10).
The plan's query contract assumes the project-view boundary already holds
(Authorization), so a materialized read path inheriting today's
attributes would inherit the gap.
Options. (a) Add the project-view policy to these endpoints in a separate PR before any consumer cutover. (b) Enforce the boundary inside the new statistics query service so the materialized path is correct even where the legacy path is not — which would make the materialized and authoritative responses differ for a non-member and violate the plan's parity requirement. © Defer.
Recommended default: (a). The plan's parity rule requires both paths to agree, so the fix belongs in the legacy path, before cutover, as its own scoped change.
D3: MembershipScreeningStats.SufficientlyIncludedAndAgree is fed the wrong facet field¶
Evidence. StudyStats.cs:199-202 passes
invScreeningDict…?.SufficientlyExcludedAndAgree ?? 0 into both the sufficientlyIncludedAndAgree
and the sufficientlyExcludedAndAgree constructor parameters of MembershipScreeningStats
(MembershipScreeningStats.cs:15-16). The Investigator facet does compute a distinct
SufficientlyIncludedAndAgree (StudyStats.cs:278-281), and it is never read. The adjacent disagree
pair is mapped correctly (StudyStats.cs:203-206).
Impact. Every MembershipScreeningStats row on the wire reports the excluded-and-agree count in the
included-and-agree field. Any materialized implementation must decide deliberately whether to reproduce
the defect (parity) or emit the correct value (a behaviour change at cutover that parity shadow mode
would flag as a mismatch).
Options. (a) Reproduce exactly, and record the defect for separate correction. (b) Fix in a separate PR before Phase 4 so the materialized family is built against corrected behaviour. © Fix inside the materialized path only.
Recommended default: (b). © is excluded outright: it would make the two paths disagree, which the
plan's coherent-fallback rule forbids. Note that the recovered validation contract in
recovered-design-reconciliation.md currently pins the duplicated value as a parity fixture ("proving the
mapper duplicates the excluded-agree value into both output fields"), which is option (a) by default.
Until (b) lands, that fixture stands and the catalogue entry for this profile key must state that the
shipped value is the excluded-and-agree count; the legacy fix must update the parity fixture in the same
change so the two paths never disagree.
D4: Reviewer screening and membership screening read two different source fields¶
Evidence. MembershipScreeningStats.Available/.Unavailable are computed as
All.InsufficientlyScreened − thisScreener.InsufficientlyScreened and
All.SufficientlyScreened − thisScreener.SufficientlyScreened (StudyStats.cs:213-216), where both
Sufficiently/Insufficiently predicates evaluate NS >= T ∧ AAR > R live from
ScreeningInfo.AgreementMeasure inside the $group stages (StudyStats.cs:274, :278). The reviewer
form counts NewStudyFilters.InsufficientlyScreenedStudies / SufficientlyScreenedStudies
(StudyRepository.cs:292-299), which match the persisted ScreeningInfo.InclusionInfo[] element for
the current threshold (Filters.cs:471-540) — a field written only by the three
UpdateStudyInclusionInfoForProjectAsync passes (StudyRepository.cs:1224-1249). On the annotation side
the same split exists: membership-stage availability derives from stage Sessioned rollups against the
hardcoded MNS = 2, while ReviewerAnnotationStats.UnexcludedAvailable/Unavailable derive from
SufficientlyAllocated against Stage.SessionCountTarget and the ActiveReviewerTrackingAvailable flag
(StudyRepository.cs:234-249).
Impact. The plan's family table instructs Phase 0 to "catalogue against membership screening to
remove aliases" and to catalogue reviewer annotation against membership-stage annotation. The evidence is
that Screened/Available/Unavailable are semantically the same quantities but are not guaranteed
to be numerically equal, because one reads a live computed field and the other a materialized-at-write
partition that can be missing, stale, or mid-rewrite. Collapsing them into one materialized family
without resolving this would silently pick one definition.
Options. (a) Declare the persisted InclusionInfo partition canonical for every family and
re-express the membership screening predicates against it, accepting a behaviour change for the
membership rows. (b) Declare the live AgreementMeasure computation canonical and re-express the
reviewer filters, accepting a large query-shape change. © Keep both as separate catalogue families with
an explicit recorded divergence, and defer the unification.
Recommended default: © for Phase 0, resolving to (a) in Phase 4. (a) is the direction the rest of the system already leans — every annotation family's population already comes from the persisted partition — but the divergence must be measured on real data before the reviewer families are folded in.
D5: Two orphaned AutoMapper value resolvers¶
Evidence. ProjectStatsResolver (ProjectStatsResolver.cs:11-29) is an
IValueResolver<Project, ProjectDetailsDto, FullStats> that calls
PmUnitOfWork.Studies.GetFullProjectStatsAsync(source, investigatorId).Result — a sync-over-async
blocking call (:27). AnnotationSummaryResolver (AnnotationSummaryResolver.cs:9-27) has the body
return new(2, 0.333, 23, 5, 3, 24); (:25) preceded by
//TODO: Get real annotation summary details (:24). Neither is wired: ProjectDetailsDto.CreateMappings
(ProjectDetailsDto.cs:44-64) is the only CreateMap<Project, ProjectDetailsDto> in production code and
calls MapFrom for neither, ProjectDetailsDto has no FullStats or AnnotationSummaryDto member
(:16-42), and the profile loader auto-wires only IMapFrom<>/IMapTo<>/IHaveCustomMappings
(src/services/api/SyRF.API.Endpoint/Infrastructure/AutoMapperConfig.cs:38-63), not bare
IValueResolver<> implementations. By contrast SearchResolver is wired, explicitly, at
ProjectDetailsDto.cs:50.
Impact. The plan's problem statement names "Project DTO mapping blocks on asynchronous I/O through
.Result" as one of four current hot-path consequences
(Problem and current architecture). The evidence is
that this specific .Result call is unreachable today. That does not weaken the case for the programme —
the other three consequences stand — but the catalogue must not claim a live blocking call that does not
execute. AnnotationSummaryResolver additionally never held real logic, so it is a stub, not a
regression.
Options. (a) Delete both in a separate cleanup PR. (b) Leave them and record them as dead. © Treat
AnnotationSummaryResolver's six hardcoded numbers as a latent requirement for a stage-annotation summary
shape and design it into the programme.
Recommended default: (a) for both, and reject ©. The plan requires a concrete authoritative aggregate and consumer before any new family is admitted, and this stub supplies neither. Correct the plan's problem statement to say the blocking resolver exists but is unreachable.
D6: NumberOfStudiesInProject is unreachable¶
Evidence. ISystematicSearchRepository.NumberOfStudiesInProject (ISystematicSearchRepository.cs:11)
and SystematicSearchRepository.NumberOfStudiesInProject (SystematicSearchRepository.cs:23-28) are the
only two matches for that name in src. Nothing calls it. It is a third, server-side implementation of
the same searches.Sum(ss => ss.NumberOfStudies) idea that ships client-side as studyCount
(project-detail.viewmodel.ts:99).
Impact. Minor, but it decides whether the Search/population family gains a server-side project total as part of Phase 2/4. The plan lists "Approved live study-population/search counts and their contribution to project totals" for this family, and today that contribution exists only in the browser.
Options. (a) Retire the dead method and materialize the project total as a derived scope. (b) Revive it as the authoritative project total. © Leave the total client-side.
Recommended default: (a). A derived project total inside the projection removes the client-side sum-of-sums, which is currently recomputed from whatever DTOs happen to be loaded.
D7: Export independently recomputes included-study counts¶
Evidence. StudyRepository.GetIncludedStudiesForProjectWithCountAsync (StudyRepository.cs:577-664)
applies NewStudyFilters.IncludedStudies(agreementThreshold) in a plain CountDocumentsAsync to size a
paged export cursor. It is the one place outside StudyStats.cs and the reviewer-stats path that counts
"included" studies. grep for GetFullProjectStatsAsync|GetReviewerStatsFor|StudyStatsQuery under
Services/DataExportServices returns nothing.
Impact. Small but real: an export header count and a screening statistic can disagree, and a reader may reasonably expect them to match. The plan classifies exports as consumers that do not mutate statistics, and does not require this count to be unified.
Options. (a) Leave it; record that export counts are paging metadata, not statistics. (b) Route the export count through the materialized query service in Phase 5. © Add a parity assertion between the two.
Recommended default: (a). It is paging metadata with a different purpose, and unifying it would couple export paging to projection freshness for no user-visible benefit.
D8: Dead frontend action pair and no-op reducer handler¶
Evidence. projectDetailActions.getScreeningSummary / screeningSummaryReceived
(project-detail.actions.ts:299-300) are declared and screeningSummaryReceived appears in the on()
clause of all four FullStats-derived entity reducers, but getScreeningSummary is never dispatched and
screeningSummaryReceived is never emitted; the shared mergeEntityState() helper no-ops when an
action's entities payload is undefined, so it is harmless. stage-ui.reducer.ts:48-50 registers a
handler for stageActions.receivedFullStats whose body is (state, props) => ({...state}). Separately,
both live producers of the FullStats graph — the SignalR push (signal-r.service.ts:792-810,
unfiltered) and the REST fetch (project-detail.effects.ts:716-740, investigator-filtered) — merge into
the same four entity slices with last-write-wins and no field-level reconciliation. That last point is
currently theoretical only because SubscribeToProjectFullStats is never invoked: an exhaustive search of
src/services/web/src finds zero references to it, and signal-r.service.ts's raw
_hubConnection.invoke(...) calls cover SubscribeToProject, SubscribeToDataExportJob,
SubscribeToProjectSummaries and the two unsubscribes, but not SubscribeToProjectFullStats.
Impact. The plan's Phase 5 SignalR migration publishes revision and scope invalidations and has clients fetch an authorized snapshot (Decision summary, item 9). That design removes the dual-producer hazard by construction, but the dead scaffolding must be retired rather than carried forward, and the currently unreachable SignalR push must not be revived in its present full-payload form.
Options. (a) Delete the dead action pair, the no-op handler, and the commented-out fullStats$ /
stageLoad$ effects (review-effects.ts:499-515, :546-570) in a separate cleanup PR. (b) Leave them.
© Wire SubscribeToProjectFullStats up as part of the migration.
Recommended default: (a), and explicitly reject ©. Wiring the existing hub method would ship the unfiltered-versus-filtered merge hazard into a live path, which is precisely what the plan's invalidation-plus-authorized-fetch model exists to avoid.
D9: The question-tally edit lock consumes a manually refreshed value¶
Evidence. design.store.ts:528-533 computes
isLocked: (question.answeredAnnotationCount?.annotations ?? 0) > 0 from
AnnotationQuestion.AnsweredAnnotationCount (AnnotationQuestion.cs:157-158), which reads the stored
Project.AnnotationQuestionAnswerTally. That stored value changes only when
PUT api/projects/{projectId}/update-annotation-answer-tally is called (§3.7); no question or annotation
mutation refreshes it. assign.store.ts:220-221 gates an assignment action on the same value.
Impact. The tally is not merely a display statistic — it gates whether a question can still be edited. A stale tally can under-lock (allowing an edit to a question that already has answers) or over-lock. Making the tally transactionally current, which is the plan's Phase 3 intent, would change the behaviour of an authorization-adjacent gate.
Options. (a) Materialize the tally transactionally and accept that isLocked becomes accurate, which
is a user-visible behaviour change at cutover. (b) Keep the edit lock on the legacy stored value and
materialize only the display path. © Replace the lock with a direct existence query, decoupling it from
the statistics family entirely.
Recommended default: (a), flagged for explicit reviewer sign-off. It is the correct behaviour and it is what Phase 3 delivers, but it must be called out at the consumer-flag cutover rather than discovered.
D10: Non-member reviewer-stats requests throw rather than returning a typed response¶
Evidence. Two independent research passes reached opposite conclusions here, and the contradiction was
re-verified for this catalogue rather than arbitrated. Project.GetMembershipFor (Project.cs:623-627)
calls ProjectMembershipGuard(investigatorId) first, which throws
"The provided Investigator is not currently registered with this project" when no matching membership
exists; the following Memberships.First(...) would itself throw InvalidOperationException even without
the guard. Both GetInvestigatorAnnotationStats (StudyRepository.cs:279) and
GetInvestigatorScreeningStats (:305) call it unconditionally, not behind a null-check, while building
the DTO. The "silent zero response" conclusion — reasoned from the filter semantics, which would indeed
evaluate to zero for a non-member — is wrong because it did not account for that unconditional call.
Impact. A non-member currently receives an unhandled exception, not a 403, a 404 or an empty
body. The plan's query contract requires typed, bounded responses and requires missing or unauthorized
selectors to fail closed in a uniform way that does not leak scope existence
(Authorization). An unhandled exception is neither typed nor uniform.
Options. (a) Add the project-membership policy (see D2),
which turns this into a proper 403 before the repository is reached. (b) Catch and map to a typed
response in the query service. © Leave as-is.
Recommended default: (a). It resolves D2 and D10 with one change and keeps the failure at the authorization boundary rather than deep in a repository.
D11: FullStats has no CalculatingInclusionInfo guard¶
Evidence. ReviewStatsQueryService.GetReviewerStatsForStageAsync throws
InvalidOperationException("Inclusion info is being calculated, please try again later.") when
project.CalculatingInclusionInfo is true (ReviewStatsQueryService.cs:40-41), and
ProjectManagementService.GetReviewerStatsForProjectAsync guards identically
(ProjectManagementService.cs:65-66). GetFullProjectStatsAsync (ReviewStatsQueryService.cs:20-24)
and both GetFullStatsForInvestigatorAsync overloads (:26-29, :31-35) have no such guard, yet
every annotation family they build partitions studies with NewStudyFilters.ExcludedStudies, which reads
the very ScreeningInfo.InclusionInfo[] field the three recalculation passes are mid-rewrite
(StudyRepository.cs:1224-1249).
Impact. FullStats can today be served against a partially rewritten inclusion partition. The plan
makes ActiveInclusionInfoCalculationJob a source-visibility fence under which both materialized and
authoritative bundles return a typed StatisticsInclusionRecalculationInProgress/503
(Event and invalidation contract). Adopting that
fence therefore changes FullStats behaviour — from silently serving a partial population to failing
closed — and that change is a feature of the plan, not a regression, but it must be recorded as an
intended behaviour change at cutover.
Options. (a) Extend the existing guard to FullStats in a separate PR before Phase 2, so the fence
behaviour is established on the authoritative path first. (b) Introduce the fence only with the
materialized path, accepting that the two paths differ during rollout. © Leave FullStats unguarded
and fence only the materialized reads.
Recommended default: (a). (b) and © both violate the plan's requirement that materialized and authoritative responses agree, and (a) makes the eventual cutover a no-op for this behaviour.
D12: The question tally has no stage or version scope key¶
Evidence. The plan's family row for Question answers specifies a
Project + stage + question/version scope and requires question scopes to carry immutable
definition/version identifiers so that renaming or deleting a definition cannot silently change historical
meaning (Catalogue baseline). The current calculation groups by
ExtractionInfo.Annotations.QuestionId alone (StudyRepository.cs:951-956), with no stage and no version
dimension. There is no version model to key on: AQVersion, QuestionSetVersion and SessionVersion
have zero matches in src, and question content is mutated in place by AnnotationQuestion.Update
(AnnotationQuestion.cs:218-238) with no history. The Identity-plus-immutable-versions pattern described
in CLAUDE.md and the platform architecture is target ("QM v2") architecture, not current state.
Impact. The plan's stage and version scope keys for this family cannot be populated from current data. A materialized question family can either adopt the current project-plus-question grain and defer the version key, or block on a versioning model that this programme has no authority to introduce.
Options. (a) Materialize at the current (ProjectId, QuestionId) grain, and record that the stage and
version key components are reserved-but-unpopulated until a question-versioning model exists. (b) Derive a
synthetic version identity from question content, which would be a new domain concept. © Defer the
question family past Phase 3 until versioning lands.
Recommended default: (a). The plan's requirement is that a historical observation must remain interpretable; recording an explicitly reserved, unpopulated version component satisfies that without inventing domain state, and the reserved key means old observations are never silently reinterpreted when versioning does arrive.
D13: Search and project deletion are fail-closed 503 today¶
Evidence. SearchController.DeleteSearch (SearchController.cs:77-87) and DeleteSearchImportJob
(:89-99) have bodies that are a bare
throw DeletionLifecycleUnavailableException.ForSystematicSearch(...), mapped to 503 with
code: "deletion_lifecycle_unavailable" by a globally registered filter
(src/services/api/SyRF.API.Endpoint/Infrastructure/DeletionLifecycleUnavailableExceptionFilter.cs:6-66,
registered at Program.cs:125). ProjectController.DeleteProject (ProjectController.cs:280-291) is
identical, and ProjectControllerTests.cs:96-111 asserts the throw for both values of the
DeletionLifecycle flag. PendingDeletion has zero matches in src; there is no soft-delete marker and
no sweep. The underlying implementations exist but are unreachable:
ProjectManagementService.DeleteProjectAsync (:538-565) and
RemoveAndDeleteSystematicSearchFromProjectAsync (:616-642) have no production callers.
Impact. The plan's event matrix carries a Systematic search create/import/rollback/delete row and a
Project deletion row. Neither deletion path can be exercised, benchmarked or parity-tested on current
main, so those rows cannot be validated in Phase 0 and their delta design rests on unreachable code.
Options. (a) Record both as design-only rows explicitly deferred until the deletion lifecycle ships, and exclude them from Phase 0 benchmark datasets. (b) Benchmark against the unreachable implementations by calling them directly from a test harness. © Block the affected families until deletion is reachable.
Recommended default: (a). (b) would benchmark code that is not on any live path and whose eventual shape is governed by a separate deletion-lifecycle design; © would block Phase 2 on unrelated work.
Appendix A: corrections applied¶
The following anchors were carried in the read-only research notes or in the drafting brief and are
corrected here. Each was re-checked directly against main at 7e0ed90c9.
| # | Claimed | Verified reality | Source of the error |
|---|---|---|---|
| 1 | DisableMembership does not exist |
It does exist, as a NotImplementedException stub with zero live call sites: ProjectMembership.cs:237-240, called only by Project.DisableMembership (Project.cs:597-604), which no controller invokes; a stale test at ProjectShould.cs:225 asserts an InvalidOperationException the stub never throws |
Drafting brief; the research notes had it right |
| 2 | ParseBulkStudyUpdateAsync at .../Services/ParserImplementations/StudyReferenceFileParser.cs:166-211 |
The file is src/libs/project-management/SyRF.ProjectManagement.Core/Services/StudyReferenceFileParser.cs (not under ParserImplementations/) and the method begins at :137; batch size 400 at :173 and :214, concurrency 8 at :169-170, AddScreening at :195, SaveManyAsync at :200, ApplySimpleUpdates at :231 |
Research notes |
| 3 | ParseStudiesAsync at .../ParserImplementations/StudyReferenceFileParser.cs:63-77 |
Same path correction; the method is at StudyReferenceFileParser.cs:63 |
Research notes |
| 4 | PendingImportJobId is the staged-visibility marker for imports |
Zero matches in src. Staged visibility is two-tier: Study.SystematicSearchId is populated during parsing, and the SystematicSearch document plus Project.SystematicSearchIds appear only at ProjectManagementService.CompleteSearchImportJob (:370-394) |
Stale March–April plan naming |
| 5 | SearchImportJobConsumer owns import |
Zero matches. The owners are the saga SearchImportJobStateMachine (src/services/project-management/SyRF.ProjectManagement.Endpoint/Sagas/SearchImportJobStateMachine.cs:15) and ReferenceFileParseJobConsumer (.../Consumers/ReferenceFileParseJobConsumer.cs:11) |
Stale March–April plan naming |
| 6 | PendingDeletion marks a reversible deletion |
Zero matches; deletion endpoints throw 503 unconditionally (D13) |
Stale March–April plan naming |
| 7 | ResetSessionsForStagesAsync implements completed-to-incomplete session reset |
Zero matches. No question or stage edit resets a session status; the only writer of session status is AnnotationSession.UpdateStatus (src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/AnnotationSession.cs:36-43), called solely from ExtractionInfo.AddAnnotations (ExtractionInfo.cs:164-169), i.e. a reviewer's own submission |
Stale March–April plan naming |
| 8 | ActiveInclusionInfoCalculationJob is a scheduled job |
It is a field on the Project aggregate acting as an in-flight mutual-exclusion token (Project.cs:194; class at :1573-1576). The asynchronous worker is the MassTransit consumer UpdateStudyScreeningStatsConsumer (src/services/project-management/SyRF.ProjectManagement.Endpoint/Consumers/UpdateStudyScreeningStatsConsumer.cs:12) |
Stale March–April plan naming |
| 9 | GetFullProjectStatsAsync spans StudyStats.cs:93-126 |
The method signature line is :94; the surrounding range is correct |
Research notes, off by one |
| 10 | GetInvestigatorScreeningStats spans StudyRepository.cs:287-307 |
:287-306; GetMembershipFor is called at :305, and GetInvestigatorAnnotationStats spans :218-283 with its GetMembershipFor at :279 |
Research notes, off by one |
| 11 | The $facet block is at StudyStats.cs:356-438 |
The .Facet<Study, BsonDocument> call begins at :355; threshold inputs are resolved at :292-295, minNumberSessions = 2 at :313 with its TODO at :311-312 |
Research notes, minor drift |
| 12 | AllScreeningPipelineGroupStage at :274-276, InvestigatorScreeningPipelineGroupStage at :278-281 |
Confirmed: the declarations are at :274 and :278 respectively |
No correction; verified |
| 13 | ProjectDailyStat has exactly one matching file |
Confirmed: three matches, all three inside ProjectDailyStat.cs (:4, :6, :8) |
No correction; count clarified |
| 14 | Research open question: does Screening.ChangeScreeningDecision retain prior-value history? |
Closed. src/libs/project-management/SyRF.ProjectManagement.Core/Model/StudyAggregate/Screening.cs:32-37 assigns ScreeningDecision and StageId and returns true. No prior value and no timestamp is retained, and the Screening entity has no time field at all (Screening.cs:23-30). A rescreen is an in-place overwrite that leaves Screenings.Count unchanged |
Research notes, left open |
| 15 | Research open question: is Project.Errors bounded? |
Closed. Project.cs:117 declares public List<Object> Errors { get; set; } = new(); — untyped and unbounded, sitting on the same document as CompletedInclusionCalculationJobs (:195) and AnnotationQuestionAnswerTally (:205) |
Research notes, left open |
Appendix B: verification method¶
Every citation above was produced by reading the named file in /home/chris/workspace/syrf/main at
commit 7e0ed90c9 on 2026-09-01, either with a line-addressed read or a grep -n whose output was
inspected. Absence claims ("zero matches") were produced with
grep -rn --include='*.cs' <name> src, excluding obj/ build output, and are stated only where the
search returned no non-excluded hits. The worktree was not modified: no file was written, no branch was
switched and no git state was changed.