example	limitation_id	severity	covered_claim	not_supported_claim	mitigation
Airway	AIRWAY-METHOD-MISMATCH	major	Direction and ranking agreement for Dex response	Exact significant-gene count or per-gene p-value/FDR reproduction	Report Spearman rho, sign concordance, sentinel genes, overlap, and Jaccard together
Airway	AIRWAY-RESUME	minor	Validated completion of the formal result directory	Independent recomputation of every resumed step	Retain command, checksum, run-summary, and repair provenance
ST93 MRSA	WGS-NO-CORE-SNP	major	MLST and mecA endpoint recovery plus external SPANDx six-isolate pairwise-distance recovery	Core-SNP phylogeny as an ABI plugin module, or full reproduction of the paper's no-recent-outbreak conclusion	Report the strict comparison tracks alongside the plugin endpoints; state that the paper conclusion also depends on tree topology and broader NT/background isolate context
ST93 MRSA	WGS-SNP-TOOLCHAIN-DELTA	major	Paper SPANDx v2.6 track pairwise range 7-60 as a paper-method distance endpoint	Treating either the paper-track pairwise range or the ABI-adjacent bcftools range 10-73 as a full paper-exact reproduction	Label the bcftools track abi_reproduction_not_paper_method and report both tracks side by side
ST93 MRSA	WGS-SMALL-COHORT	minor	Recovery within the six-isolate study set	General MLST or AMR accuracy across species and lineages	Use additional organisms and resistance mechanisms for external validation
SCAPP	SCAPP-FP-INTERPRETATION	major	Strict reference concordance (precision 12/157, recall 64/88) under the frozen paper-method reconstruction	Treating the 145 unmatched predictions as proven biological false positives	Report auxiliary evidence stratification (circularity, PlasmidFinder, MOB replicon, relaxase, oriT, abundance) alongside concordance metrics
SCAPP	SCAPP-DB-SCOPE	major	Paper-method reconstruction against the frozen public PLSDB archive	Paper-exact reconstruction of the unpublished 13,469-record deduplication	Record both archive sizes and label the evaluation paper-method reconstruction, not paper-exact
Local-model benchmark	BENCH-OLD-RESULTS-EXCLUDED	critical	A new preregistered local-model experiment	Any claim based on pre-existing benchmark outcomes	Do not import old scores; start from a clean commit and new study manifest
Local-model benchmark	BENCH-LOCAL-GENERALIZATION	major	Performance of measured self-hosted model configurations	All local models or hosted frontier models	Use at least three model families with matched small/medium configurations and family-level inference
Local-model benchmark	BENCH-QUANTIZATION	critical	Within-configuration G1/G2/G3/G4 comparisons	Parameter-scale effects from unmatched native and quantized runs	Match precision/quantization within family and record a runtime attestation