A regulatory submission is a set of documents that must agree with one another. The protocol, the statistical analysis plan, the clinical study report, the tables and listings, and the summary documents all state numbers, definitions, endpoints, and populations. A mismatch, such as a different sample size in two places or an endpoint defined two ways, can trigger questions and delay. Finding them is careful, tedious reading across thousands of pages, usually near a deadline.
Cross-document consistency checking is a good fit for current models. They accept very long inputs, they read closely when the question is specific, and each finding can point to a place in a document that a reviewer can verify in seconds.
The design
This is a parallelization workflow with a checking step. The checks are known in advance, so a fixed set of workers is better than an open-ended agent.
- Document setprotocol, SAP, CSR, tables
- Indexsections, definitions, key values
- Check workersone per check type, in parallel
- Verifysecond model re-reads each finding
- Rankseverity, dedupe
- QC revieweraccepts or rejects
- Findings logstatus, reason, source
- Index. A first pass records definitions, populations, endpoints, and key numeric values with their locations. This gives the workers a map instead of making each one reread everything.
- Check workers. Each worker owns one kind of check: numbers that should match, terms defined differently, cross-references that point to the wrong section, abbreviations used before definition, and statements in a summary that the detailed section does not support. Narrow objectives work better than one request to find all problems. This follows Anthropic's finding that separate calls for separate considerations often outperform one call handling everything.
- Citations. Every finding states the two places that disagree, with the quoted text and the page or location. A reviewer opens both and decides.
- Verify. A second pass re-reads each finding against the source and removes ones it cannot support. False positives destroy reviewer trust faster than missed items do, so this step is worth its cost. This is the evaluator-optimizer idea applied to filtering.
- Table values. For numbers inside tables and listings, extract them to structured data and compare in code. Do not ask the model to compare two columns of figures by eye.
Context handling
These document sets are large. Anthropic's context engineering guidance applies directly: context is finite, recall degrades as it grows, and the aim is the smallest set of high-signal tokens. Give each worker the sections relevant to its check plus the index, not everything. Use the long window where a check needs cross-document view, and keep notes outside the context for long runs.
Validation and quality systems
Regulated firms will ask how this fits their quality system. Treat the system as a QC aid. It adds a check and does not replace the review of record. Your validation team will have requirements for computerized systems used in GxP work, and in the US for electronic records and signatures under 21 CFR Part 11. Involve them from the start with the evaluation plan, the versioning approach for prompts and models, and the audit trail. Pin model versions and re-run the evaluation before changing them.
How to evaluate it
- Seeded errors. Take a clean, already-reviewed document set and insert known inconsistencies of each type. Measure recall per check type.
- Historical findings. Run it on past submissions where QC found real issues. Did the system find them?
- Precision. What share of reported findings does a reviewer accept? Track by check type, and tune or retire checks with low precision.
- Reviewer effort. Time to disposition each finding. If it is high, the citations are not doing their job.
Report the types the system misses as plainly as the ones it finds. A reviewer who knows where it is weak knows where to read.
This is a reference design. It describes how we would build the system and cites Anthropic's documentation for platform facts. It is not a client case study, and the figures it uses are list prices or worked examples, not results.
Have a process like this?
Bring it to a 45-minute working session.