Part 1. How to Choose and Use a Check
Choose a Check for the Mistake That Matters
A quality control answers a specific question about a piece of work. Start with the mistake you need to catch and the consequence of missing it. Check only what the supplied material can establish. Keep a record of the rule, the evidence, the finding and the decision that follows.
For an indicator record, you might check that the required source field is present. For a progress report, you might check a quotation against its source. For a financial narrative, you might recompute its total. Passing one of these checks does not establish that the document is sound overall.
A Short Workflow
- Name the material and the decision it will inform.
- State the possible mistake and choose an applicable control.
- Supply the actual source and the rule. Identify unavailable evidence.
- Inspect the finding and its location. Keep unsupported extra findings separate.
- Make an appropriate revision, then repeat the relevant check.
- Retain the evidence and route the final professional decision to its owner.
Three Instructions to Copy
Paste one into an AI assistant with your record or document. Each tells the reviewer what to check and what not to conclude.
Required fields
Check this record against the supplied required-field and type list. Identify missing required fields and wrong types by location. Optional fields may be absent. Do not treat a structural pass as a quality judgment.Quotations
Compare each quotation with its declared source location using the stated matching rules. Report a literal mismatch or unresolved location. Do not infer that a quotation is relevant or supports the surrounding interpretation merely because the wording occurs.Arithmetic
Recompute the total from the named components in the same unit. State any mismatch and the components used. Do not add historical comparators or subtotals twice. If units, rounding or missing values are unresolved, report that limitation.Four Checks, Each with Its Limit
These four small examples use made-up, fictional inputs and no client material. Each shows a check working and the point where it stops. Schema validation, exact-match grounding, arithmetic reconciliation and gates with human routing are established practice; nothing here claims to invent them. No AI model was used: each example is a short deterministic script.
1. Schema
3 of 3 cases matched (1A pass, 1B fail, 1C boundary)
What it shows. An indicator record missing its unit, with the baseline written as text, is flagged by field and reason.
The boundary it cannot cross. Case 1C has every required field with the right type, so it passes. Yet "Things improved" with the unit "yes" and a target equal to the baseline is a poor indicator. A schema check says the record is well formed, not that the indicator is good.
2. Exact Quotation
5 of 5 cases matched (2A pass, 2B and 2B2 fail, 2C and 2D boundary)
What it shows. A quotation is tested as an exact, case-sensitive piece of the supplied text. An invented quote (2B) and a lightly altered one (2B2) are not found.
The boundary it cannot cross. Case 2C is real text, but the speaker said "I would not say the training was useful for record keeping", so the quote reverses the meaning and still passes. Case 2D credits Speaker B’s sentence to Speaker A and also passes. Presence is not meaning and not attribution.
3. Arithmetic
3 of 3 cases matched (3A pass, 3B fail, 3C boundary)
What it shows. Components are summed against a reported total, and a percentage is recomputed from its stated numerator and denominator. Case 3B flags a total that is off by 10 and a stated 30.0 percent against a computed 30.8.
The boundary it cannot cross. Case 3C reconciles (45 of 90 is 50.0 percent), but the claim says "of all 250 households" while the denominator named is the 90 North-only households. The check trusts the denominator it is given.
4. Stop and Handoff
3 of 3 cases matched (4A pass, 4B fail, 4C boundary)
What it shows. Any failed check stops the run and writes a handoff record (id, run, role, failed checks, time, state). A run where every check passes is released with no record.
The boundary it cannot cross. In case 4C a person logs "fixed it". The record says acknowledged but unverified, because a logged acknowledgement is not proof the issue was fixed. The demo never outputs a resolved state and does not include re-check or sign-off.
What a Check Cannot Do
- A structural pass is not a quality judgment: a complete indicator record can still be a poor indicator.
- A real quotation can still be misread. The words being present does not verify who said it or what it supports.
- Correct arithmetic can still use the wrong denominator or population.
- A flag sent to a person is not evidence that the person resolved it.
Keep an Evidence Record
For each check, write down the following. Separate what you observed from what you propose to change.
- Document and version
- Control selected and the rule applied
- Source location
- Result
- Uncertainty
- Revision or escalation
- Decision owner
Part 2. Evidence: What Has Been Tested
Two small checks have been tested on their own. No study of the controls as a set has been run.
What Was Run
- Saved Component Checks: an arithmetic equality check and a literal-containment (exact quotation) check, 12 saved cases each, 24 in all. Every case gave its expected result. The cases use supplied, already-extracted figures and text, and some overlap. The cases were saved on Oct 26 and the counts re-read on Nov 7; the tests were not rerun for this page.
- The Four Examples Above: 14 hand-built cases, 14 of 14 matching what was written down in advance, and no model calls. They are small illustrations chosen to show each boundary, not trials. Fourteen cases cannot support any rate or effectiveness claim.
What This Does Not Show
The saved checks show that those two functions behave as specified on those inputs. They do not show:
- that the checks find the right figures or quotations on their own
- that any other control works
- that combined checks reduce errors
- that a person resolves what a check flags
What Is Still Planned
A study comparing ordinary model review with explicit control instructions on matched examples is planned, not run, and not yet approved. No held-out effectiveness result exists. When results exist, each row will show the tested property, the number of distinct cases, the condition, detections with denominators, wrong flags and failures. Unknown values will stay unknown.
Where to Go Next
Tried a check on your own material? Tell us which instruction, example or wording worked, and which did not. Specific feedback is the most useful kind.