Thursday Field Note · Decision guide
Before AI Learns From a Correction, Decide What the Correction Is Allowed to Become
A practical five-question gate for deciding whether one Human correction should stay local, become bounded shared context, or expire.
Operating tension: reusable corrections can compound capability or quietly spread one person's exception as policy.
Decision: keep the correction local, promote it with boundaries, or retire it.
A correction is evidence; it is not automatically organizational truth.
A person fixes an AI-generated answer. The next temptation is obvious: make the system remember so nobody has to fix the same thing again. Sometimes that is exactly right. Sometimes the person approved a one-time exception, interpreted incomplete evidence, or solved a case whose conditions will never repeat.
The dangerous step is not correction. It is silent promotion: a local judgment crosses into shared context without a decision about authority, scope, conflict, or expiration.
The valuable correction is not merely remembered. It is classified, bounded, checked for conflict, and assigned a review condition.
Ask Five Questions at the Gate
Before a correction influences another case, ask:
- Who had authority to correct the work? Expertise, approval authority, and proximity to the consequence are not interchangeable.
- What exact scope did the correction cover? Name the customer, product, jurisdiction, date, workflow state, and exception that made it appropriate.
- Does evidence show recurrence? One case proves that one case occurred. Reuse needs evidence that the pattern travels.
- Does it conflict with accepted context? Compare the correction with current policy, source records, other approved corrections, and unresolved disagreements.
- When should it expire or be checked again? Connect reuse to a date, source change, policy revision, performance signal, or responsible owner.
NIST's voluntary AI Risk Management Framework Playbook recommends post-deployment monitoring, feedback mechanisms, documented risk decisions, and controls that can include rectification or deletion.1 It does not prescribe this guide. It does reinforce the reason the guide exists: system value and risk can change after deployment.
Choose One of Three Outcomes
Useful Here; Unproven Elsewhere
Attach the correction to the original case and preserve its reasoning. Do not let it influence another case until recurrence and authority are clearer.
Reusable Under Named Conditions
Record the allowed scope, evidence, approver, conflicts checked, review date, and the signal that withdraws the correction.
Do Not Carry It Forward
Supersede an outdated correction, reject an unsupported one, or preserve it only as incident history so the system does not relearn the same mistake.
NIST's initial TEVV-Athlon draft proposes tailoring AI evaluation to organizational objectives and real-world impacts.2 Because it is still a draft, it should not be treated as a final standard. Its useful reminder is that a correction's value depends on the objective and context in which it will be reused.
Test the Rule With One Ordinary Exception
Consider a synthetic customer-support example. A long-standing customer asks to return equipment outside the ordinary window after a documented shipping disruption. A manager approves the exception. If the system stores “late returns are allowed,” it has converted one authorized accommodation into a general policy.
The better record is narrower: which customer condition mattered, who approved the exception, what evidence supported it, whether similar cases should be escalated, and when the guidance must be reviewed. The system can become more useful without pretending the exception changed the rule.
Expiry is not lost intelligence. It prevents an old lesson from governing a new situation after its evidence, policy, or owner has changed.
What This Guide Does Not Decide
This guide is not a database design, retention schedule, legal rule, or proof that a correction is correct. It cannot replace domain expertise, privacy review, recordkeeping obligations, worker consultation, or deeper evaluation for consequential systems. NIST's Generative AI Profile likewise treats monitoring, incident response, recovery, and deactivation as broader organizational responsibilities.3
Its purpose is smaller and practical: place a decision between correction and reuse. The system may learn; the organization first decides what the lesson is allowed to become.
Review the Last Correction Your Team Asked AI to Remember.
Write its authority, scope, recurrence evidence, conflicts, and expiry condition. If those answers are missing, keep it local until the lesson earns a wider field.
Research Record
References and Evidence
Sources were rechecked on September 10, 2026. NIST materials are voluntary guidance or an initial public draft; they do not endorse the decision guide, CPF, or VerShep. The five questions, three outcomes, correction gate, and return-policy example are my operating model and a synthetic illustration, not measured customer results, legal advice, or a data-retention specification.
- NIST AI Risk Management Framework Playbook: ManageVoluntary Government Guidance · NIST AI Resource Center; rechecked September 10, 2026
The Playbook recommends post-deployment monitoring, end-user feedback, documented risk acceptance, Human-AI configuration tracking, and controls for rectification or deletion. The correction decision guide is my narrower operating interpretation; NIST does not endorse it or VerShep.
- The TEVV-Athlon Framework for Evaluating AI SystemsInitial Government Standards Draft · NIST AI 200-2 Initial Public Draft; August 2026; rechecked September 10, 2026
NIST proposes a customizable method for evaluating AI systems against organizational objectives and real-world impacts. It remains an initial public draft open for comment through October 6, 2026; it is not a final standard.
- Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence ProfileVoluntary Government Guidance · NIST AI 600-1; July 2024; rechecked September 10, 2026
The profile emphasizes monitoring, incident response, recovery, downstream communication, and mechanisms to supersede or deactivate AI systems. It supports the need to govern change over time; it does not prescribe the three outcomes or five questions used here.