- Learn
- Align
- Perform
- Review
Review5 min read
The After-Action Review Is Where Review Happens
By J Damien Scott, Trusted Advisor
Exercises produce findings. After-action reviews produce change, and only when the corrective action has an owner, a date, and a place in the next plan. Four conflict-affected operating environments, two country evacuations, and a post-earthquake recovery taught what a rigorous after-action review looks like, who has to own what comes out of it, and how findings feed threat intelligence rather than a filing cabinet.
Exercises produce findings; reviews produce change
The Homeland Security Exercise and Evaluation Program describes exercises as the opportunity to shape planning, assess and validate capabilities, and address areas for improvement. Its evaluation step documents strengths, areas for improvement, capability performance, and corrective actions in an After-Action Report and Improvement Plan. The doctrine is explicit that improvement planning is where organizations take the corrective actions needed to improve plans, build and sustain capabilities, and maintain readiness. The exercise is the cheap part. The improvement plan is where the value is created or lost.
NIST's guide to test, training, and exercise programs, SP 800-84, makes the same separation for information technology contingency planning: the exercise exists to prepare the organization to respond to and recover from adverse events, and it only does so if the results change something. The Cybersecurity Framework 2.0 carries the requirement into its outcome ID.IM-02: improvements are identified from security tests and exercises. Identified is a low bar. The rest of this article is about clearing it and then going further.
“A finding with no owner is a sentence in a document. A finding with an owner and a date is a change to the program.”
What a rigorous after-action review looks like
Across four Level 4 and conflict-affected operating environments, Haiti, Afghanistan, Kurdistan, and South Sudan, I ran an extensive program of tabletop exercises, scenario planning, and drills built specifically to test business continuity and recovery under disruption, with after-action review as the closing step of every one. The environments did not permit exercises that ended in a debrief and a coffee. The scenarios were later run for real: a full-scale country evacuation from Kurdistan during acute civil unrest, and post-earthquake disaster recovery in Haiti that held the client's business continuity intact while coordinating life support and logistics for more than 60 expatriate personnel.
The review that works has a fixed structure. First, the timeline is reconstructed from records, not memory, because memory in a crisis compresses and reorders. Second, each decision point is examined against the plan: what the plan said, what was done, and why they differed. Third, every difference is classified as a plan failure, a training failure, a resource failure, or a correct deviation from a plan that was wrong. Fourth, each classified finding becomes a corrective action with a named owner and a completion date. Fifth, the review closes with the list read aloud and each owner acknowledging the item.
The classification step is where most reviews go soft. A team that has performed under pressure wants to record that it performed well, and it usually did. The review is not a judgment of the team. It is a judgment of the plan, and a plan that survived only because people improvised around it has been found deficient, no matter how well the improvising went.
Who owns the corrective action
FEMA's improvement-planning guidance states that an effective corrective action program develops improvement plans that are dynamic documents, with corrective actions continually monitored and implemented as part of improving preparedness. Dynamic means someone is holding the list open. In practice that is one accountable person per finding, with the authority to change the plan, the training, or the resource that failed, and a date by which the change is verified rather than promised.
Ownership has to sit at the level that can act. A finding that the emergency communications plan named a telephone number that no longer worked is owned by whoever maintains the plan, not by the team lead who discovered it at two in the morning. A finding that the evacuation assembly point was unreachable under the actual road conditions is owned by the security manager who set it, and the fix is verified by driving the route, not by editing a document.
The Institute of Internal Auditors' Three Lines Model is useful here as a discipline rather than a diagram. The people who own the plan fix the plan. Someone independent of them checks that the fix landed. When the same person writes the finding, owns the fix, and signs it off, the after-action review has become a diary.
Feeding findings back into intelligence and capability
Review feeds Learn. The findings from an after-action review are the most reliable intelligence a program has about its own vulnerabilities, and they belong in the threat model alongside the external reporting. A drill that showed the compound could not be sealed in under twenty minutes has told the threat analyst which adversary courses of action are viable. A review that found the medical evacuation plan depended on a single aircraft operator has written the next capability requirement.
NIST SP 800-61 Revision 3 describes the older incident response model as a separate team performing post-incident activities that identified needed improvements and fed them into the preparation stage. It then argues that the current state demands more: lessons learned should be shared as soon as they are identified, not delayed until after recovery concludes, and continuous improvement is necessary across every facet of risk management. The protective equivalent is an after-action finding that reaches the country security plan the same week, not at the annual review.
The measure of whether the loop closes is plan currency. Across those four environments, country security plans, site assessments, route risk assessments, and mitigation actions were held at 98% or better currency against fast-changing threat conditions and client requirements. Currency is not a document-control statistic. It is the proportion of the plan that has absorbed what the environment taught since the last exercise.
The filing cabinet
The failure mode is familiar to anyone who has read an old after-action report. The report is thorough, the findings are sharp, and the corrective actions are listed without owners, without dates, and without a place in the next plan. Two years later the same finding appears in a new report, written by a new team, in slightly different words. The organization has learned nothing and documented everything.
The fix is not a better template. It is the decision, made before the exercise, that the review will end with named owners and dates, that an independent person will verify closure, and that the findings will be written into the threat model and the plan on a stated schedule. An after-action review run that way is the Review phase of protection in its purest form: assess effectiveness, test the controls, feed the findings back, and refine the measures based on what the environment teaches.
Field Notes · by email
One email when a new article publishes. Nothing else.
Field notes on converged security from J Damien Scott, Trusted Advisor: the article, its summary, and the phase it belongs to. No digests, no offers, no third party reading over your shoulder.
Email delivery is being set up. The feed carries every article the day it publishes. About Field Notes
Related reading
More from Review
Review · 11 September 2026
Post Coverage Is a Protective Audit, Not a Finance Task
An unfilled post is an unprotected site, and the record that proves the post was filled is the same record that bills the client. Redesigning timekeeping controls, billing reconciliation, contract compliance, post coverage validation, and exception review cut revenue leakage by 95% at a 127-account security enterprise. The finance result was real. The protective result was larger, and it is the one most security leaders never claim.
5 min readReview · 9 September 2026
Three Numbers a Protective Program Must Report
Activity counts are not measures. A protective program that reports patrols completed and alerts reviewed is describing effort, and effort is not what leadership is paying for. Three numbers describe effectiveness: detection time, response time, and mitigation effectiveness. Each needs a defined clock and a defined denominator, or the number is theatre. This article sets out how each was defined and moved in practice.
5 min readReview · 8 September 2026
The Internal Audit Is a Protective Control
Operations cannot see its own gaps from inside. An internal audit against a standard and the organization's own documented practice finds what daily work hides, and the closing meeting is where leadership decides what to do about it. Six ISO/IEC 27001 Clause 9.2 audits delivered for client organizations show how the method works, and why protective programs, which are almost never audited this way, need it most.
5 min read