# Internal Controls Decision Guide: Five-Step Framework to Act at 95% Detection

Hunter Gibson · October 20, 2026

> Takeaway Detail A control exception is actionable only if its threshold was pre-registered with an error-cost ratio and a 5% false-negative budget before testin

| Takeaway | Detail |
| --- | --- |
| A control exception is actionable only if its threshold was pre-registered with an error-cost ratio and a 5% false-negative budget before testing. | The rule requires β = 0.05 (95% detection power) and an explicit error-cost ratio documented prior to any test execution. |
| The flag’s evidence package must contain three items: input-data snapshot, logic version, and reviewer sign-off. | Missing any one of these three components invalidates the flag, regardless of detection power. |
| The framework uses a PCA-based similarity-window for vegetation controls built from 2001–2024 NDVI and precipitation records. | Thresholds are examined for spatial structure and associated changes in vegetation controls using this dataset. |
| Cost-sensitive threshold tuning is reported to improve robustness in accelerometer-based fall detection via a computationally efficient framework; check the cited study's reported results before relying on them for your control. | Results show enhanced detection reliability when thresholds are adjusted for cost sensitivity. |

This guide delivers a five-step framework for acting on control exceptions only when pre-registered thresholds fire at ≥95% detection power.

Every actionable flag must carry an input-data snapshot, logic version, and reviewer sign-off to be valid.

![Internal Controls Decision Guide](https://static.mm-ais.com/article-images-ai/internal-controls-decision-guide-five-st-ai-067cdfa9.jpg)

## How 95% Detection Actually Works in a Control

Begin with the control objective and the error-cost ratio, because every downstream decision derives from what you are trying to stop and what you are willing to pay to stop it. Next, define the evidence unit and the population so that the rule applies to a precise set of records and the threshold can be calibrated against a known base rate. Only then set the threshold itself, reserving a 5 % false-negative budget so that the rule fires with at least 95 % detection power (β = 0.05). After the threshold is locked, run the detection with full metadata capture, and finally schedule a re-tune or retirement date. Starting at step three—picking a threshold before the objective or population is fixed—is the single most common cause of an indefensible detection.

The Detection Engineering Framework (Ke0xes, GitHub) carries each detection through its full lifecycle: it starts with the business reason the detection exists, moves through telemetry selection and logic authoring, then enters the calibration phase where the threshold is set against the pre-registered error-cost ratio and the 0.05 false-negative budget. Once deployed, the framework mandates that every alert carry an evidence package containing the input-data snapshot, the threshold and logic version, and a reviewer sign-off. A flag that is technically correct but missing any one of those three items remains unactionable, because the audit trail is incomplete.

Adopting this lifecycle prevents the common failure mode of tuning a threshold in isolation. When the objective, population, and error-cost ratio are fixed first, the threshold is derived from the arithmetic rather than from intuition. The framework also enforces that each detection be revisited on a schedule, so that drift in the underlying data or business conditions triggers a re-tune or retirement instead of silent degradation.

Convergence on a single threshold rule comes from four sources: the business objective, the population definition, the error-cost ratio, and the false-negative budget. Each source contributes a constraint, and the intersection of those constraints yields the threshold. Skipping any one source invites a rule that is either too sensitive, too conservative, or unexplainable to stakeholders.

| Source | Requirement |
| --- | --- |
| Business objective | Define what must be stopped and the cost of missing it |
| Population definition | Specify the exact set of records the rule will evaluate |
| Error-cost ratio | Pre-register the acceptable trade-off between false positives and false negatives |
| False-negative budget | Reserve 5 % of detections for the false-negative allowance (β = 0.05) |

![How 95% Detection Actually Works in a Control — Internal Controls Decision Guide](https://static.mm-ais.com/article-images-ai/internal-controls-decision-guide-five-st-ai-ff20e9b5.jpg)

## Five Sources That Converge on One Threshold Rule

A control flag becomes auditable only when every source points to the same threshold rule and the same evidence trail. The baseline definition comes from AWASC’s January 23, 2026 guide: A detection threshold is the specific value or rule that determines whether a measured signal indicates the presence or absence of a target. Such a threshold is required when measurement noise or background variability could make that distinction ambiguous. The practical test is simple: Could another reviewer apply the rule to the same data and reach the same classification? If the answer depends on judgment rather than an explicit threshold or condition, the flag is not yet reproducible.

| Source | Specific requirement it contributes |
| --- | --- |
| AWASC, “How Detection Thresholds Are Defined” | State the value or rule that separates target presence from absence, especially when noise or background variability is present. |
| arXiv 2509.11789, “Watch Your Step” | Use cost-sensitive threshold tuning rather than a default cutoff when the costs of different detection errors are unequal. |
| Microsoft FrameworkDetector | Preserve relevant metadata—including input data and the checks performed—alongside results so that determinations remain open to audit. |
| Detection Engineering Framework | Connect the detection’s business purpose to its telemetry, logic, and operational lifecycle so that the threshold remains tied to its intended control purpose. |

The second source builds on the first. The September 15, 2025 arXiv study reports that cost-sensitive threshold tuning improves the robustness

s of accelerometer-based fall detection compared with a default cutoff. In a control setting, the check is not merely whether a threshold improved performance; it is whether the threshold was registered before testing with the relative error costs made explicit. Otherwise, moving the cutoff after seeing results can turn a weak signal into a favorable-looking flag. The fifth source, AI Website Detector’s July 29, 2026 framework comparison, adds a job-fit check: the final detector choice should match the task and rest on evidence-led analysis rather than on a generic claim that a tool “detects.”

For an exception to be actionable, the pre-registered rule must fire at the required detection power, including the specified false-negative budget of 5%, and its evidence package must contain three items: a snapshot of the input data, the threshold or logic version, and reviewer sign-off. A flag can therefore be technically correct yet still be non-actionable if its input snapshot is missing, its logic cannot be reconstructed, or no reviewer has accepted responsibility for the determination.

Use this final review sequence. Confirm that the threshold was written before testing, verify that it was tuned to the registered error-cost ratio, and check that the flag was produced by that exact version. Then reconcile the stored input snapshot with the observed signal, attach reviewer sign-off, and retain the evidence with the exception. This is the convergence point: definition, cost-sensitive tuning, reproducible execution, lifecycle traceability, and task-appropriate selection all support one threshold rule—and only a flag supported by that rule and its complete evidence package warrants action.

![Five Sources That Converge on One Threshold Rule — Internal Controls Decision Guide](https://static.mm-ais.com/article-images-pixabay/internal-controls-decision-guide-five-st-29ab0517.jpg)

## Three Threshold Methods Compared, One Winner

Three threshold methods compete for the control slot, and only one survives an audit. Option A is the single dollar cut-off: “review anything over a fixed amount.” It is easy to state and cheap to run, but it is defeated by splitting transactions just below the line, and its defensibility is low because no cost ratio or power target is ever recorded. Option B is the fixed attribute sample with a pass/fail tolerance. It is defensible on sampling theory, yet the tolerance is usually set by convention rather than by the error cost of the control. Option C is the cost-sensitive pre-registered rule: a written rule, a declared β = 0.05, and a captured metadata package. It is the only method that can be audited after the fact, because the reviewer can re-derive the threshold from the recorded error-cost ratio and verify that the detection power was met before the test began.

The winner is Option C, and the reason is not elegance—it is that a rule ignoring the base rate will miss a large share of real events. Option C forces the analyst to state the error-cost ratio up front, which in turn fixes the sample size and the detection power. The arithmetic is simple: if the cost of a false negative is six times the cost of a false positive, the ratio is 6:1, and the threshold must be set so that the expected loss is minimized. The rule is then locked, the β is declared, and the evidence package is captured before any data is touched.

The evidence package must contain three items, and a correct flag missing any of them is not actionable. First, the input-data snapshot: the exact rows, columns, and time window that were scored. Second, the threshold and logic version: the numeric cut-off or rule expression, plus the software commit or model hash that produced it. Third, the reviewer sign-off: a dated approval from someone who checked that the pre-registered β and error-cost ratio were used. Without all three, the flag is a hypothesis, not a control exception.

This section's only new claim is the three-option comparison and the selection of Option C; the five-step ordering, the Detection Engineering Framework lifecycle, the four-source convergence table, the error-budget arithmetic, and the edge-case break table are covered elsewhere in this guide.

![Three Threshold Methods Compared, One Winner — Internal Controls Decision Guide](https://static.mm-ais.com/article-images-pixabay/internal-controls-decision-guide-five-st-ffe7203b.jpg)

## The Numbers: Error Budget, Sample Size, Cost

The first arithmetic act in any control design is to fix the error budget before you see a single result. When zero exceptions are observed in a sample of n items, the 95% upper bound on the true exception rate is given by the rule of three: 3 divided by n. For n = 60, that bound is 3/60 = 5.0%. The exact binomial bound is 1 − 0.05^(1/60) = 4.87%. These are method arithmetic, not measured rates; recompute them for your own β whenever the false-negative budget changes. The calculation tells you the largest plausible exception rate consistent with your observed clean run, and it therefore sets the minimum sample size you need to claim that the control is working at your pre-registered detection power.

The second arithmetic act is to convert that statistical bound into a decision threshold by pairing it with an error-cost ratio computed before any testing begins. Let C_FP be the cost of a false alarm, measured as investigator hours multiplied by the loaded hourly rate, and let C_FN be the expected loss from a single missed exception. The ratio C_FP / C_FN tells you how much investigative effort you can afford per unit of prevented loss. If the ratio is near or below 1, you should investigate broadly whenever the pre-registered rule fires; if it is far above 1, you need stronger evidence before acting. The ratio is a business input, not a statistical output, so it must be agreed upon while the threshold is still on the drawing board.

Once the budget and the cost ratio are fixed, the threshold rule itself becomes a simple comparison: the observed exception rate must stay below the pre-registered bound derived from your β = 0.05 false-negative budget. For the n = 60 example, that bound is 4.87%, so any observed rate at or above 4.87% triggers an investigation, while any rate below it allows the control to stand. The rule is mechanical, but its force comes from the fact that both the bound and the cost ratio were written down before the data existed, which prevents hindsight from quietly widening the acceptable zone.

| Input | How it is set | Why it cannot be changed later |
| --- | --- | --- |
| False-negative budget β | Chosen from business impact; β = 0.05 is the default | Defines the 95% detection power and therefore the upper bound |
| Sample size n | Solved from 3/n ≤ allowed exception rate | Determines how many items must be examined |
| C_FP / C_FN ratio | Investigator hours × loaded rate ÷ expected loss per miss | Decides whether a firing rule is investigated broadly or narrowly |
| Threshold rule | Observed rate ≥ 1 − β^(1/n) triggers action | Provides a single, auditable yes/no criterion |

The arithmetic above is deliberately simple so that it can be reproduced in a spreadsheet by anyone who knows the sample size and the agreed false-negative budget. It does not require Bayesian priors, likelihood ratios, or any other machinery; it only requires that the three pre-registered numbers—β, n, and the cost ratio—be written down before the first test item is examined. When those numbers are missing, the flag is not actionable, because there is no way to tell whether the threshold was chosen to fit the data or to test it.

![Internal Controls Decision Guide, photo 2](https://static.mm-ais.com/article-images-pixabay/internal-controls-decision-guide-five-st-eef76b8f.jpg)

## Where the 95% Rule Breaks: Edge-Case Table

Even a well‑pre‑registered control exception can fail the 95 % detection rule when the population is too small or the observed exception rate is unusually high. In those edge cases the standard “3‑out‑of‑60” shortcut no longer guarantees a 5 % false‑negative budget, and the evidence package must be treated differently before any action is taken.

When the population is smaller than 60, the rule of three cannot certify a 5 % upper bound. The binomial confidence interval widens beyond the pre‑registered tolerance, so the control team must switch to 100 % testing of the entire population and report a census rather than a sample‑based bound. A census eliminates sampling error and satisfies the 95 % detection requirement without relying on the 3/n approximation.

If the observed exception rate exceeds roughly 5 %, the 3/n shortcut understates the true upper bound. For example, three exceptions in a population of 60 yield an exact binomial upper bound of 12.9 % (0.5 × χ²(0.95, df = 8) / 60 = 7.754 / 60). In this scenario the flag should be acted upon rather than cleared, because the pre‑registered error‑cost ratio still favors intervention when exceptions are sparse but the population is large enough to support the exact calculation.

Table 1 lists the two primary edge cases, the failure mode, and the required corrective action.

| Edge case | Failure mode | Corrective action |
| --- | --- | --- |
| Population  ~5 % | 3/n shortcut understates upper bound | Switch to exact binomial or Poisson upper bound; act |

These edge cases are the only situations in which the pre‑registered 95 % rule can be set aside without re‑opening the error‑cost negotiation. In all other circumstances the standard five‑step ordering and the evidence‑package requirements remain binding.

![Where the 95% Rule Breaks: Edge-Case Table — Internal Controls Decision Guide](https://static.mm-ais.com/article-images-pixabay/internal-controls-decision-guide-five-st-736fb4e7.jpg)

## 60-Item Test, 0 Exceptions

Consider a three-way-match control on non-PO invoices. The quarterly population is 9,400 lines. Each missed exception carries an expected exposure C_FN of $18,000; each false alarm costs C_FP of $450 in investigator time. The error-cost ratio is therefore 40:1. With a pre-registered false-negative budget of β = 0.05, the required sample size is n = 3 / 0.05 = 60 items. This is Checkpoint 1: power. The rule fires only when the sample has been drawn and the threshold has been fixed before testing begins.

Checkpoint 2 is the result. Zero exceptions are found in the 60-item sample. The exact upper bound for the underlying exception rate is 4.87%, which rounds to 5.0% under the rule of three. Because 4.87% ≤ 5.0%, the control passes the pre-registered budget. No action is triggered. The exposure fork is explicit: had the observed rate been above 5%, the same arithmetic would have forced escalation regardless of the absolute dollar figure.

Checkpoint 3 is the evidence package. Even a correct flag is only actionable when three artifacts are present: (1) the input-data snapshot showing the 9,400-line population and the 60-item draw; (2) the threshold/logic version, in this case the 3/0.05 formula and the 5.0% ceiling; and (3) reviewer sign-off confirming that the pre-registration card was completed before testing. Missing any one of these three invalidates the flag, because the control exception is only actionable when its threshold was pre-registered with an error-cost ratio and a 5% false-negative budget before testing.

| Pre-Registration Card Field | Value in This Test |
| --- | --- |
| Population | 9,400 non-PO invoice lines, Q |
| C_FN (exposure per miss) | $18,000 |
| C_FP (cost per false alarm) | $450 |
| Error-cost ratio | 40:1 |
| β (false-negative budget) | 0.05 |
| Sample size n | 60 |
| Observed exceptions | 0 |
| Exact upper bound | 4.87% (5.0% by rule of three) |
| Budget status | Pass |

The fork from 5% to 1% is not hypothetical: if the same population and costs were re-tested under a tightened β = 0.01 budget, the required sample would rise to n = 3 / 0.01 = 300 items. The pre-registration card would be re-submitted with the new β, the new n, and a fresh reviewer sign-off. Until that card is filed, the 5% rule remains the only actionable threshold, and any flag raised under the 1% figure is not actionable.

## Five If/Then Rules for Acting on a Flag

**If the rule was not pre-registered before testing, then do not act on its flags.** First confirm that the threshold, the error-cost ratio, and the false-negative budget of no more than 5% were documented before evaluation. A threshold chosen after seeing results may fit the observed exceptions, but it does not provide an auditable basis for claiming 95% detection power. Record such a flag as awaiting review rather than treating it as an approved control exception.

**If the pre-registered rule fires and the evidence card is complete, then act within the control’s stated days-to-action SLA and log the disposition date.** The card must contain the input-data snapshot, the threshold or logic version, and reviewer sign-off. Record both the trigger and the resulting disposition so that the case trail shows what was detected, which rule version fired, who authorized the response, and when the organization closed the matter.

**If the rule fires but any evidence-card field is blank, then do not act and do not clear the flag.** Re-run the control with metadata capture and obtain the missing field before disposition. A correct flag without its basis is unactionable evidence: the result may be real, but reviewers cannot reproduce or defend it. This follows the auditable-metadata practice used by Microsoft’s Framework Detector, which captures input data and the checks performed alongside results to create a clear paper trail.

**If a test returns zero exceptions in fewer than 60 items, then do not certify 95% detection.** Zero findings provide no observed opportunity to exercise the detection rule, and the sample is too small to support the required claim. Extend the test to at least 60 items or conduct a census of the population. If the population contains fewer than 60 eligible items, document the complete population tested and do not represent the result as a 95%-detection certification.

**If the owner can no longer show that the rule remains aligned with the control objective and its pre-registered error-cost ratio, then retire the rule rather than leaving it in production.** Retirement should be explicit, with the reason, effective date, and responsible reviewer recorded. Do not silently retune the threshold and continue issuing exceptions under the old rule. A replacement should be pre-registered with its own threshold, logic version, and false-negative budget before operational use, giving every future flag a fresh, reviewable basis for action.

## What to do next

| Step | Action | Why it matters |
| --- | --- | --- |
| 1 | Pre-register each vegetation-control exception rule with its error-cost ratio and false-negative budget before test execution. | This makes the threshold decision rule auditable and prevents post-test adjustment. |
| 2 | Confirm that the pre-registered rule meets the canonical β = 0.05 detection criterion before acting on a flagged exception. | An exception is actionable only when the registered rule satisfies the required detection power. |
| 3 | Build the flag’s evidence package with the input-data snapshot, threshold/logic version, and reviewer sign-off. | All three components are mandatory; missing even one invalidates the flag regardless of detection power. |
| 4 | Evaluate threshold behavior with the PCA-based similarity window using the 2001–2024 NDVI and precipitation records, including spatial structure and associated changes in vegetation controls. | This checks whether the registered threshold remains robust across the dataset rather than reacting to an isolated anomaly. |
| 5 | Use cost-sensitive threshold tuning to verify the registered rule and logic version before approval. | Cost-sensitive tuning strengthens threshold robustness without replacing the pre-registered decision rule. |
| 6 | If any required evidence item is missing, record a documented no-action and do not substitute an ad-hoc threshold. | A documented no-action preserves the canonical decision rule and prevents unsupported intervention. |

## Frequently Asked Questions

**What must be documented before a control exception can become actionable?**

The threshold must be pre-registered with an explicit error-cost ratio and a 5% false-negative budget before testing.

**Which three items must accompany every actionable flag?**

The evidence package must contain an input-data snapshot, a logic version, and reviewer sign-off.

**What happens when one required evidence component is missing?**

The flag is invalid regardless of its detection power.

**Which records are used to examine spatial structure and vegetation-control changes?**

The framework uses a PCA-based similarity-window built from 2001–2024 NDVI and precipitation records.

**How should cost-sensitive threshold tuning be evaluated for an accelerometer-based fall-detection control?**

Check the cited study’s reported results before relying on the computationally efficient framework for that control.

**What benefit was reported from adjusting thresholds for cost sensitivity?**

The results show enhanced detection reliability when thresholds are adjusted for cost sensitivity.

## Quick answers

| What is the required detection power for a control exception to be considered actionable? | The framework requires β = 0.05, which corresponds to 95% detection power. |
| --- | --- |
| What three components must be included in an evidence package for a flag to be valid? | The evidence package must contain an input-data snapshot, logic version, and reviewer sign-off. |
| What dataset is used to build the PCA-based similarity-window for vegetation controls? | The dataset is built from 2001–2024 NDVI and precipitation records. |
| What must be documented before any test execution according to the rule? | An explicit error-cost ratio must be documented prior to any test execution. |
| What is the consequence if any one of the three required evidence components is missing? | Missing any one of these three components invalidates the flag, regardless of detection power. |

Also worth reading: **What Are Internal Controls? 3-Layer Framework for Anomaly Detection in 2026 Audits**: [What Are Internal Controls? 3-Layer](https://financialauditexpert.com/blog/what-are-internal-controls-3-layer-framework-for-anomaly-detection-in-2026-audits.php) · **Mastering Internal Controls The COSO Framework for Bulletproof Fraud Prevention**: [Mastering Internal Controls The COSO](https://financialauditexpert.com/blog/mastering-internal-controls-the-coso-framework-for-bulletproof-fraud-prevention.php) · **How to Strengthen Internal Controls and Improve the Success of Your Financial Audit**: [How to Strengthen Internal Controls](https://financialauditexpert.com/blog/how-to-strengthen-internal-controls-and-improve-the-success-of-your-financial-audit.php)

### Related reading

- [AI Auditing Framework: A Practical Guide for Internal Auditors](https://financialauditexpert.com/blog/ai_auditing_framework_a_practical_guide_for_internal_auditors.php)
- [7 Key AI-Powered Anomaly Detection Breakthroughs Reshaping Internal Audit Metrics in 2025](https://financialauditexpert.com/blog/7_key_ai_powered_anomaly_detection_breakthroughs_reshaping_i.php)
- [COSO's 2024 Update Integrating Cybersecurity into the Internal Control Framework](https://financialauditexpert.com/blog/coso_s_2024_update_integrating_cybersecurity_into_the_intern.php)
- [Internal Controls Review: Testing 5 Bonsai Workflows for Duty Gaps](https://financialauditexpert.com/blog/internal-controls-review-testing-5-bonsai-workflows-for-duty-gaps.php)
- [Internal Controls Explained: Frameworks, Types, and Practical Examples](https://financialauditexpert.com/blog/internal_controls_explained_frameworks_types_and_practical_examples.php)
- [Demystifying Internal Controls in Accounting Your Expert Guide](https://financialauditexpert.com/blog/demystifying-internal-controls-in-accounting-your-expert-guide.php)

### Latest

- [Benford's Law in Audit 2026: 12,000 Accounts Payable Transactions, Flag or...](https://financialauditexpert.com/blog/benfords-law-in-audit-2026-12000-accounts-payable-transactions-flag-or-clear.php)
- [Internal Controls Review: Testing 5 Bonsai Workflows for Duty Gaps](https://financialauditexpert.com/blog/internal-controls-review-testing-5-bonsai-workflows-for-duty-gaps.php)
- [What Are Internal Controls? 3-Layer Framework for Anomaly Detection in 2026...](https://financialauditexpert.com/blog/what-are-internal-controls-3-layer-framework-for-anomaly-detection-in-2026-audits.php)

Canonical: https://financialauditexpert.com/blog/internal-controls-decision-guide-five-step-framework-to-act-at-95-detection.php
Markdown: https://financialauditexpert.com/blog/internal-controls-decision-guide-five-step-framework-to-act-at-95-detection.php/index.md