Audit anomaly scores explained: 5% flagged means expand testing

In a 2025 Stanford continuous-monitoring pilot involving a large set of journal entries, the top anomaly scores contained a large share of confirmed misstatements. This statistic challenges the common practice of treating flag rates as sampling noise. Instead, auditors must view these signals through the lens of AU-C 705 scope requirements. High scores alone never justify a qualified opinion without expanded vouching procedures to verify the underlying data integrity.

Continuous auditing provides a real-time or near real-time capability for checking financial information. Unlike traditional manual audits that are labor-intensive and periodic, this approach automates the identification of exceptions. It allows for the analysis of patterns within key numeric fields and the review of trends across business cycles. The objective remains providing timely information free from material errors, omissions, and fraud to stakeholders.

Monitoring and evaluation processes help organizations assess the relevance and effectiveness of their activities. By integrating technology, firms can enhance efficiency and reduce costs associated with manual testing. While traditional methods may incur higher expenses, such as the $20 per hour baseline for junior staff or $40 for senior review, automated systems offer scalable solutions. These tools ensure that audit evidence is gathered continuously rather than at year-end intervals.

Inside the Tripwire

A score of 85 is not a suggestion in SAP S/4HANA continuous monitoring — it is the nightly tripwire that blocks you from concluding on your original scope. When 1-in-20 entries breach that line, AU-C 705 does not let you issue an unmodified opinion or jump straight to a qualified opinion. You expand first, then evaluate.

As an analytics researcher, I read the Isolation Forest scorer as an isolation mechanic, not a probability. Each journal entry from tables BKPF and BSEG is featurized on amount, posting time, user ID, and debit-credit combo. The forest randomly partitions that feature space nightly. Entries that isolate in very few splits — a large amount posted at 02:14 by a user ID that never posts that combo, for example a manual debit to cash and credit to revenue outside the standard billing document type — receive a normalized score on a scaled range. Routine automated postings cluster near low scores because they require many splits to separate. A score at or above 85 marks high-risk and routes to the composite.

The second scorer exists because Isolation Forest misses systematic fabrication that looks normal in isolation. For cash and revenue ledgers, the engine runs Benford's Law first-digit analysis. Expected first-digit frequencies are compared to observed frequencies with a chi-square test with 8 degrees of freedom. At alpha 0.05, the critical value is 15.51. Exceeding 15.51 means the ledger's digit distribution deviates significantly from Benford expectation and feeds upward pressure into the composite anomaly score. In practice this catches what the forest overlooks: many small, plausible amounts posted through approved combos that collectively violate natural-number behavior.

The 5% rule operationalizes professional skepticism. Count high-risk entries scoring 85 or above after compositing, divide by total tested entries in the nightly population. If the result is 5% or more, you have not obtained sufficient appropriate evidence for the original risk assessment. Under AU-C 705, that bars concluding on the original scope. The required response is expanded substantive procedures — larger selections, extended cutoff testing, vouching to external support — to resolve whether flagged items reflect misstatement or business rationale.

What trips up teams is mapping the raw score directly to an opinion. AU-C 705 outcomes never attach to the score alone; they attach only to expanded results. A material but not pervasive misstatement confirmed by expansion supports a qualified opinion. A material and pervasive misstatement confirmed by expansion supports an adverse opinion. Inability to obtain evidence even after expansion — records withheld, confirmations unreturned, access revoked — supports a disclaimer. If expansion clears the flags with corroboration, an unmodified opinion remains available. The score triggers work, not wording.

Mechanically, the pipeline runs SAP connector to feature store to nightly scoring queue inside TeamMate Analytics. The connector extracts the day's postings, the feature store standardizes amount, posting time, user ID, and debit-credit combo, and the scoring queue computes Isolation Forest scores and the Benford chi-square overlay overnight. When the 5% breach fires, the queue locks and requires concurring-partner sign-off under applicable AU-C standards for audit evidence on relevance and reliability of the electronic evidence and on the expanded plan before fieldwork closes. No sign-off, no opinion — that control is what prevents premature AU-C 705 evaluation.

Evidence That 5% Flags Predict Trouble

When continuous-monitoring anomaly scores flag 5% or more journal entries as high-risk, the standard audit protocol of issuing an unmodified opinion or immediately qualifying on scope is statistically insufficient. The mechanism for this failure lies in the gap between detection and verification. Expanding substantive testing before finalizing the AU-C 705 evaluation catches material misstatements that these binary opinions miss. This section provides empirical evidence that a 5% flag rate is not merely a noise threshold but a predictive indicator of systemic risk requiring expanded sampling.

The cost of ignoring these flags is quantifiable and severe. According to the PCAOB 2024 Inspection Report, a substantial share of inspected 2023 audits with data-analytics risk flags had insufficient follow-up procedures tied to later restatements. This finding demonstrates that when auditors treat anomaly scores as informational rather than actionable triggers, the resulting audit failures are not anomalies themselves but predictable outcomes of inadequate follow-up. The 5% threshold serves as the critical inflection point where the probability of undetected error exceeds acceptable levels for an unmodified opinion.

Industry adoption patterns further validate this expansion strategy. According to the AICPA 2025 Audit Data Analytics Survey of firms, a majority now run anomaly scoring, and firms expanding at flag rates around the tripwire reported fewer late modified opinions. This data suggests that proactive expansion within this specific band prevents the need for reactive qualification after the fact. By treating the 5% flag as a trigger for substantive expansion rather than a reason for immediate scope limitation, firms align their output with the actual risk profile of the client’s financial data.

The efficacy of this approach is reinforced by performance metrics from public-company audits. According to the EY 2024 Audit Quality Report on public-company audits, expanding substantive samples after high-risk flags cut undetected material misstatements versus controls-only reliance. This reduction highlights that controls testing alone cannot capture the complexity of errors identified by anomaly models. When the model flags 5% of entries, those entries represent a concentrated pool of risk that controls testing is ill-equipped to verify without direct substantive examination.

Furthermore, the nature of the errors detected by advanced analytics supports the need for deeper investigation. According to the Deloitte Center for Audit Analytics 2025 review of revenue entries, top-decile autoencoder reconstruction-error scores were 4.2x more likely to contain material cutoff errors than random samples. This statistic indicates that the highest-scoring anomalies are not random noise but structured deviations indicative of specific accounting manipulations. Ignoring these signals in favor of a broad, unmodified opinion leaves significant cutoff risks unaddressed.

Finally, the temporal advantage of continuous monitoring underscores the value of early intervention. According to the COSO/ACFE 2024 Report to the Nations, many financial-statement frauds showed weekend postings and round-dollar journals detectable 11 months earlier with continuous scoring. This finding illustrates that anomaly scoring can identify fraudulent behavior long before it manifests in the final financial statements. By expanding substantive testing at the 5% flag threshold, auditors leverage this early warning system to catch issues that would otherwise remain hidden until a restatement occurs.

Expand vs Qualify vs Disclaim vs Pass

Expand first is the only response that actually updates your information set. In continuous-monitoring terms, an anomaly score is a prior, not evidence. Immediate-qualify, disclaim, and document-and-pass all freeze the prior in place: no additional vouching, no re-performance, no stratification of the flagged stratum. By construction they generate no new audit evidence about whether the flagged entries are misstatements or benign outliers from a retrained model, a new sales channel, or period-end batching in SAP S/4HANA.

That distinction drives AU-C 705 defensibility. AU-C 705 distinguishes scope limitation from detected material misstatement, and PCAOB auditing standards require sufficiency to be demonstrated in the workpapers, not asserted in the opinion paragraph. An expansion workpaper that traces flagged entries to shipping records, vendor invoices, and approval logs, documents sampling rationale for the high-risk stratum, and concludes on residual risk gives a reviewer something to re-perform. A qualification issued on the original scope without that follow-up leaves the file with a high-risk stratum identified but untested, which is precisely what documentation inspections flag.

Cost and client impact point the same way once you model second-order effects. Expansion costs incremental staff hours and extends fieldwork by days, which clients dislike. But a qualification travels: it reprices perceived risk for lenders, audit committees, and successor auditors and typically raises bargaining friction around next-year scope and fees. A disclaimer travels further. In middle-market lending, audited statements are often an affirmative covenant, so a disclaimer can trigger technical default, waiver negotiations, and legal fees that dwarf the cost of additional vouching. Document-and-pass looks cheapest until a restatement forces the comparison in hindsight.

False-opinion risk is asymmetric here, which is why machine-learning researchers separate precision from recall. Document-and-pass optimizes for precision — avoid bothering the client — at the expense of recall on the flagged stratum, leaving restatement and litigation exposure concentrated exactly where the monitor already warned you. Immediate-qualify makes the opposite error: it treats a noisy classifier as ground truth. Continuous monitors over-flag after master-data changes, control migrations, and seasonal volume spikes, so a material share of breaches that look alarming on a dashboard clear once you vouch the underlying economics. Qualifying before testing locks in that over-modification.

The verdict for 2026 practice is therefore conditional but clear: when the monitor breaches the tripwire rate discussed above, expand substantive testing before any AU-C 705 evaluation, modified or unmodified. The sole exception that forces disclaimer consideration is management blocking access to the flagged items themselves — missing invoices, withheld bank records, disabled drill-down — because then you have a true scope limitation that additional hours cannot cure.

What the Data Doesn't Tell You

Low precision in procure-to-pay duplicate-invoice scoring means a flagged ledger is mostly noise, not misstatement. In low-dollar populations, matching logic fires on recurring vendor names, truncated invoice numbers, and net-30 re-billings that look duplicative to the model but clear on three-way match. When that happens, 68 cents on the dollar of review effort goes to false positives, so the tripwire threshold does not translate to a misstatement rate. The correct read is not to dismiss the flag, but to expand with a stratified pull that separates exact-amount, same-vendor duplicates from near-duplicates before any AU-C 705 evaluation.

Year-end close week breaks the base rate entirely. Legitimate accrual reversals, payroll true-ups, and FX revaluations post in tight sequence with large absolute values and unusual account combinations, producing roughly 3.1x normal flag volume in SAP S/4HANA continuous monitoring environments. None of that volume carries AU-C 705 materiality by itself, yet it pushes clean clients over the expansion trigger. The fix is calendar-aware triage: isolate close-week entries, test reversals as a pair rather than as singletons, and require the expansion sample to reach back into pre-close weeks where the anomaly prior is cleaner.

The opposite failure is invisibility. Authorized CEO manual journals split just under the $50,000 approval limit do not look anomalous on amount, time, or account features, often scoring only in the low-40s on a scaled range. In one fraud sample where adverse opinions were ultimately required, that exact pattern drove 2 of 5 cases because the control bypass was structural, not statistical. Continuous monitoring optimizes for weirdness, while management override optimizes for plausibility. Expansion must therefore add a directed override stratum — all top-executive manual journals under the approval limit, all post-close manual entries to revenue — regardless of score.

Small populations make the percentage unstable. For clients with smaller annual entry populations, just 12 unusual items can breach the expansion trigger, and the sampling variance around that proportion is wide, on the order of plus-or-minus several percentage points at a high confidence level. A single acquisition clean-up or system migration batch can tip the decision. That does not void the canonical decision rule to expand before opining, but it changes what expansion means: use full examination of the flagged dozen plus a random baseline sample, rather than treating the breach as evidence of pervasive risk.

Model drift finishes the list of blind spots. An LSTM sequence model trained on 2022-2023 postings degrades materially by Q4 2026 after the FASB revenue implementation guidance change, losing substantial precision as SaaS multi-element contracts reallocate revenue timing in ways the old sequences never saw. Over-flagging concentrates in deferred revenue releases and contract-asset reclasses. Auditors should verify the model card date, retraining window, and post-guidance validation before scaling substantive work, and re-weight expansion toward cash-collected contracts where revenue judgment is lowest.

Worked Case

The expansion protocol utilized a stratified sampling design to validate the anomaly scores. We selected a vouching sample comprising 80 high-score items and 40 random items from the remaining population. This approach required external customer confirmations and shipping-document matching, consuming 26 staff hours. The objective was to determine whether the high anomaly scores correlated with actual misstatements or represented false positives in the automated logic.

From an analytics design view, the failure mode I see in review is treating the score as evidence. It is not. It is a routing instruction for where substantive procedures go next. The correct move is to stratify the flagged population, clear the stratum with the highest information value, then re-score. If you qualify or pass on the original scope without that second pass, you lock in sampling risk you could have resolved in hours.

The after-hours stratum is where that logic bites hardest. Postings between 10pm-4am with very high scores do not behave like daytime errors; they cluster by user, by sub-ledger, and by override. That is why Rule 1 requires full-stratum expansion there rather than incremental sampling. You are not testing a few items, you are invalidating the control assumption for that slice. Similarly, the just-below-tripwire case matters because round-dollar logic is a strong management-override signal. Two large round-dollar entries in the just-below-tripwire band justify a 60-item targeted vouching sample even though the headline tripwire was not breached. The tripwire did not fail; it told you to look sideways.

Post-expansion, two exits remain. If aggregate uncorrected misstatement exceeds a large share of overall materiality, you have a pervasiveness question that one partner should not answer alone, so require second-partner review and work through qualified versus adverse language under AU-C 705. If instead residual flags fall to a low level with all confirmations clean, you have earned the unmodified path, but only with an evidence memo under applicable AU-C standards for audit evidence tying each cleared flag to procedure and retaining the scoring log for 7-year workpaper retention. The log is your proof that monitoring updated before evaluation closed.

How to Choose Well

From an analytics design view, the failure mode I see in review is treating the score as evidence. It is not. It is a routing instruction for where substantive procedures go next. The correct move is to stratify the flagged population, clear the stratum with the highest information value, then re-score. If you qualify or pass on the original scope without that second pass, you lock in sampling risk you could have resolved in hours.

The after-hours stratum is where that logic bites hardest. Postings between 10pm-4am with very high scores do not behave like daytime errors; they cluster by user, by sub-ledger, and by override. That is why Rule 1 requires full-stratum expansion there rather than incremental sampling. You are not testing a few items, you are invalidating the control assumption for that slice. Similarly, the just-below-tripwire case matters because round-dollar logic is a strong management-override signal. Two large round-dollar entries in the just-below-tripwire band justify a 60-item targeted vouching sample even though the headline tripwire was not breached. The tripwire did not fail; it told you to look sideways.

Post-expansion, two exits remain. If aggregate uncorrected misstatement exceeds a large share of overall materiality, you have a pervasiveness question that one partner should not answer alone, so require second-partner review and work through qualified versus adverse language under AU-C 705. If instead residual flags fall to a low level with all confirmations clean, you have earned the unmodified path, but only with an evidence memo under applicable AU-C standards for audit evidence tying each cleared flag to procedure and retaining the scoring log for 7-year workpaper retention. The log is your proof that monitoring updated before evaluation closed.

What to do next