Verified Security Findings: How to Tell Proof From Inference

A security finding arrives in your tracker. It names a class of bug, points at a file or an endpoint, and asserts a severity. What it usually does not say is how anyone knows the bug is real.

That gap is where most of the friction between security tooling and engineering teams lives. An engineer who has closed three unreproducible tickets in a row starts reading the fourth with justified suspicion, and the one genuine finding in the batch pays the price. The fix is not a louder severity label. It is being explicit about the kind of evidence behind each finding, and refusing to let two very different claims wear the same word.

The word "verified" is doing at least three jobs

Ask different vendors what a verified finding means and you may get different answers, because the term is not used consistently across security tooling.

For some, verified means a human looked at it. An analyst read the output, agreed the pattern was plausible, and let it through. That is a filter on obvious noise, and it is worth something, but it is a judgement about code rather than an observation of behaviour.

For others, verified means the tool reached the code path. Taint analysis traced input to a sink, or a scanner got a response that differed from the baseline. Something real happened, but whether it amounts to an exploitable condition in the running application is still an open question.

For others again, verified means an attack was attempted against the live system and the outcome was recorded. The system either gave up the data or it did not.

These are not degrees of confidence in the same claim. They are different claims. Collapsing them into one word is how a buyer ends up comparing two products on a number that does not mean the same thing on either side.

The evidence ladder, in plain terms

It helps to separate findings by how the conclusion was reached rather than by how alarming the title is.

Inferred from code. Someone or something read the source and concluded that a dangerous pattern exists. The evidence is a file, a line, and an argument about why the pattern is dangerous. This is genuinely useful: it can reach code that never executes during a test run, and it can catch a whole class of problem at once. What it cannot do on its own is tell you whether the path is reachable by a real request, whether a control elsewhere already blocks it, or whether the data behind it matters.

Observed as an anomaly. Something was sent to the running application and the response was not what a healthy system should return. An error leaked a stack trace, a timing difference suggested a branch, a response body contained a field it should not. The evidence is a request and a response. This is stronger than inference because the system actually did something. It is still short of proof, because an anomaly is not the same as an attacker reaching data they should not have.

Attempted and proved. An attack was run end to end against the running application and the outcome was recorded. The evidence is the sequence of requests, the identity each one ran as, and what came back. When this exists, there is direct evidence that the finding is real under the tested conditions, and the conversation can move to how bad it is, where else it may apply, and what to change.

Application security work can produce a mix of all three, and that is fine. The problem is when the report does not say which is which.

What evidence each level can produce

The last row is the one most often left out of vendor comparisons, and it is the one where the label carries the least information on its own. A human-raised finding can be the best-evidenced item in the whole report or the weakest, and the only way to tell is to read what they actually did.

How Borg records this on each finding

Borg's platform keeps this distinction on the finding itself rather than in prose. Findings from Mjolnir, Borg's agentic AI pentest engine, carry a confirmation type alongside severity:

• Exploited. Mjolnir ran the attack end to end and proved it works.
• Partial. Mjolnir found evidence the issue is real but stopped short of a working exploit.
• Static. Mjolnir confirmed the issue by reading source code without running an attack.

Severity answers how bad this would be. Confirmation type answers how we came to believe it. They are deliberately separate fields, because a high-severity finding established statically and a high-severity finding proved end to end call for different conversations with the engineer who has to act on them.

Findings raised by an analyst rather than by the engine carry no confirmation type at all. That is not an oversight. The field describes how an automated attempt concluded, and applying it to human work would imply a uniformity that is not there.

This is also a useful answer to a common concern about automated offensive security: that it can produce output without enough evidence to act on. The answer is not a confidence percentage. It is being able to say, per finding, what the system actually established. For an exploited finding, that means it attempted the attack, observed what happened, and recorded the steps needed to reproduce it.

Where this still does not settle the question

Three limits are worth stating plainly, because a page that claims proof solves everything is making the same mistake in the opposite direction.

A proof is a proof in one environment. An attack that ran against a staging deployment tells you about that deployment. Configuration, data, and network controls can differ in production in ways that make the same path harder or easier. The finding is real; its reach still needs judgement.

Static confirmation is a reading, and readings can be wrong. Confirming an issue by reading code can establish that a potentially dangerous condition exists in the source, and it reaches places a live attempt cannot, particularly code behind flags or rarely-exercised branches. It is still an assessment of what the code would do rather than a record of what it did at runtime.

Proof says nothing about worth. An exploited finding that exposes a public marketing record and an exploited finding that exposes another tenant's ledger have identical confirmation types. Deciding which to fix first is a separate exercise, and if the finding has no CVE attached the usual scoring inputs do not help much either. We have written about that scoring problem separately in how to prioritise security findings that have no CVE.

None of this makes confirmation type less useful. It makes it one axis of several, which is what it was designed to be.

The same bug at all three levels

A concrete case makes the distinction less abstract. Take a project record in a multi-tenant application, reachable at an endpoint that takes an organisation identifier and a project identifier.

As an inference, it looks like this: the handler reads the project identifier from the path and loads the record, and the authorisation check above it validates the organisation identifier only. A reviewer reading that code can see that the two values are never checked against each other. That is a real observation, and it is also a guess about runtime, because a middleware two files away might already be constraining the query.

As an anomaly, it looks like this: a request carrying a project identifier belonging to another organisation returns 200 rather than 403. Something is wrong. Whether the body contains that organisation's data, an empty record, or a redacted stub is not yet established.

As a proof, it looks like this: authenticated as an ordinary user of organisation A, a request for a project belonging to organisation B returned that project's data, and the response is recorded next to the request that produced it. That provides direct evidence of unauthorised access under the tested conditions, leaving much less uncertainty about whether the described behaviour can actually occur.

Same bug, same severity, three very different tickets. The third one provides substantially stronger evidence because the request was actually run and the resulting access was observed. The difference between the first and the third is not simply how hard anyone looked at the code; it is that the runtime behaviour was tested directly.

Why finding counts do not compare across vendors

This is also the reason a headline number is a poor way to choose between two security products. A tool that reports every inferred pattern and a tool that reports only what it proved will produce wildly different counts on the same codebase, and the larger number is not evidence of better coverage. It may simply be a different threshold for what gets written down.

The comparison that carries information is narrower: on a codebase resembling yours, how many findings arrived with a recorded attempt and its outcome, and how many arrived as an argument about code. Ask for that split rather than the total. If a vendor tracks the distinction internally, it should be possible to ask how its findings break down by evidence type and what those categories mean.

The questions worth asking about any finding

Whether the finding came from a vendor, an internal tool, or a consultant, the same short list separates evidence from assertion.

What was attempted, and what came back? If the answer describes a code path rather than an interaction with the running system, you have an inference. That may be the right answer for that class of bug, but name it.

Which identity was the attack running as? For anything touching authorisation, tenancy, or access control, this is essential context. A request that returned another tenant's data while authenticated as a legitimate user of a different tenant may demonstrate broken access control, provided that user is not authorised to access that data. The same request run as an administrator may be expected behaviour if that administrator is intentionally authorised to access it.

Can I reproduce it from what you gave me? Reproduction steps that an engineer can follow without asking a follow-up question are the practical test of whether the work was done. This is worth more than any confidence score, because it transfers.

If it was not proved, what stopped it? A tool that reached an anomaly but could not turn it into access should say so. That is a more useful answer than a severity number, and it tells your team where to spend the next hour.

What would change your assessment? A vendor who can name the condition under which their own finding would be wrong is describing a method. One who cannot is describing a conclusion.

Using this in your own triage

You do not need a new process for this. You need the evidence level visible at the moment someone decides what to do, which usually means carrying it into the ticket rather than leaving it in the report.

A finding that was attempted and proved, in a path that handles customer data, has strong evidence behind it. It is still a prioritisation question: the team needs to consider the impact, affected assets, exposure, mitigations, and business context when deciding what to fix first.

A finding observed as an anomaly is a question about how far it goes. Often the cheapest next step is a short, bounded attempt to push it one stage further, because turning it into either a proof or a dead end is worth more than a week of debate about the severity label.

A finding inferred from code sits with the team who owns that code, because the team may have context about reachability and intended behaviour that an outside tool does not. That is not a dismissal. Plenty of genuine bugs only ever appear as inferences, particularly in code paths that no test run exercises, which is part of why static analysis keeps a place alongside live testing. We have covered where that boundary sits in what static analysis can reach.

The practical gain from all of this is not a cleaner dashboard. It is that the engineer reading the fourth ticket can tell, without asking, which of the four is the one that was proved. That is the difference between a security function that gets acted on and one that gets filtered.

Where the evidence question is sharpest

The classes of bug where this matters most are the ones that depend on context rather than pattern: broken access control between tenants, a multi-step workflow that can be entered out of order, an API that trusts a value it should re-check. None of them look dangerous as a line of code, and all of them are unambiguous once an attack has run and the data came back. That is precisely the ground where an inferred finding and a proved one look identical in a ticket and are worlds apart in reality. We have walked through testing that class directly in how to test business-logic vulnerabilities in a SaaS application.

Ask for the evidence, not the adjective. A report that tells you how each conclusion was reached is more useful than one that tells you how worried to be, and it is considerably harder to write, which is rather the point.