Offensive Security Testing in Production: What to Agree First

Every team evaluating continuous security testing arrives at the same question, usually late in the conversation and usually phrased carefully: are you going to run this against production?

It is the right thing to worry about and the wrong shape of question. "Is it safe" invites a yes, and a yes is not something any testing vendor can honestly give you about your own application. The useful version is narrower. What can this thing actually do to my system, which of those things do I want to permit, and what happens if it goes wrong at two in the morning.

That question has real answers. Many of them are about configuration, boundaries and controls rather than taking a vendor's assurances on trust.

What actually creates the risk

Offensive testing is not a scanner reading headers. The requests are real, and several properties of them matter operationally.

The traffic looks like an attack, because it is one. Borg's own setup documentation puts this plainly in a warning on the scope step: only add URLs you own and are authorised to test, because Mjolnir will send real HTTP requests, including probes that may look like attack traffic, to everything in scope. Any vendor whose documentation does not say something equivalent is worth a question.

Some requests change state. Testing an authorisation boundary often means attempting the action, not just looking at the endpoint. A test that proves an account in one organisation can reach another organisation's record is more convincing when it actually retrieves the record. Extend that to workflows, payment paths or admin functions and some tests may involve requests that create or modify state.

A realistic test often needs to be authenticated. The classes that matter most in a multi-tenant application, broken access control, tenant isolation and privilege escalation, often require testing in the context of an authenticated user. So a realistic test may need to run as a user of your system, which means exercising the application inside the same identity and data model that customers use.

Volume meets your own defences. Your WAF and rate limiter do not know the difference between a test and an attack. They will do what you configured them to do, which is usually to throttle or block. That is not a safety problem so much as a measurement one, and it is the most common reason a first run produces a thin report.

None of that argues against testing production. It argues for deciding the boundaries deliberately rather than discovering them during the run.

The four things worth agreeing before anything runs

Scope, timing, rate and permitted actions. Each is a real decision with a real tradeoff.

Scope, and proving you are allowed

Scope is the list of URLs and repositories a run may touch, and everything absent from that list is out. The part teams underestimate is that scope is also a legal question. Testing a domain you do not control is not a configuration mistake, so a serious platform should make you demonstrate control before it sends anything.

Borg requires domain verification before a run starts, once per domain, via DNS record, and its documentation notes that the verification requests only the records it needs rather than broader DNS or account permissions. Whatever vendor you are evaluating, the question is the same: what stops someone entering a domain they do not own.

The narrower point about scope is that a smaller scope is not automatically a safer one. A scope that excludes your payment flow excludes the findings that would have mattered most. What you want is a scope you chose, not a scope that happened.

Timing, which is weaker than it sounds

The instinct is to run at low traffic. It is reasonable and it is worth doing, but it carries less protection than teams expect. A global product has no quiet hour that is quiet everywhere. A scheduled assessment that repeats monthly will eventually land on a release day. And the failure modes that actually hurt, a state change you did not anticipate or a boundary that turns out to be load bearing, do not care what time it is.

Timing is worth setting. It is not the control that does the work.

Rate, and the tradeoff nobody names

This is where the honest answer is uncomfortable. If your WAF throttles the tester, some parts of the test may no longer be representative. If you allowlist the tester, you may bypass some of the WAF's protections for the duration of the run, and you are measuring more of the application's behaviour without those particular controls in the request path.

Both readings are legitimate and they answer different questions. Testing through the WAF tells you what a casual attacker meets. Testing behind it tells you what your application would do if the WAF were bypassed, misconfigured or simply not in the request path, which is a scenario worth understanding rather than assuming away.

Borg's setup runs a connectivity check per URL and reports one of three states: reachable, reachable with warnings where requests are getting through but being flagged or throttled, and unreachable. That middle state is the one to read carefully, because a run that completes against a throttling WAF can produce a clean report for the wrong reason.

Decide which question you are asking. Do not let a rate limiter decide it for you by accident.

Permitted actions

The last one is the most specific to your application and the least standardised across the industry. Which actions may a test perform. May it create records. May it modify them. May it exercise a deletion path, a refund path, an invitation flow, an export.

There is no universal right answer here, which is exactly why it belongs in the conversation before the first run rather than in an incident review afterwards. A team that has thought about it usually arrives with a short list of things that must never be touched, and that list is more useful than any general assurance a vendor can offer.

One practical note that belongs here. Borg's documentation says Mjolnir generates its own user contexts as it explores rather than requiring you to supply test accounts. That is convenient at setup, and it is also a thing to factor in when you decide what a run may touch, because it means the test is operating as a user of your system rather than alongside it.

Production, staging, or both

The reflex answer is to point the first run at staging. It is a sensible instinct and it is often the right first move. It is also worth being clear that staging answers a different question rather than the same question more safely.

Staging usually differs from production in the three ways that matter to an authorisation test. The data is synthetic, so a cross tenant read that would be serious in production may look trivial. The configuration drifts, so the auth provider, feature flags and middleware may not match. And the scale differs, so anything that depends on volume or timing behaves differently or not at all.

The consequence is that a clean staging run is weaker evidence than it feels, and a finding on staging still needs to be assessed against the environment you actually care about before you assume the same condition exists there.

What to insist on, from anyone

Four things are reasonable to require of any vendor running offensive tests against a system you are responsible for, and none of them are exotic.

An abort control you can reach without opening a support ticket. Scope you define explicitly rather than one inferred from your domain. Proof of ownership before testing begins. And enough activity and coverage information to establish whether the test was involved at all, because the first thing you will want during an incident is to establish what the assessment was doing.

That last one is worth pushing on. A platform that reports findings but not activity leaves you unable to answer the only question that matters at the moment something breaks.

Where Borg sits, with its limits

Being specific about our own product is more useful than a reassurance, so here is the shape of it, including the parts that constrain you.

A run operates within the target URLs and repositories you configure, while Mjolnir uses the connected source code and reconnaissance to map and test the application's attack surface within that scope. Domain verification is required before a run starts. The connectivity check tells you up front whether your WAF is throttling the test rather than leaving you to infer it from a thin report. Runs can be started from the Mjolnir setup, and teams should revisit their configuration as the application and its testing requirements change.

The limitations are real and worth stating plainly. A run cannot be paused once it has started: you can abort it, but the assessment then has to be restarted from the beginning, so aborting late is expensive. Only one run can be active at a time. A scheduled assessment keeps the config it was created from and does not pick up later setup changes, so a recurring assessment should be revisited as the application changes: a scope that was right in March may no longer cover everything you care about in July. And, as above, a run generates its own user contexts rather than using accounts you provision.

None of that makes the operational question disappear. It makes it a configuration you can inspect rather than a promise you have to accept.

When something does go wrong

The preparation that pays is not the part that prevents incidents. It is the part that lets you identify one quickly.

When an alert fires during a test window, the expensive version of that night is the one where nobody can establish whether the testing is involved. On-call starts debugging a production fault while a security tool is sending attack-shaped traffic at the same system, and the two investigations run in parallel for an hour before anyone connects them. That is a coordination failure rather than a tooling failure, and it is avoidable in about ten minutes of preparation.

Three things make it avoidable.

Know how to recognise the traffic before you need to. This is the underrated benefit of allowlisting, and it has nothing to do with getting through the WAF. Once you have added a known set of source addresses, you have a filter your on-call can apply to any log in seconds. The same is true of a custom header attached to the requests. Either way the question "is this us" becomes a query rather than a debate.

Decide the abort threshold in advance, and know what it costs. Aborting is the control you actually have during a run, so it is worth knowing beforehand what would trigger it and who is allowed to make that call at three in the morning. It is also worth knowing the price. In Borg's case a run cannot be paused, so an abort means starting the assessment again from the beginning rather than resuming, which is a real consideration when a run is most of the way through.

Tell the people who will see it. On-call, whoever watches the error budget, and anyone who receives automated alerts from the target system. The failure here is rarely dramatic. It is usually a support engineer spending an afternoon on anomalous account activity that a colleague authorised the week before and nobody wrote down.

None of this is sophisticated. It is the difference between a test that produces findings and a test that produces findings plus an afternoon of confusion, and the second outcome is almost always down to preparation nobody thought was worth the ten minutes.

What this does not cover

Two honest limits.

The first is that everything above is about bounding operational risk during a test, not about eliminating it. Offensive testing against a live system carries some, and a vendor claiming otherwise is telling you something that cannot be true of any tool that sends real requests. The right posture is a scope you chose, an abort you can reach and a team who knows the run is happening.

The second is that this is a question about testing safely, not about what testing finds. A perfectly bounded run against a scope that excludes your most interesting paths is safe and not very useful. The scoping conversation and the coverage conversation pull against each other, and treating them as one conversation is how teams end up with a clean repo|rt that means very little.

If you want the reasoning behind why reading the code changes which paths are worth attacking in the first place, that sits in code-aware application security testing. If the question is narrower, whether a specific change is safe rather than whether the system is, testing bound to a pull request is the smaller version of this decision.