Preview Environment Security Testing: Authenticated Checks

Preview Environment Security Testing: Authenticated Checks

To run authenticated security tests safely in a pull request preview, give the scan its own synthetic tenants and short-lived least-privilege credentials, keep those credentials out of reach of untrusted PR code, prove the session holds the intended role before and during the run, and treat a login that silently failed as incomplete coverage, not a clean result.

That sounds obvious. In practice, preview environment security testing goes wrong in quiet ways. Imagine a scan that reports zero findings against a preview. The login step failed on request one, the crawler spent forty minutes on the public marketing pages, and the pipeline went green. Nothing was clean. Nothing was tested.

This guide is the operational harness: what to pin down before the scan starts, how to hand it credentials without leaking them, how to verify the session, how to bound the run, and how to read the result. It is not a scanner comparison. For what runs on the diff itself, see PR security review tools compared.

Why an unauthenticated crawl of a preview tells you very little

An unauthenticated crawl of a preview environment mostly tests the login page, because almost everything a SaaS, fintech or AI product does sits behind a session. The routes that matter for a pull request, such as a new PATCH /api/v1/projects/{id} handler or a changed billing endpoint, are never requested.

The result looks reassuring and means little. Missing security headers and exposed debug routes will show up. Access control on an object, a role check on an admin action, or a tenant filter on a query will not, because the scanner never held an identity that could reach them.

Preview environments make this worse. Per-PR deployments are often reachable from the internet, run branch code nobody has reviewed yet, and are wired to staging services with more access than anyone remembers granting. Testing them as an anonymous visitor skips the part you most need to check: what each logged-in user can do on code that has not merged.

Four approaches compared: crawl, single role, role and tenant fixtures, code-informed runtime assessment

The four approaches differ in what they need as input, what evidence they produce, and what they cannot tell you. Pick by the question you are asking, not by what is easiest to wire into CI.

The honest reading: the first approach is hygiene, the second is a minimum for authenticated apps, and the third is where authorisation and tenancy actually get tested. The fourth answers a different question. It uses the repository to decide which paths to attack and reports what the running system confirms. It suits scheduled or release-level assessment more than every push. For a PR, the useful pairing is usually a fixture-based authenticated run on the preview plus code-aware review of the diff. Full pentest vs PR security review covers how those two questions differ.

Know what you are testing: preview identity, scope allowlist and synthetic data

Before any credential exists, pin down which deployment is under test and what it is allowed to touch. Three things need to be explicit.

Preview identity. Record the PR number, the commit SHA and the preview URL, and refuse to start until the deployment reports that same SHA. A scan against a preview that is still serving the previous commit produces evidence about the wrong code. Expose the SHA on a health or version endpoint and have the harness compare it.

A scope allowlist. List the exact hostnames the run may request, and make everything else out of scope by default. Preview apps call out to identity providers, analytics, object storage and third-party APIs. A scanner following links must not wander into any of them. An allowlist beats an exclusion list because new dependencies appear without anyone updating the exclusions.

Synthetic data only. The preview should connect to a seeded database with fabricated tenants, never a copy of production and never real customer records. If your previews point at a shared staging database, fix that first. Authenticated testing writes data, and writing into a shared store is how a PR scan ends up creating records other people find. Seed per preview, or reset on teardown.

This is also where the risk of the preview itself sits. If the environment can reach production services with real credentials, the test harness inherits that reach. Cut it before testing.

Test credentials: least privilege, short-lived, and out of reach of untrusted PR code

Test credentials for a preview should be minted per run, scoped to synthetic tenants, expire within the run, and never be readable by code from the pull request under test. Each of those four properties closes a different hole.

Least privilege. Create one account per role you intend to test, each with only that role's permissions. Do not reuse a staff or admin login because it is convenient. If a test account leaks, it should open nothing real.

Short-lived. Issue credentials when the preview is created and revoke them at teardown. A static password stored in CI for months is a standing credential on an environment that changes every push. Where your identity provider supports it, mint tokens with a lifetime slightly longer than the scan.

Out of reach of untrusted code. This is the CI boundary, and it is where most real leaks happen. GitHub's guidance is that workflows triggered by pull_request from forks do not receive repository secrets, while pull_request_target runs in the context of the base repository with access to secrets, so checking out and running pull request code there is dangerous (GitHub Docs, secure use reference). The same reference recommends restricting GITHUB_TOKEN permissions and passing untrusted input through intermediate environment variables rather than interpolating it into scripts. It also notes that secret redaction is not guaranteed because values can be transformed.

Practical consequences, offered as recommendations rather than guarantees:

• Run authenticated scans for fork PRs only after a maintainer reviews and approves, or skip them and run on the merged branch.
• Never expose test credentials to a job that builds or executes PR code. Have a separate, trusted job read them and talk to the preview over the network.
• Treat the preview's own logs as readable by the PR author. Do not print tokens, and do not let the app echo the Authorization header.

ZAP supports environment variables for adding authentication headers to requests generally, and the Automation Framework can also use environment-variable tokens in header-based session management (ZAP Automation Framework authentication).

Prove the session is authenticated: verify the intended role, not HTTP 200

A successful login request proves nothing about the session you hold afterwards, so verify the role the scan is actually acting as, and keep verifying while it runs. HTTP 200 is the weakest possible signal. Login pages, error pages and redirected SPAs all return it.

Use a verification request that returns identity, such as GET /api/me, and assert on the response body: the user ID, the role, the tenant ID. Then assert it matches the fixture you meant to log in as. A session that is authenticated as the wrong role is worse than no session, because findings and non-findings get attributed to the wrong viewpoint.

ZAP's Automation Framework supports this directly. You configure authentication verification so ZAP can tell logged-in from logged-out traffic. That includes a poll method that periodically requests a URL and checks the response against logged-in and logged-out indicators, with the frequency set in requests or seconds. Session management is configured separately to match how the app carries its session (ZAP Automation Framework authentication). Choose a poll URL that clearly differs between an authenticated and an expired session, and make the logged-in indicator specific to the role, not just "Welcome".

Two habits make the check trustworthy:

• Fail loudly on the first poll. If verification fails before the crawl starts, abort. Do not let the scan proceed unauthenticated and report success.
• Watch for mid-run expiry. Tokens expire, CSRF tokens rotate, and an app can log you out after a rejected request. Count verification failures during the run. A run that spent half its time logged out covered half the surface.

Bound the run: exclusions, sandboxed side effects, timeouts and concurrency

An authenticated crawl clicks everything it can reach, so bound it before it starts. A logged-in scanner will submit forms, delete records and trigger emails if you let it.

Exclude destructive and business actions. Exclude routes such as account deletion, password change on the test account itself, bulk export, and anything that sends money or messages. Better, make those actions harmless in the preview rather than relying on exclusions alone.

Sandbox side effects. Point email at a capture sink, payments at the provider's test mode, and webhooks at an internal receiver or a no-op. A preview that can send real email or post to a real Slack workspace will do so, once, at scale, during a scan. Confirm each outbound integration in the preview uses a sandbox credential.

Set timeouts and a request budget. Cap total run time and set a ceiling on requests per second. A scan that hammers a small ephemeral instance causes errors that look like findings and slows the deployment other developers are waiting on.

Limit concurrency. Use one authenticated session per role, run in sequence or with a small fixed parallelism. Sessions that share a user can invalidate each other. Also keep the number of simultaneous preview scans across open PRs bounded, or a busy afternoon turns the scan farm into a load test.

Write these limits into the harness config, not into a wiki page. If the runner doesn't enforce them, they are suggestions.

Role and tenant fixtures are specified tests, not automatic authorisation proof

Logging in as several roles and tenants does not prove authorisation is correct. It gives you the ingredients for tests, and the tests still have to be written.

A scanner that holds a tenant A session and a tenant B session does not know that tenant A must never read tenant B's invoices. Unless someone states that expectation, the scanner sees two valid sessions and two valid sets of responses. Do not claim a clean result as proof of tenant isolation merely because the login worked.

Treat each fixture as a specification. Imagine a billing API with admin, member and viewer roles across two synthetic tenants. Write the expectations down:

• viewer in tenant A may GET invoices in tenant A and may not POST a refund.
• member in tenant A may not GET any invoice in tenant B, by direct ID or by listing.
• admin in tenant A may manage users in tenant A and gets a denial on tenant B's user list.

Each line is a request, an identity and an expected allow or deny. Run them as assertions. A failure is a finding with an obvious reproduction. A pass is a pass for that line only. Routes and object types nobody wrote a line for remain untested, and your report should say so.

This is the same discipline behind IDOR testing for APIs and tenant isolation testing. The fixtures you build here feed directly into security regression testing for authorisation fixes once a bug is found and patched.

Evidence, teardown and a decision table: fail, investigate, or complete with limits

A preview scan should end in one of three states, and "green" is not one of them. Three questions get blurred together here: how severe is a finding, how was it confirmed, and how much of the application did the run actually cover. Severity is impact. Confirmation is whether the behaviour was demonstrated or only inferred. Coverage is whether the run reached the routes at all. A low-severity, fully confirmed finding and a high-severity, unconfirmed one need different handling. See verified security findings for the proof-versus-inference distinction.

Tie evidence to the exact commit SHA, preview URL, run time, roles exercised and the verification results. Without that, nobody can tell later whether a result applied to the code that merged.

Then tear down. Destroy the preview, delete the seeded tenants, revoke the test credentials and discard captured traffic that contains session tokens. An ephemeral environment that lingers is a standing target with test accounts on it.

Use a decision table rather than a pass or fail boolean:

The best outcome is still "complete with limits". That is deliberate. A per-PR runtime check answers a bounded question, and the report should state the bounds.

Make a clean result mean something, then add depth where the diff is risky

A preview scan is only worth gating on if a pass cannot be produced by a failed login, a stale deployment or a crawl that never left the homepage. Build the harness around verified roles, synthetic tenants, credentials the PR cannot read, and a report that states its coverage. Then pair it with code-aware testing where the risk is highest: authorisation, tenancy and billing logic. Borg runs code-aware PR security review on the diff and full-application pentests against the running application, with findings confirmed and retested after fixes. To see how that fits alongside your preview pipeline, Book a Demo.