Loglune debug · Engineering blog
How to reproduce a bug that only happens for one customer
A bug that only fires in one customer's account usually lives in their configuration, not your code. Capture that customer's feature-flag state and inject it offline through the SDK's own API to reproduce it on your own infrastructure.
A bug that only shows up in one customer’s account usually is not in your code — it is in their configuration. Their feature flags, plan, and settings combine into a state your own environment never recreates. The reliable way to reproduce it: capture that customer’s current feature-flag state, inject it into your own test environment through the flag SDK’s official offline mode, and run the reproduction on infrastructure you already own.
Undo.io reports that “91% of software developers admit to having unresolved defects because these defects cannot be reproduced.” A configuration-specific defect is one of the hardest of those cases, because the failing state lives in a system you cannot see from your desk.
Why it will not reproduce at your desk
When a defect fires only under one customer’s flag, plan, and config combination, your local environment is evaluating a different state. The code path is the same; the inputs are not. Support and engineering end up guessing at a configuration they cannot observe, and every “works on my machine” is technically true — you are running the same program against different flags.
So the goal is narrow: get your process to evaluate the customer’s actual flag state, without touching production and without rebuilding their whole account by hand.
Capture the flag state, then inject it offline
There is a dependable four-step path, and only two of the steps are yours to build:
- Capture — dump the customer’s current flag definitions straight from their provider’s SDK endpoint. No transformation; you want the definitions exactly as the provider serves them.
- Store — keep that snapshot alongside its metadata: which customer, which provider, which version, captured when.
- Inject — hand the snapshot to your process through the SDK’s own official offline source, so the SDK reads the snapshot instead of calling the live service.
- Launch — run the reproduction in your own dev, CI, or staging environment.
The injection runs inside your process, so there is no shared environment to stand up and wait on. Capture and inject are the only steps that need tooling; launch happens wherever you already run your tests.
Every major SDK already ships an official offline mode
You do not need to fake the provider or stub the network. Each SDK has a documented way to read a local snapshot instead of calling the live service — the same mechanism teams already use for tests, pointed at a real customer’s captured state:
| Provider | Official offline source |
|---|---|
| LaunchDarkly | FileData source (server-side SDK; client-side uses the TestData source) |
| GrowthBook | init({ payload }) |
| ConfigCat | flag overrides with LocalOnly behaviour |
| flagd | offlineFlagSourcePath |
| Unleash | bootstrap |
Some providers accept the dump without any transformation — LaunchDarkly, GrowthBook, and Unleash inject the captured definitions as-is. Others need the snapshot mapped into their own offline format first: ConfigCat and flagd expect the flags in a specific shape, so an adapter converts the dump before injection. Either way, the SDK does the evaluation, exactly as it would in production.
Why offline injection beats a shared staging clone
A cloned staging environment has to be provisioned, kept in sync with production, and paid for while it sits idle. Offline injection avoids all three:
- It runs in the process you already have. The snapshot is delivered into your own dev, CI, or staging run, so there is no separate environment between you and the bug — and no compute for anyone else to host.
- Nothing is normalised across providers. Each provider’s flags are injected in that provider’s own format, so no evaluation logic is lost translating between shapes. The SDK you ship is the SDK that evaluates.
- It is scoped to flags on purpose. The reproduction targets the configuration layer — flags, plan, config — which is where tenant-specific bugs actually live.
What you can and cannot reproduce this way
This method reproduces the customer’s current flag state — a snapshot of their flags as they are now, not as they were at the exact moment the bug was reported. Restoring a past state across providers is not possible, because there is no cross-provider time-travel API to rewind to. In practice that is rarely a blocker: a tenant-specific bug is usually still living in the present configuration when you go to reproduce it, and the current snapshot is what lets you see it fire.
Reproduce it in one step with debug
debug runs exactly this pipeline. It snapshots a customer’s live feature-flag state and injects it into your own environment through each SDK’s official offline API — LaunchDarkly, GrowthBook, ConfigCat, flagd, and Unleash, each handled by its own adapter. You pick the provider and the customer’s state; the reproduction runs on the infrastructure you already maintain.
Billing follows the work: one inject delivery counts as one reproduction, metered per run, with no idle-hour charge, and the pricing is per reproduction rather than per environment-hour. More engineering write-ups are on the debug blog.