Skip to main content
darkship docs
Open console

Concepts · Caching

Snapshots, ETags and failing static

The database is never on the evaluation path. Every change becomes a versioned snapshot that is pushed through Redis to memory, and when a dependency breaks we keep serving the last good one.

Updated Oct 28, 20265 min read Edit this page on GitHub (opens in a new tab)

The ruleset snapshot#

Each environment has exactly one snapshot: the complete, self-contained ruleset for that environment. It holds every flag with its variations, targeting rules and per-flag salt, the resolved definitions of the segments those rules reference, and any JSON Schemas. It does not reference anything else, so a server SDK needs one request to evaluate every flag.

The snapshot's content hash is its version, and the same value is the HTTP ETag. Two snapshots with the same hash are identical, which is what makes revalidation cheap.

Derived data

A snapshot is always rebuildable from the control-plane database. Losing Redis loses speed, never data.

Snapshot propagation#

When you save a flag, toggle an environment, edit a segment or change a key's state, the API regenerates that environment's snapshot in the same operation as the committed change and writes it to Redis (write-through). It then publishes an invalidation on Redis pub/sub; every eval instance reloads the snapshot from Redis into its in-memory (L1) copy. A periodic comparison of version hashes catches any missed message.

Propagation budget, console click to SDKA timeline from 0 to 5 seconds. Snapshot build and write take about 0.45 seconds, the eval pods reload by 0.9 seconds, then a server SDK (streaming by default) is pushed the change almost immediately, while an SDK that fell back to polling waits up to its poll interval. The p95 target is 5 seconds. 0 s1 s2 s3 s4 s5 s Save + build snapshot Redis write-through · pub/sub · L1 reload Fallback: polling SDK waits for its next poll (≤ interval, with jitter) Server SDK (default streaming: true): SSE push p95 budget: 5 s Illustrative budget; values are targets from the design, tuned from load tests.
FIG 1 Where the five seconds go. Server work is a fraction of a second and server SDKs stream by default, so the change lands well inside the budget; only an SDK that fell back to polling waits for its next poll.
  1. Commit and snapshotAPI commits the change and regenerates the environment snapshot, computing its hash.
    api
  2. Write-through to RedisThe snapshot is stored in the same operation as the committed change. Reads never fill Redis from Postgres on a miss.
    redis
  3. InvalidateA pub/sub message tells every eval instance to reload; a periodic hash comparison guards against a lost message.
    pub/sub
  4. SDK observesA polling SDK sends If-None-Match and receives the new body on its next poll; a streaming SDK is told immediately.
    sdk
Streaming first, polling as the fallback

Server SDKs (JavaScript and Python) stream by default (GET /v1/sdk/stream; any event triggers an ETag refetch of the snapshot), which is what keeps the console-to-SDK p95 under 5 s. While the stream is down they poll every 30 seconds (jittered by ±20%) and reconnect; while it is up they still poll every 5 minutes as a safety net. With streaming: false an SDK sees a change within about one poll interval, and every poll, including a 304, counts as a request against your plan. The server can tell SDKs to slow down with Retry-After.

ETag revalidation#

A server SDK stores the version it holds and sends it back on every poll. If nothing changed, the response is a tiny 304 with no body:

HTTPsteady state
GET /v1/sdk/snapshot HTTP/1.1
Host: sdk.darkship.tech
Authorization: Bearer ds_srv_prod_9Qm...
If-None-Match: "492be076feb012e6aa08cf33ebb022944ef343728b997fa343cad22f29ad8712"

HTTP/1.1 304 Not Modified
ETag: "492be076feb012e6aa08cf33ebb022944ef343728b997fa343cad22f29ad8712"
Cache-Control: private, max-age=0, must-revalidate

After a change the same request returns 200 with the new snapshot and a new ETag. The exact Cache-Control values are placeholders pending load tests; the contract (ETag and conditional requests) is stable, and a CDN could later sit in front of it without changing SDKs.

Fail static#

If Redis or PostgreSQL becomes unavailable, eval keeps serving the last snapshot it holds in memory. It does not return errors, and it does not fall back to querying Postgres. Staleness is bounded only by how long the dependency is down, and is surfaced in metrics and in the /ready health output.

Try it. Break something and see what stays up.

Fail-static simulator#

All dependencies up
Fail-static behaviourPostgreSQL and Redis feed the API and eval service. When either is down, eval keeps serving its in-memory snapshot to SDKs, but flag writes stop. Postgressource of truth APIaccepts writes Redissnapshots evalL1 snapshotage 0 s SDKsserving control planecontrol planeshared cachedata plane control plane (writes)data plane (reads) never queries Postgres
Normal operation

Console writes publish a new snapshot; evaluations resolve from memory. Nothing is degraded.

Dependency outage matrix#

What each failure means for you.
DownEvaluationsFlag changesVisible as
RedisServed from L1 memory; correct as of the last snapshotSaved but not propagated until Redis returnsSnapshot age rising on /ready
PostgreSQLUnaffectedRejected; console shows a bannerControl-plane errors only
BothServed from L1 memoryNot possibleSnapshot age rising; banner in console
eval itselfServer SDKs keep their in-memory ruleset; client SDKs use session cacheUnaffectedSDK error events; status page

Cold start edge cases#

  • A fresh eval instance starts while Redis is down. It has no snapshot, so it reports not-ready and receives no traffic. Other instances keep serving.
  • A fresh server SDK starts while we are unreachable. With no ruleset it returns your defaults (reason NOT_READY). Supply a bootstrap snapshot you saved earlier to avoid even that.
  • A running server SDK loses us. It keeps evaluating from its last ruleset indefinitely, retrying with exponential backoff and jitter, and emits an error event.
Static means stale, on purpose

During an outage, a flag you flipped minutes ago may not be flipped for running SDKs. We choose "stale but consistent with a real past state" over errors. If you need an emergency off-switch that doesn't depend on us, wrap the risky path with your own config as well.

Quota never stops evaluation#

The same philosophy applies to billing and quota: evaluation endpoints keep serving in every plan state, including over-limit. Only control-plane writes that create new things (flags, keys, members) are restricted after a grace period.

Was this helpful?