System Design LabSystem Design QuestionsDesign Feature Flag Service

Design Feature Flag Service

EasyInfrastructureconfigurationcachinghashingstreaming

Question Overview

Design a LaunchDarkly-style service that lets teams toggle features, target segments, and run sticky percentage rollouts without redeploying. The interesting parts are evaluating flags locally inside SDKs, streaming rule changes in seconds, and staying safe when the flag service itself is down.…

Sign up to see the full question and AI interviewer

Requirements

  • Boolean and multi-variant flags per environment, with targeting rules on user attributes and explicit user lists
  • Sticky percentage rollouts: a user keeps the same variation as the rollout percentage increases
  • Server-side SDKs evaluate locally from cached rules, with no network call per evaluation
  • Client-side SDKs receive only evaluated values, never raw rules or other users' targeting data
  • Flag changes reach connected SDKs within 5 seconds; every change is audited and can be rolled back
  • Applications keep running on last-known or default values if the flag service is unavailable

Back-of-the-envelope numbers

  • Evaluations: 10M/s × 86,400 s ≈ 864B/day, all in-process at about a microsecond each; a remote call per evaluation is infeasible
  • Streaming: 500K SDK connections ÷ ~50K connections per node ≈ 10 streaming nodes, ~20 with headroom and zone redundancy
  • Change fan-out: 20K changes/day ≈ 0.23/s, but one change can push a ~1 KB patch to 50K SDKs ≈ 50 MB in a burst
  • Reconnect storm: 500K SDKs × 500 KB full ruleset ≈ 250 GB, so serve snapshots from a CDN with ETags and jittered reconnects
  • Client-side: 20M users × 5 app starts/day = 100M evaluation requests/day ≈ 1.2K/s average, ~3.5K/s at 3× peak
  • Config storage: 100K flags × ~5 KB ≈ 500 MB, plus 20K changes × 5 KB ≈ 100 MB/day of audit history

Key components

  • Flag management API and UI writing versioned flags, rules, and segments to a relational database with an append-only audit log
  • Ruleset publisher: on each change, builds a versioned per-environment snapshot, stores it in object storage behind a CDN, and emits a patch event
  • Streaming service (SSE or WebSocket) pushing patches to connected SDKs, with polling plus ETags as a fallback
  • SDK evaluator: rules checked in order, then rollout bucket = hash(flag key + salt + user key) mod 100,000, compared to the percentage
  • Client-side evaluation endpoint that evaluates all flags for one user server-side and returns only the resulting values
  • SDK resilience: bootstrap from the last snapshot cached on disk, fall back to coded defaults, and never block startup indefinitely
  • Evaluation telemetry: SDKs send aggregated per-flag variation counts every minute for experiment analysis and stale-flag detection

Common mistakes

  • Calling the flag service over the network for every evaluation, adding latency and a hard dependency to every request
  • Using random() for percentage rollouts, so users flip between variations on every request
  • Hashing only the user ID without the flag key, so the same 10% of users land in every rollout
  • Shipping full targeting rules, including email lists, to browser and mobile SDKs, exposing PII and unreleased feature names
  • Having no safe default when an SDK cannot reach the service at startup, which crashes or stalls the application
  • Letting all SDKs reconnect at once after a streaming deploy, creating a self-inflicted thundering herd

Likely follow-ups

  • How would you implement a global kill switch that must take effect within 1 second?
  • How would you guarantee a user sees the same variation on the server SDK and the mobile SDK?
  • How would you run A/B experiments on top of flags and measure the results correctly?
  • How would you support prerequisite flags that depend on other flags?
  • How would the system behave if the flag service were completely down for an hour?
  • How would you find and clean up stale flags that have been at 100% for months?

No community solutions yet

Be the first to publish your solution

Practice ‘Design Feature Flag Service’ with an AI Interviewer

Get scored feedback on your diagram, scalability approach, and trade-offs. Free while we grow — up to 3 full interviews a day.