System Design Interview Questions and Answers
39 hand-picked System Design interview questions with
detailed answers. Open the interactive version above to search, filter
by difficulty, run code, bookmark questions and track your progress.
What is a system design interview, and what are interviewers evaluating?
A system design interview asks you to design a real-world system (URL shortener, chat, feed, etc.) on a whiteboard or shared doc. Interviewers evaluate:
- Structured thinking — clarify requirements before jumping to boxes.
- Trade-off reasoning — why this DB, why cache here, what breaks at scale.
- Scalability & reliability — bottlenecks, failure modes, redundancy.
- Communication — narrate assumptions, ask questions, iterate.
At 2–4 YOE, depth on every component isn't expected — a clear framework and sensible choices matter more than memorized architectures.
Functional vs non-functional requirements — how do you gather them?
Functional requirements describe what the system does: features, user flows, APIs, data stored.
Non-functional requirements (NFRs) describe how well it must perform: latency, throughput, availability, consistency, security, cost.
Always clarify both before designing. Example for a URL shortener:
- Functional: shorten URL, redirect, optional custom alias, analytics.
- Non-functional: 100M URLs, 10K redirects/sec, 99.9% uptime, redirect < 100ms p99.
Vertical vs horizontal scaling — when do you use each?
Vertical scaling (scale up) — add more CPU/RAM/disk to one machine. Simple, but hits hardware limits and creates a single point of failure.
Horizontal scaling (scale out) — add more machines behind a load balancer. Better fault tolerance and theoretically unlimited scale, but requires stateless app servers, distributed data, and operational complexity.
Typical path: optimize code → vertical scale → horizontal scale + caching + DB scaling.
What is load balancing and what algorithms do you know?
A load balancer distributes incoming traffic across multiple servers to improve throughput and availability. It also enables health checks and zero-downtime deploys.
Common algorithms:
- Round robin — simple, even distribution.
- Least connections — send to server with fewest active connections.
- Consistent hashing — sticky routing by key (user ID) — useful for caches.
- IP hash — same client → same server (session stickiness).
Layers: L4 (TCP, fast) vs L7 (HTTP-aware, path-based routing).
What caching strategies do you use in system design?
Caching stores frequently accessed data closer to the consumer to reduce latency and DB load.
- Cache-aside — app checks cache, on miss reads DB and populates cache. Most common.
- Read-through — cache library fetches from DB on miss.
- Write-through — write to cache and DB together.
- Write-behind — write to cache, async flush to DB (fast, riskier).
Always plan for cache invalidation, TTLs, and cache stampede (many misses at once).
What is a CDN and when should you use one?
A CDN (Content Delivery Network) caches static content (images, JS, CSS, videos) at edge servers geographically close to users. Benefits:
- Lower latency — content served from nearest PoP.
- Reduced origin load — 80–95% of static traffic off your servers.
- DDoS absorption and TLS termination at the edge.
Use for static assets and cacheable API responses. Don't put personalized or highly dynamic data on CDN without careful cache keys.
How do you scale a database — read replicas and sharding?
Read replicas — copy primary DB data to replicas; route reads to replicas, writes to primary. Good for read-heavy workloads. Watch replication lag for stale reads.
Sharding (partitioning) — split data across multiple DBs by a shard key (user_id, tenant_id). Scales writes and storage, but cross-shard queries and rebalancing are hard.
Typical progression: optimize queries → indexes → read replicas → caching → sharding → specialized stores.
Explain the CAP theorem and what it means in practice.
CAP theorem: in a network partition, a distributed system can guarantee at most two of:
- C — Consistency — every read returns the latest write.
- A — Availability — every request gets a response (not error/timeout).
- P — Partition tolerance — system works despite network failures between nodes.
In practice, partitions happen, so you choose between CP (consistent but may reject requests) and AP (available but may return stale data). Most web systems are AP with tunable consistency.
Strong consistency vs eventual consistency — when to use each?
Strong consistency — after a write, all reads see the new value immediately. Requires coordination (single leader, quorum, locks). Higher latency, lower availability on failures.
Eventual consistency — replicas converge over time; reads may be stale briefly. Higher availability and performance.
Variants: read-your-writes, monotonic reads, causal consistency — pragmatic middle ground for user-facing apps.
REST APIs vs event-driven design — when do you use each?
REST (synchronous) — client calls API, waits for response. Simple, good for request/response flows (CRUD, queries). Couples caller to callee availability.
Events (asynchronous) — producer publishes event; consumers process independently via message broker. Decouples services, handles spikes, enables replay. Harder to debug; eventual consistency.
Use REST for user-facing queries and commands needing immediate feedback. Use events for side effects (send email, update search index, analytics).
What role do message queues play in system design?
Message queues (Kafka, RabbitMQ, SQS) enable async communication between services:
- Decoupling — producer doesn't need consumer online.
- Buffering — absorb traffic spikes without overwhelming downstream.
- Reliability — retry failed processing; dead-letter queues for poison messages.
- Scalability — add consumers to parallelize work.
Design for at-least-once delivery + idempotent consumers unless you truly need exactly-once.
How does rate limiting work and where do you apply it?
Rate limiting caps requests per user/IP/API key over a time window to prevent abuse, DDoS, and cost overruns.
Algorithms:
- Token bucket — tokens refill at fixed rate; burst allowed up to bucket size.
- Fixed window — N requests per minute; simple but boundary spikes.
- Sliding window — smoother; more accurate, slightly more complex.
Implement at API gateway or edge (Cloudflare, Kong). Store counters in Redis for distributed rate limiting. Return 429 Too Many Requests with Retry-After header.
Microservices vs monolith — tradeoffs and when to choose each.
Monolith — single deployable unit. Pros: simple dev, debug, deploy, transactions. Cons: scales as one block, tight coupling over time.
Microservices — independently deployable services. Pros: team autonomy, scale per service, tech diversity. Cons: distributed complexity, network latency, eventual consistency, operational overhead.
At 2–4 YOE startups: start monolith, split when you have clear bounded contexts and team pain — not preemptively.
What is observability — logs, metrics, and traces?
Observability is the ability to understand system health and debug issues from external outputs.
- Logs — discrete events with context (errors, request details). Centralize with ELK/Datadog. Structured JSON logs are searchable.
- Metrics — aggregated numbers over time (QPS, latency p99, error rate, CPU). Alert on thresholds. Prometheus + Grafana.
- Traces — follow one request across services (trace_id spans). Essential for microservices. Jaeger, Zipkin, OpenTelemetry.
Design for observability from day one — you can't debug production without it.
Back-of-envelope estimation basics for system design.
Round aggressively. Useful numbers:
- 1 day ≈ 86,400 sec ≈ 100K sec
- 1M requests/day ≈ 12/sec average (×10 for peak ≈ 120/sec)
- 1 char ≈ 1 byte; 1 KB text ≈ 1K chars
- 1M users × 1 KB profile ≈ 1 GB
- SSD read ≈ 1ms; network round-trip ≈ 1–100ms
Template: users → actions/day → QPS → storage per record × records → bandwidth.
What are single points of failure and how do you eliminate them?
A single point of failure (SPOF) is any component whose failure takes down the whole system.
Common SPOFs: single app server, single DB, single load balancer, single region.
Mitigations:
- Redundancy — multiple instances behind load balancer.
- Replication — DB primary + replicas with automatic failover.
- Multi-AZ / multi-region — survive datacenter outage.
- Health checks — LB removes unhealthy nodes.
What is idempotency and why does it matter in distributed systems?
An operation is idempotent if performing it multiple times has the same effect as once.
Critical because networks fail and clients retry. Without idempotency:
- Retrying POST /pay → double charge.
- Retrying message queue consumer → duplicate email.
Solutions: idempotency keys (client sends unique key, server dedupes), natural idempotency (PUT, DELETE), or store processed IDs and skip duplicates.
SQL vs NoSQL — how do you choose a database?
SQL (PostgreSQL, MySQL) — structured schema, ACID transactions, joins, strong consistency. Best for relational data: orders, accounts, inventory with constraints.
NoSQL — flexible schema, horizontal scale, eventual consistency. Types:
- Document (MongoDB) — nested JSON, catalogs.
- Key-value (Redis, DynamoDB) — caching, sessions.
- Wide-column (Cassandra) — time-series, high writes.
- Graph (Neo4j) — social connections, recommendations.
Default to SQL unless you have a clear reason for NoSQL (massive write scale, flexible schema, geo-distribution).
DRY, KISS, YAGNI, separation of concerns — what do they mean in practice?
- DRY — one authoritative place per piece of knowledge. The nuance: duplicated code that represents different concepts should NOT be merged — wrong abstraction is worse than duplication.
- KISS — the simplest design that meets today's requirement; cleverness is a maintenance tax.
- YAGNI — don't build for imagined future needs (config flags nobody sets, generic engines with one use).
- Separation of concerns — each module owns one aspect (controller/service/repository; smart/dumb components) so changes stay local.
WebSockets vs SSE vs long polling — and why choose SSE for streaming AI responses?
- Long polling — client asks, server holds the request until data arrives, repeat. Works everywhere; high overhead per message.
- SSE (Server-Sent Events) — one long-lived HTTP response streaming
text/event-stream. Server→client only; auto-reconnect built into EventSource; plain HTTP (proxies, LBs, HTTP/2 friendly). - WebSockets — full-duplex TCP after an HTTP upgrade; both directions, binary support; costs a stateful protocol, trickier LB/proxy story, manual reconnect/heartbeats.
LLM streaming is exactly SSE's shape: client sends one request, server streams tokens one way. OpenAI/Anthropic APIs stream via SSE; the UI reads chunks and renders incrementally.
// browser side of an SSE token stream
const es = new EventSource('/api/chat/stream?id=42');
es.onmessage = (e) => {
if (e.data === '[DONE]') { es.close(); return; }
const delta = JSON.parse(e.data).token;
answerEl.textContent += delta; // incremental render
};
es.onerror = () => { /* EventSource auto-reconnects with Last-Event-ID */ };
Serverless functions vs an always-on server — and what are cold starts?
Always-on (VM/container/Spring Boot pod) — a process you run, size and pay for 24/7; state in memory possible; predictable latency.
Serverless (Lambda, Vercel/Netlify functions) — the platform spins up an instance per demand, scales to zero, bills per invocation. No server management, but: stateless by force, execution time limits, and cold starts — the first request after idle pays for provisioning + runtime boot (ms for Node, notoriously seconds for JVM — hence GraalVM native images for serverless Java).
Mitigations: provisioned concurrency/warm-up pings, small bundles, lighter runtimes, keeping DB connections in a pooler (serverless + naive connection-per-invocation exhausts Postgres).
What does 99.5% (or 99.9%) uptime actually mean? How do you design for it?
Availability translates directly into an allowed-downtime budget:
- 99% ≈ 7.3 hours/month
- 99.5% ≈ 3.6 hours/month
- 99.9% ≈ 43 minutes/month
- 99.99% ≈ 4.4 minutes/month
Each nine multiplies cost: redundancy (no SPOF, multi-instance/multi-AZ), health checks + automatic failover, zero-downtime deploys, graceful degradation (serve cached/partial rather than error), and an error budget mindset — planned maintenance and bad deploys spend the same budget.
Monorepo vs polyrepo — trade-offs (especially for microfrontends)?
Monorepo — all apps/libs in one repo (Nx/Turborepo): atomic cross-cutting changes (shared component + all consumers in one PR), one dependency version policy, easy code sharing and refactoring; needs task-graph tooling to build/test only what changed, and CI/permissions get complex at scale.
Polyrepo — repo per app/team: clear ownership and independent pipelines (the natural fit for independently-deployed microfrontends/microservices); the price is version drift, cross-repo changes spanning N PRs, and shared libraries distributed via registries with lagging upgrades.
What is consistent hashing and what problem does it solve?
The naive way to distribute keys across N servers is hash(key) % N. It works — until N changes. Adding or removing one server changes the modulus, so almost every key remaps to a different server. Your entire cache is invalidated at once and the database gets crushed.
Consistent hashing maps both servers and keys onto the same circular hash space (0 to 2³²−1). A key belongs to the first server found moving clockwise. Now adding or removing a node only moves the keys in that node's arc — on average K/N keys instead of nearly all of them.
Virtual nodes fix the remaining problem: with few servers, the ring is unevenly divided and one node gets a disproportionate share. Each physical server is placed at many points on the ring (100–200 virtual nodes), which smooths distribution and lets you weight more powerful machines with more virtual nodes.
Used by: Cassandra and DynamoDB (partitioning), Redis Cluster (hash slots — a related fixed-slot variant), memcached clients, CDN edge selection, and load balancers doing session affinity.
// naive: adding a 5th server remaps ~80% of keys
server = hash(key) % 4 → hash(key) % 5
// consistent hashing: only the new node's arc moves
ring = sorted map of hash(virtualNodeId) -> physicalServer
function getServer(key) {
h = hash(key)
entry = ring.ceilingEntry(h) // first node clockwise
return entry ?? ring.firstEntry() // wrap around the circle
}
// 150 virtual nodes per server → even distribution
for (i = 0; i < 150; i++) ring.put(hash(server.id + "#" + i), server)
Explain circuit breakers, retries, timeouts, and bulkheads.
These four patterns stop one slow dependency from taking down your whole system (a cascading failure).
- Timeout — the foundation. Never make an unbounded network call. Without a timeout, threads pile up waiting on a dead dependency until your service exhausts its pool and dies too. Set it based on the dependency's p99, not a guess.
- Retry with exponential backoff + jitter — retry transient failures (timeout, 503, connection reset), never non-idempotent operations or 4xx. Backoff prevents hammering a struggling service; jitter (randomised delay) prevents the thundering herd where every client retries at the same instant.
- Circuit breaker — after N consecutive failures the breaker opens and calls fail instantly without hitting the network. After a cooldown it goes half-open and lets one trial request through: success closes it, failure re-opens it. This gives the failing service room to recover instead of being retry-stormed.
- Bulkhead — isolate resources per dependency (separate thread pools/connection pools), so a hang in the recommendations service can't consume every thread and block checkout.
Plus graceful degradation: when the breaker is open, serve a cached or default response rather than an error where the product allows it.
// Resilience4j-style configuration
CircuitBreakerConfig.custom()
.failureRateThreshold(50) // open at 50% failures
.slowCallDurationThreshold(Duration.ofSeconds(2))
.waitDurationInOpenState(Duration.ofSeconds(30)) // cooldown before half-open
.permittedNumberOfCallsInHalfOpenState(3)
.slidingWindowSize(100)
.build();
// retry with exponential backoff AND jitter
delay = min(cap, base * 2^attempt)
sleep(random(0, delay)) // full jitter — spreads the retry storm
// bulkhead: recommendations cannot starve checkout
recommendationsPool = ThreadPool(size=10)
checkoutPool = ThreadPool(size=50)
How do you handle transactions across microservices? Explain the Saga pattern.
In a monolith one ACID transaction covers everything. Across services each owns its own database, so there's no shared transaction — you need a different model.
Two-phase commit (2PC) is the classic answer and is usually wrong for microservices: it's synchronous and blocking, holds locks across services for the whole transaction, and the coordinator is a single point of failure. It doesn't scale and it hurts availability.
Saga — break the distributed transaction into a sequence of local transactions, each publishing an event that triggers the next. If a step fails, run compensating transactions to semantically undo the completed steps.
- Choreography — services react to each other's events. No central coordinator, loosely coupled; but the overall flow is implicit and hard to follow or debug once you pass ~4 steps.
- Orchestration — a coordinator service explicitly drives each step and issues compensations. Flow is visible, testable, and easier to reason about; the cost is a component that must itself be reliable.
Key consequence: sagas give you eventual consistency and no isolation — intermediate states are visible to other reads. You must design for it (e.g. an order sits in PENDING until confirmed).
// Saga: place order
Order Service → create order (PENDING) | compensate: cancel order
Payment Service → charge card | compensate: refund
Inventory Service → reserve stock | compensate: release stock
Shipping Service → schedule shipment | compensate: cancel shipment
Order Service → mark order CONFIRMED
// If inventory fails at step 3:
// → release nothing (it failed)
// → refund payment (compensating transaction)
// → cancel order (compensating transaction)
// Orchestrated with a state machine (Temporal / Step Functions / Camunda)
state: CREATED → PAID → RESERVED → SHIPPED → CONFIRMED
↘ FAILED → COMPENSATING → CANCELLED
What is the dual-write problem, and how do the Outbox pattern and CDC solve it?
The dual-write problem: your service must write to the database and publish an event. These are two separate systems with no shared transaction, so any failure between them leaves you inconsistent:
- DB commit succeeds, publish fails → the order exists but no one downstream knows. Silent data loss.
- Publish succeeds, DB commit fails → downstream reacts to an order that doesn't exist. Phantom events.
Wrapping them in application code doesn't help — there's no atomicity across a database and a broker.
Transactional Outbox: in the same local transaction that writes your business data, insert the event into an outbox table. Now it's one atomic commit. A separate relay process reads unpublished outbox rows and pushes them to the broker, marking them sent. If publishing fails it simply retries — the event is durably stored.
CDC (Change Data Capture) is the more elegant relay: a tool like Debezium tails the database's write-ahead log and streams committed changes to Kafka. No polling, no extra load on the DB, and it captures every change including ones made outside your service.
Both give at-least-once delivery, so consumers must be idempotent.
-- ❌ dual write: no atomicity between these two systems
BEGIN; INSERT INTO orders ...; COMMIT;
kafka.publish("order.created", event); -- broker down → event lost forever
-- ✅ outbox: ONE atomic local transaction
BEGIN;
INSERT INTO orders (id, user_id, total, status)
VALUES ('o_991', 'u_88', 4999, 'PENDING');
INSERT INTO outbox (id, aggregate_id, type, payload, created_at)
VALUES (gen_random_uuid(), 'o_991', 'order.created',
'{"orderId":"o_991","total":4999}', now());
COMMIT;
-- relay (or Debezium tailing the WAL) publishes and marks sent
SELECT * FROM outbox WHERE published_at IS NULL ORDER BY created_at LIMIT 100;
Explain CQRS and Event Sourcing. When are they worth the complexity?
CQRS (Command Query Responsibility Segregation) — separate the write model from the read model. Commands go to a normalised transactional store optimised for correctness; queries hit denormalised read models (materialised views, Elasticsearch, a cache) optimised for the exact shapes the UI needs. The two are kept in sync asynchronously via events.
Why: reads and writes have wildly different requirements. Reads are usually 100× more frequent and want pre-joined data; writes want normalisation and constraints. CQRS lets you scale and model them independently.
Event Sourcing — don't store current state; store the ordered sequence of events that produced it. Current state is a fold over the event log. The events are the source of truth and are immutable.
What it buys you: a complete audit trail for free, the ability to reconstruct state at any past point in time ("what did this account look like on 3 March?"), and the ability to build a brand-new read model by replaying history.
The costs are real: eventual consistency between write and read models, event schema versioning forever (you can never delete an old event format), snapshotting so you don't replay millions of events, and the GDPR problem — an immutable log conflicts with the right to erasure (usually solved with crypto-shredding).
// Event-sourced account: state is derived, never stored directly
events = [
{ type: "AccountOpened", balance: 0, at: "2026-01-02" },
{ type: "MoneyDeposited", amount: 5000, at: "2026-02-11" },
{ type: "MoneyWithdrawn", amount: 1200, at: "2026-03-03" },
{ type: "MoneyDeposited", amount: 700, at: "2026-04-19" }
];
balance = events.reduce(apply, initialState); // → 4500
// snapshot every N events so replay stays fast
snapshot = { version: 1000, balance: 4500 };
state = replay(snapshot, eventsAfter(1000));
// CQRS: same events project into purpose-built read models
// → Postgres (transactional writes)
// → Elasticsearch (search)
// → Redis (dashboard counters)
What are cache stampede, hot keys, and the cache invalidation problem?
Cache stampede / thundering herd — a popular key expires and 10,000 concurrent requests all miss simultaneously, so all 10,000 hit the database at once. The DB falls over, and often the resulting slowness prevents the cache being repopulated, so it keeps happening. Fixes:
- Lock / single-flight — only the first request recomputes; the rest wait for that result.
- Probabilistic early expiry — refresh slightly before TTL, with randomisation, so expiries don't align.
- Stale-while-revalidate — serve the stale value and refresh in the background. Usually the best UX.
- Jittered TTLs — never give a batch of keys the identical TTL.
Hot key — one key (a celebrity's profile, a flash-sale product) receives a disproportionate share of traffic and saturates the single shard holding it. Fixes: replicate the key across shards with a suffix, add a small local in-process cache in front of Redis, or serve it from the CDN.
Cache penetration — repeated requests for keys that don't exist pass straight through to the DB every time. Fix: cache the negative result with a short TTL, or use a Bloom filter to reject known-missing keys.
Invalidation — the genuinely hard part. Options: TTL (simple, but serves stale data), write-through/write-behind, explicit delete-on-write (race-prone), or event-driven invalidation via CDC.
// single-flight: only one request recomputes, others wait
async function get(key) {
const hit = await redis.get(key);
if (hit) return JSON.parse(hit);
const lock = await redis.set(`lock:${key}`, 1, { NX: true, PX: 5000 });
if (!lock) { // someone else is already recomputing
await sleep(50);
return get(key);
}
const value = await db.query(key);
await redis.set(key, JSON.stringify(value), { EX: jitter(3600) });
await redis.del(`lock:${key}`);
return value;
}
// jittered TTL — never let a batch of keys expire together
const jitter = (base) => base + Math.floor(Math.random() * base * 0.2);
// negative caching against penetration
if (!value) await redis.set(key, "NULL", { EX: 60 });
How do you choose a partition/shard key? What goes wrong if you get it wrong?
A good partition key must satisfy three things at once, and they often conflict:
- Even distribution — no hot partitions.
- Query alignment — your most common queries should be answerable from a single partition. Cross-partition scatter-gather queries are slow and don't scale.
- Stability — the value shouldn't change (changing a partition key means deleting and re-inserting the row).
Partitioning strategies:
- Hash —
hash(user_id). Even distribution, but range queries must hit every shard. - Range — by date or alphabetical. Great for range scans, but prone to hotspots (all of today's writes land on one shard).
- Directory/lookup — an explicit map of key → shard. Maximum flexibility (you can rebalance individual tenants), at the cost of a lookup service that must be highly available.
- Geo — by region, for data residency and latency.
What goes wrong: a hot partition (sharding a multi-tenant SaaS by tenant_id when one customer is 40% of your traffic), scatter-gather queries because the key doesn't match access patterns, and resharding — which is genuinely painful, requiring dual-writes and a backfill. Choosing badly is expensive to undo.
-- ❌ range by date: every write today hits ONE shard
PARTITION BY RANGE (created_at)
-- ❌ low cardinality: only 3 possible partitions
PARTITION BY (country) -- 90% of rows are 'IN'
-- ✅ hash on a high-cardinality key aligned with queries
PARTITION BY HASH (user_id) -- "get my orders" = one partition
-- ✅ composite key: split whale tenants across buckets
partition_key = tenant_id + "#" + (hash(order_id) % 10)
-- DynamoDB: partition key + sort key gives single-partition range queries
PK = "USER#u_8821"
SK = "ORDER#2026-08-05#o_991" -- "my orders in date range" = 1 partition
How do distributed systems agree on anything? Explain leader election, quorum, and Raft.
The core problem: multiple nodes, unreliable network, and you need them to agree on a single value (who's the leader, what's the committed order of writes) even when some nodes fail.
Quorum — require a majority to agree. With N nodes a quorum is N/2 + 1, which guarantees any two quorums overlap, so there can never be two conflicting decisions. This is why clusters are sized in odd numbers (3, 5, 7): 4 nodes tolerate the same single failure as 3 but need a larger majority.
Raft — the consensus algorithm most systems use because it's understandable:
- Nodes are follower, candidate, or leader.
- A follower that stops hearing heartbeats becomes a candidate and requests votes for a new term.
- Winning a majority makes it leader; all writes go through the leader, which replicates to followers.
- An entry is committed once a majority has persisted it.
- Randomised election timeouts prevent repeated split votes.
Split brain — a network partition leaves two halves each thinking they're in charge, both accepting writes, and the data diverges. Quorum prevents it: the minority side cannot reach a majority so it stops accepting writes (choosing consistency over availability — the CP side of CAP).
You rarely implement this; you use it: etcd (Kubernetes), ZooKeeper (Kafka's older versions), Consul, and the Raft inside most modern distributed databases.
// Why odd numbers: quorum = floor(N/2) + 1
// N=3 → quorum 2 → tolerates 1 failure
// N=4 → quorum 3 → tolerates 1 failure (no gain, more cost)
// N=5 → quorum 3 → tolerates 2 failures
// Leader election via etcd lease — the practical version
lease = etcd.grantLease(ttl=10s)
won = etcd.putIfAbsent("/service/leader", nodeId, lease)
if (won) {
startHeartbeat(lease) // keep the lease alive
runLeaderDuties() // e.g. the only node running the cron
} else {
watch("/service/leader") // stand by, take over if the key disappears
}
What latency numbers should you know, and how do you use them in a design?
| Operation | Rough latency |
|---|
| L1 / L2 cache reference | ~1 ns / ~4 ns |
| Main memory reference | ~100 ns |
| Read 1 MB sequentially from memory | ~10 µs |
| SSD random read | ~100 µs |
| Read 1 MB from SSD | ~500 µs |
| Round trip within the same datacentre | ~0.5 ms |
| Redis / memcached GET | ~1 ms |
| Database query (indexed, warm) | 1–10 ms |
| Disk seek (spinning) | ~10 ms |
| India → US round trip | ~200 ms |
The ratios matter more than the numbers: memory is ~100× faster than SSD, SSD is ~100× faster than a disk seek, and a cross-continent round trip is ~400× a same-datacentre one.
How to use them in a design:
- Justify caching: 1ms Redis vs 10ms DB is a 10× win on a read-heavy path.
- Justify a CDN: you cannot beat the speed of light — a user in Mumbai hitting a US origin pays ~200ms no matter how fast your code is.
- Spot N+1 problems in your own design: 100 sequential 1ms calls is 100ms; batch or parallelise them.
- Sanity-check an SLO: a 100ms p99 budget cannot contain three sequential cross-region calls.
// budget a request against the numbers
API gateway ~1 ms
Auth (cached token) ~1 ms
Redis lookup ~1 ms
DB query (indexed) ~5 ms
Serialise + network ~3 ms
------
~11 ms ✅ well inside a 100ms p99 budget
// the same design done badly
50 × sequential DB call (5 ms) = 250 ms ❌ N+1
→ batch into one IN-clause query = 8 ms ✅
What are backpressure and load shedding? How do you handle more traffic than you can serve?
When demand exceeds capacity you have exactly three options: queue it, shed it, or fall over. Falling over is the default if you don't design for the other two.
Backpressure — the system signals upstream to slow down instead of silently accumulating work. Without it, an unbounded queue grows until memory is exhausted, and by the time items are processed they're already useless (the client timed out 30 seconds ago). Mechanisms: bounded queues, TCP flow control, reactive streams request(n), consumer-lag-driven producer throttling, and returning 429 with Retry-After.
Load shedding — deliberately reject a fraction of requests to keep the rest healthy. Serving 70% of traffic well beats serving 100% at a latency where everything times out anyway. Shed by priority: drop background/batch/analytics traffic before checkout traffic, and drop anonymous before paying users.
Supporting techniques: admission control based on queue depth or latency, bounded queues with a drop policy, autoscaling (too slow to be your only answer — it takes minutes), and graceful degradation (turn off recommendations, keep checkout).
// bounded queue + drop policy — never let it grow unbounded
executor = ThreadPoolExecutor(
corePoolSize = 20,
queue = ArrayBlockingQueue(capacity = 500), // BOUNDED
rejectionPolicy = CallerRunsPolicy() // natural backpressure
);
// admission control: shed by priority when saturated
if (queueDepth > HIGH_WATERMARK) {
if (request.priority == BACKGROUND) return 503; // shed first
if (request.user.tier == ANONYMOUS) return 429;
}
// drop work the client no longer wants
if (now() - request.enqueuedAt > request.clientTimeout) {
metrics.increment("requests.dropped.stale");
return; // the caller gave up already — processing it is pure waste
}
How do you change a database schema (or migrate a database) with zero downtime?
The constraint that drives everything: during a rolling deploy, old and new application code run simultaneously against the same database. So every schema change must be compatible with both versions.
Expand / Migrate / Contract — split the change across releases:
- Expand — add the new column as nullable (never
NOT NULL with a default on a huge table — that can lock it). Deploy code that writes to both old and new, reads from old. - Migrate — backfill existing rows in small batches, throttled so you don't saturate the DB.
- Switch reads — deploy code that reads from the new column. Verify.
- Contract — in a later release, stop writing the old column, then drop it.
Never rename a column, drop a column still referenced by running code, or change a type in place — each breaks the old version mid-deploy and makes rollback impossible.
Migrating a whole database (e.g. Postgres → a new cluster, or monolith DB → service DB) uses the same shape at a larger scale: dual-write to both, backfill historical data, run a continuous reconciliation job comparing the two, shift reads gradually (shadow reads → % of traffic → 100%), then decommission. Keep the rollback path alive until you're confident.
-- ❌ breaks the old version instantly, and can lock a large table
ALTER TABLE users RENAME COLUMN phone TO phone_number;
ALTER TABLE users ADD COLUMN tier VARCHAR NOT NULL DEFAULT 'free';
-- ✅ Release 1: EXPAND (nullable, no lock, dual-write in code)
ALTER TABLE users ADD COLUMN phone_number VARCHAR NULL;
CREATE INDEX CONCURRENTLY idx_users_phone_number ON users(phone_number);
-- ✅ Release 1.5: BACKFILL in throttled batches
UPDATE users SET phone_number = phone
WHERE phone_number IS NULL AND id BETWEEN :lo AND :hi; -- loop, sleep between
-- ✅ Release 2: switch reads to phone_number, verify
-- ✅ Release 3: CONTRACT — stop writing `phone`, then drop it
ALTER TABLE users DROP COLUMN phone;
How do you design for multi-region? Explain RTO, RPO, and the DR strategies.
RPO (Recovery Point Objective) — how much data you can afford to lose, measured in time. RPO of 5 minutes means you accept losing the last 5 minutes of writes.
RTO (Recovery Time Objective) — how long you can afford to be down. These two numbers drive the entire design and the cost.
DR strategies, cheapest to most expensive:
- Backup & restore — restore from snapshots. RTO hours, RPO hours. Cheapest.
- Pilot light — core data replicated continuously, minimal infra running; scale up on failover. RTO ~tens of minutes.
- Warm standby — a scaled-down but fully functional copy running; scale up and shift traffic. RTO minutes.
- Active-active (multi-site) — all regions serve traffic. RTO near zero, but you now own the hard problems: conflicting concurrent writes, data residency, and cross-region latency.
The hard part is always data. Stateless services are trivial to run in many regions. Databases are not: synchronous cross-region replication buys consistency at the cost of ~100ms+ added write latency; asynchronous replication keeps writes fast but means a regional failure loses recent writes (your RPO) and creates conflicts if both regions accept writes.
Common pragmatic pattern: active-active for reads, single-region for writes (or write-partitioned by user home region), with global load balancing (Route 53 / GSLB) for failover.
// Strategy comparison
Backup & restore RTO: hours RPO: hours Cost: $
Pilot light RTO: 10-30m RPO: minutes Cost: $
Warm standby RTO: minutes RPO: seconds Cost: $$
Active-active RTO: ~0 RPO: ~0 Cost: $$
// Pragmatic pattern: global reads, home-region writes
Route53 latency routing → nearest region
reads → local read replica (fast)
writes → user's home region primary (no write conflicts)
async replication between regions (RPO = replication lag)
// Test it, or it doesn't work
// Scheduled game day: fail over the primary region for real.
How do you handle authentication and authorization in a distributed system?
Authentication = who you are. Authorization = what you may do. Keep them separate: authenticate once at the edge, authorize at each service.
Session vs JWT — the central trade-off:
- Server-side sessions — the session ID is opaque; state lives in Redis. Instantly revocable, small cookie, but requires a lookup on every request and shared session storage.
- JWT — self-contained signed token; any service can verify it with the public key, no lookup, no shared state. The problem: you cannot revoke it. A stolen token is valid until it expires.
The standard resolution: short-lived access tokens (5–15 minutes) plus a long-lived refresh token that is stored server-side and can be revoked. The revocation window is then bounded by the access token's TTL. Add a denylist for immediate revocation of high-value sessions.
OAuth2 / OIDC — OAuth2 is for delegated authorization ("let this app read my calendar"); OIDC layers authentication on top and issues an ID token. Use the authorization-code flow with PKCE for web and mobile; never the implicit flow.
Service-to-service: mTLS or short-lived service tokens (SPIFFE, workload identity) — never a shared static API key. Assume the network is hostile (zero trust): verify every call, even internal ones.
// JWT: signed, self-contained, verifiable without a lookup
{
"sub": "u_8821",
"iss": "https://auth.example.com",
"aud": "api.example.com",
"exp": 1786800000, // SHORT — 15 minutes
"scope": "orders:read orders:write",
"roles": ["customer"]
}
// Edge authenticates once; services authorize locally
API Gateway → verify signature (public key, cached JWKS) → forward claims
Order Service → check scope "orders:write" → allow/deny
// Refresh flow — the revocable half
POST /token { grant_type: "refresh_token", refresh_token: "..." }
→ refresh token looked up server-side (revocable, rotated on use)
→ new 15-minute access token
// Service-to-service: mTLS, not a shared secret
What is a Bloom filter and where would you use one in a system design?
A Bloom filter is a probabilistic set-membership structure: a bit array plus k hash functions. Adding an element sets k bits; checking hashes the element and tests those k bits.
The defining property: false positives are possible, false negatives are not. "Definitely not present" is a guarantee; "probably present" needs verification. It also cannot store or enumerate the elements themselves, and standard Bloom filters don't support deletion (Counting Bloom filters do).
Why bother? Space. Tracking 1 billion items in a hash set needs tens of GB; a Bloom filter with a 1% false-positive rate needs about 1.2 GB — and lookups are O(k), independent of the number of elements.
Where it's used in real systems:
- Cache penetration defence — before hitting the DB for a key, check the filter. "Definitely not present" → return 404 immediately, no DB query.
- LSM-tree databases (Cassandra, RocksDB, HBase) — each SSTable has a Bloom filter, so a read skips files that definitely don't contain the key. This is the classic production use.
- Web crawlers — "have I already crawled this URL?" across billions of URLs.
- Chrome's malicious-URL check, spam filters, and duplicate detection in stream processing.
// k hash functions, m bits
add(key) → for i in 1..k: bits[hash_i(key) % m] = 1
mightContain → for i in 1..k: if bits[hash_i(key) % m] == 0 → DEFINITELY NOT
→ else PROBABLY
// cache penetration defence
if (!bloom.mightContain(userId)) {
return 404; // definitely doesn't exist — no DB hit
}
user = cache.get(userId) ?? db.query(userId); // verify (could be a false positive)
// sizing: 1B items @ 1% FP ≈ 1.2 GB (vs ~40+ GB for a hash set)
// m = -(n * ln p) / (ln 2)^2 , k = (m/n) * ln 2
How does full-text search work? Why not just use SQL LIKE?
LIKE '%laptop%' can't use a B-tree index (the leading wildcard defeats it), so it's a full table scan — fine at 10k rows, fatal at 10M. It also has no relevance ranking, no stemming ("running" won't match "run"), no typo tolerance, and no multi-field scoring.
An inverted index flips the mapping: instead of document → words, it stores word → list of documents containing it. Searching "wireless laptop" becomes two cheap lookups plus a set intersection.
The indexing pipeline (analysis):
- Tokenise — split text into terms.
- Normalise — lowercase, strip punctuation and accents.
- Remove stop words — "the", "a" (sometimes kept for phrase queries).
- Stem / lemmatise — "running", "ran" → "run".
- Optionally add synonyms and n-grams for autocomplete/fuzzy matching.
Ranking uses BM25 (the modern successor to TF-IDF): a term is more significant if it appears often in this document (term frequency) and rarely across all documents (inverse document frequency), normalised by document length.
Architecture: Elasticsearch/OpenSearch shards the index across nodes, each shard with replicas. A query scatters to all shards and gathers the top-K results. The search index is a derived store — the source of truth stays in your database, synced via CDC or a background indexer, which means search is eventually consistent.
// Inverted index
"laptop" → [doc1, doc5, doc9, doc23]
"wireless" → [doc5, doc9, doc41]
// query "wireless laptop" → intersect → [doc5, doc9] → rank by BM25
// Elasticsearch: analysis + multi-field scoring
PUT /products
{
"settings": { "number_of_shards": 3, "number_of_replicas": 1 },
"mappings": { "properties": {
"title": { "type": "text", "analyzer": "english" },
"brand": { "type": "keyword" }, // exact match + aggregations
"price": { "type": "scaled_float", "scaling_factor": 100 }
}}
}
GET /products/_search
{
"query": { "bool": {
"must": [{ "multi_match": {
"query": "wireles laptop",
"fields": ["title^3", "description"], // boost title 3x
"fuzziness": "AUTO" }}], // typo tolerance
"filter": [{ "range": { "price": { "lte": 80000 }}}] // filters don't score
}}
}
// zero-downtime reindex via an alias
// products_v1 ← alias "products" → reindex into products_v2 → atomically swap
How do you design large file upload and storage?
Never proxy file bytes through your application servers. A 2 GB upload through your API ties up a request thread for minutes, consumes memory and bandwidth, and doesn't scale. Use presigned URLs: the client asks your API for permission, your API returns a short-lived signed URL, and the client uploads directly to object storage (S3/GCS/Azure Blob). Your server handles metadata only.
Multipart upload for large files — split into chunks (5–100 MB), upload them in parallel, and retry only the failed chunk instead of the whole file. This also gives you resumable uploads and a progress bar.
The full flow:
- Client requests an upload → API validates (size, type, quota), creates a
PENDING metadata row, returns presigned URL(s). - Client uploads directly to object storage.
- Storage fires an event (S3 event → SQS/Lambda) on completion.
- A worker validates (real MIME type, virus scan), generates derivatives (thumbnails, transcodes), and marks the record
READY. - Downloads are served via CDN, using presigned URLs for private content.
Also handle: deduplication by content hash, lifecycle policies to move cold objects to cheaper storage tiers, and a cleanup job for orphaned PENDING records where the client abandoned the upload.
// 1. client asks for permission — server never touches the bytes
POST /api/uploads { filename, sizeBytes, contentType }
→ 200 {
uploadId: "up_8821",
parts: [ { partNumber: 1, url: "https://s3...&X-Amz-Expires=900" }, ... ]
}
// 2. client PUTs each part directly to S3, in parallel, retrying failures
// 3. client completes the multipart upload
POST /api/uploads/up_8821/complete { parts: [{ partNumber, etag }] }
// 4. S3 event → queue → worker (async)
// verify magic bytes, virus scan, generate thumbnails, mark READY
// metadata model
files: id, user_id, s3_key, size, content_hash, status(PENDING|READY|FAILED),
created_at
// cleanup job: DELETE FROM files WHERE status='PENDING' AND created_at < now()-1d
