Chapter 8: System Design Fundamentals

In 2025, 73% of candidates who failed Staff Engineer interviews at top product companies failed at System Design — not DSA.

Let that land.

You can invert a binary tree in your sleep. You can solve two-sum in three languages. You've memorized the time complexity of every sorting algorithm known to computer science. And none of it matters, because when the interviewer says "design a URL shortener," your mind goes blank.

This is not your fault. The Indian engineering education system — and most company L&D programs — treat system design as something you "pick up on the job." They're wrong. System design is a skill with a learnable framework, repeatable patterns, and concrete building blocks. This chapter gives you those blocks.

By the end of this chapter, you will understand scaling, caching, databases, message queues, the CAP theorem, API design, and back-of-the-envelope estimation — not as textbook definitions, but as tools you can reach for in an interview and on the job. You will never stare at a whiteboard in panic again.

Scaling: The Art of Not Falling Over

Every system you have ever built works fine with one user. The problem starts at ten thousand. At a million. At a hundred million.

Scaling is the discipline of making your system handle more load without degrading. There are exactly two ways to do it, and every architecture decision you will ever make is a choice between them.

Vertical scaling means buying a bigger machine. Your Node.js process is running out of memory? Give it 64 GB instead of 8. CPU pegged at 100%? Upgrade to 32 cores. This is the instinctive move. It is also a dead end.

Vertical scaling hits a hard ceiling — there is only so much RAM you can put in one server, and only so much money your CFO will approve. Worse, a single machine is a single point of failure. When it goes down, everything goes down.

Horizontal scaling means adding more machines. Instead of one beefy server, you run ten modest ones. Instead of one database, you run a cluster. This is how Google, Amazon, and every company you want to work at operates.

Here is the thing nobody tells you: horizontal scaling is not free. It introduces distributed systems problems — consistency, partitioning, coordination — that vertical scaling sidesteps entirely. The art is knowing when to pay that complexity tax.

A Node.js application scales horizontally almost trivially — until it doesn't.

// This works fine on one server
const sessions = new Map();

app.post('/login', async (req, res) => {
  const user = await authenticate(req.body);
  const sessionId = crypto.randomUUID();
  sessions.set(sessionId, { userId: user.id, createdAt: Date.now() });
  res.cookie('sessionId', sessionId);
  res.json({ ok: true });
});

app.get('/me', (req, res) => {
  const session = sessions.get(req.cookies.sessionId);
  if (!session) return res.status(401).json({ error: 'Unauthorized' });
  res.json({ userId: session.userId });
});

Run this on two servers behind a load balancer, and your users will be randomly logged out. Why? Because sessions lives in process memory. Server A has no idea what Server B stored. This is the fundamental tension of distributed systems: state that was free on one machine now costs real engineering to share.

The fix is to externalize state. Move sessions to Redis. Move file uploads to S3. Move your database to a dedicated cluster. Every piece of state that lives outside your application process is a piece of state that survives a server restart and works across instances.

Load balancers sit between your users and your servers. They distribute requests using an algorithm — round-robin, least connections, or IP hash. In the AWS world, this is an Application Load Balancer. In the self-hosted world, it is Nginx or HAProxy.

The load balancer is also your first line of defense. It terminates SSL. It can reject malformed requests before they touch your application. It can serve a static "under maintenance" page while you deploy. Treat it as infrastructure, not an afterthought.

Consistent hashing is the algorithm that makes distributed caches and databases work when servers come and go. Standard hashing — hash(key) % N — breaks when N changes. If you have 4 cache servers and one dies, every single key remaps to a different server. Your cache hit rate drops to zero. Your database melts.

Consistent hashing fixes this by placing both servers and keys on a ring. When a server joins or leaves, only the keys that mapped to that server need to move. Everything else stays put.

// Naive hashing — breaks when servers change
function getServer(key, servers) {
  const hash = hashCode(key);
  return servers[hash % servers.length];
}

// Consistent hashing — survives server changes
class ConsistentHash {
  constructor(servers, virtualNodes = 150) {
    this.ring = new Map(); // sorted map of hash -> server
    this.sortedHashes = [];
    for (const server of servers) {
      for (let i = 0; i < virtualNodes; i++) {
        const hash = hashCode(`${server}-${i}`);
        this.ring.set(hash, server);
        this.sortedHashes.push(hash);
      }
    }
    this.sortedHashes.sort((a, b) => a - b);
  }

  getServer(key) {
    const hash = hashCode(key);
    // Binary search for the first server hash >= key hash
    for (const serverHash of this.sortedHashes) {
      if (serverHash >= hash) return this.ring.get(serverHash);
    }
    // Wrap around to the first server
    return this.ring.get(this.sortedHashes[0]);
  }
}

Virtual nodes — the 150 replicas per server in that code — are the secret sauce. Without them, a server that leaves dumps its entire load onto one neighbor. With them, the load spreads evenly across the remaining servers. This is not an optimization. It is the difference between a cache that works and a cache that causes an outage.

Caching: The Cheapest Performance Win You Will Ever Get

Caching is the answer to the question "why is this slow?" — and the answer is almost always "because you are computing something you already computed."

The math is brutal. A database query that takes 10 ms seems fast. Multiply by 10,000 concurrent users, and you are burning 100 seconds of database time per second of wall clock. Your database cannot keep up. Your p99 latency spikes. Your users leave.

A cache hit in Redis takes under 1 ms. That is a 10x improvement before you have written a single line of application code. Caching is not a nice-to-have. It is the foundation of every system that serves more than a handful of users.

Cache strategies are patterns for keeping your cache useful without serving stale data. The four you must know:

Cache-Aside (Lazy Loading): Your application checks the cache first. On a miss, it fetches from the database, writes to the cache, and returns. This is the default. It is simple and it works.

async function getUser(userId) {
  const cacheKey = `user:${userId}`;
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);

  const user = await db.query('SELECT * FROM users WHERE id = $1', [userId]);
  if (!user) return null;

  await redis.set(cacheKey, JSON.stringify(user), 'EX', 3600); // 1 hour TTL
  return user;
}

The downside: the first request after a cache expiry is always slow. And if your cache is empty — after a deploy, after a restart — every request hits the database. This is called the thundering herd problem, and it can take down your database in seconds.

Write-Through: Your application writes to the cache and the database simultaneously. The cache is always fresh. Reads never miss. The trade-off: every write is slower because it touches two systems.

Write-Behind (Write-Back): Your application writes to the cache and returns immediately. A background process flushes to the database asynchronously. This is fast. It is also dangerous — if the cache dies before the flush, you lose data. Use this for analytics, not for payment transactions.

Refresh-Ahead: Your cache predicts which keys will expire soon and refreshes them before they do. This eliminates the thundering herd without the complexity of write-through. Redis does not support this natively, but you can build it with a background worker that scans TTLs.

Cache invalidation is the hard problem. Phil Karlton's famous quote — "There are only two hard things in Computer Science: cache invalidation and naming things" — is famous because it is true. When you update a user's email in the database, how does the cache know to evict user:42?

The answer is discipline. Every write path must know every cache key it invalidates. If you add a new cache key, you must audit every write path that could make it stale. There is no shortcut.

async function updateUser(userId, fields) {
  await db.query('UPDATE users SET ... WHERE id = $1', [userId]);

  // Invalidate all cache keys that contain this user's data
  await redis.del(`user:${userId}`);
  await redis.del(`user:profile:${userId}`);
  // If you forget one, that cache key serves stale data forever
}

A better pattern for complex objects: cache the database row, not the API response. When the row changes, invalidate one key. Let your API layer assemble the response from cached rows. This is called materialized view caching, and it is how systems at scale stay consistent.

Redis vs. Memcached is a decision you will face. Here is the short answer: use Redis. Memcached is a pure key-value store with no persistence, no data structures beyond strings, and no clustering built in. Redis gives you strings, hashes, lists, sets, sorted sets, streams, pub/sub, Lua scripting, and persistence. The only reason to choose Memcached in 2026 is if you are maintaining a legacy system that already uses it.

CDNs are caching at the edge. CloudFront, Cloudflare, Akamai — they put your static assets (and increasingly your dynamic content) on servers physically close to your users. A user in Mumbai hitting a server in Mumbai gets sub-10ms latency. A user in Mumbai hitting a server in Virginia gets 200ms. That 190ms difference is the CDN's value proposition.

For an Indian audience, CDN placement matters. CloudFront has edge locations in Mumbai, Chennai, New Delhi, Hyderabad, Bangalore, and Kolkata. If your users are in India, your CDN should be too. This is not a theoretical optimization — it is the difference between a snappy app and one that feels broken on a 4G connection in Pune.

---

The Database Decision That Defines Your Architecture

Every system design interview eventually arrives at the same question: "SQL or NoSQL?"

The answer is never "NoSQL is web scale." That was a meme in 2010. It is bad advice in 2026.

The real answer: start with SQL. PostgreSQL and MySQL solve an enormous class of problems with strong consistency guarantees, a query language that every engineer knows, and decades of tooling. You move to NoSQL when — and only when — you hit a specific limitation that SQL cannot solve economically.

Here is the decision framework:

Use SQL (PostgreSQL, MySQL) when:

• Your data is relational — users have orders, orders have line items, line items reference products
• You need ACID transactions — money is involved, and "eventually consistent" is not acceptable
• Your query patterns are varied and unpredictable — you need joins, aggregations, and ad-hoc queries
• You are building something new and do not yet know your access patterns

Use MongoDB when:

• Your data is document-shaped — a user profile with nested arrays of addresses, preferences, and payment methods that you always fetch together
• Your schema changes frequently and migrations are painful
• You need to write fast and read fast on a single document, and you never need joins across documents

Use Cassandra when:

• You need to write at a rate that would melt any single-machine database — IoT sensor data, clickstreams, logs
• You can tolerate eventual consistency
• Your read patterns are known in advance and you can design your partition keys around them

Use DynamoDB when:

• You are on AWS and want a managed NoSQL database with predictable latency at any scale
• You can design your access patterns upfront — DynamoDB punishes ad-hoc queries brutally
• You are willing to pay per request and never think about provisioning servers

The Indian startup ecosystem has a particular love affair with MongoDB. I have seen five-person teams choose MongoDB for a billing system. That is a mistake. Money is relational. Invoices have line items. Line items have tax rates. Tax rates depend on jurisdiction. This is a SQL problem. Use SQL.

Sharding is how you scale a database horizontally. You split your data across multiple database instances by a shard key — usually user_id or tenant_id. All data for a given shard key lives on the same instance.

The shard key is the most important decision you will make about your database architecture. Choose wrong, and you cannot fix it without a multi-month migration.

A good shard key:

• Distributes data evenly — no "hot shard" that gets 80% of traffic
• Matches your most common query pattern — if you always query by user_id, shard by user_id
• Is present in every query — cross-shard queries are expensive or impossible

-- This query works great when sharded by user_id
SELECT * FROM orders WHERE user_id = 42;

-- This query is a disaster — it hits every shard
SELECT * FROM orders WHERE status = 'pending' AND created_at < NOW() - INTERVAL '7 days';

The second query is why you need a data warehouse or a search index (Elasticsearch) alongside your sharded database. OLTP databases are for point queries and small range scans. OLAP is for analytics. Do not mix them.

Replication is how you keep your database available when a server dies. A primary instance accepts writes. One or more replicas receive a stream of changes and apply them. If the primary dies, a replica is promoted.

The trade-off is between synchronous and asynchronous replication. Synchronous replication means the primary waits for the replica to confirm every write before returning to the client. Your data is safe. Your latency is higher. Asynchronous replication means the primary writes and returns immediately. Your latency is low. Your data might be lost if the primary dies before the replica catches up.

For most applications, asynchronous replication with a small replication lag (under 100ms) is the right call. For financial systems, synchronous replication on the write is non-negotiable.

Indexing is the skill that separates engineers who understand databases from engineers who use them as black boxes.

An index is a data structure — usually a B-tree — that lets the database find rows without scanning the entire table. Without an index on email, finding a user by email requires reading every row. With an index, it requires reading a few pages.

The cost: every index slows down writes. When you INSERT a row, every index on that table must be updated. An unindexed table can absorb writes at line rate. A table with ten indexes cannot.

-- This index makes login fast
CREATE INDEX idx_users_email ON users (email);

-- This composite index makes "find all orders for a user, sorted by date" fast
CREATE INDEX idx_orders_user_date ON orders (user_id, created_at DESC);

-- This partial index is smaller and faster — only indexes pending orders
CREATE INDEX idx_orders_pending_date ON orders (created_at)
  WHERE status = 'pending';

The composite index on (user_id, created_at DESC) is worth understanding deeply. The column order matters. The index is sorted by user_id first, then by created_at within each user_id. This means it can answer queries that filter by user_id alone, or by user_id and created_at — but not by created_at alone. The leftmost column rule: an index can serve any query that uses a prefix of its columns.

Use EXPLAIN ANALYZE before you ship. I have seen a missing index turn a 2ms query into a 45-second query under load. The fix was one line of SQL. The outage lasted two hours.

Message Queues: The Backbone of Async Systems

Here is a scenario you have lived: a user uploads a profile picture. Your server resizes it to five dimensions, runs it through a moderation API, generates a WebP version, and uploads all six variants to S3. The user stares at a spinner for 12 seconds. They close the tab.

The fix is not faster resizing. The fix is to stop doing work synchronously that does not need to be synchronous.

A message queue decouples the request from the work. The user uploads the picture. Your server saves the original, drops a message on a queue, and returns "processing" to the client. A worker picks up the message, does the heavy lifting, and updates the database when it is done. The user sees a result in 3 seconds instead of 12, and your server can handle 4x the traffic because it is not blocking on image processing.

Kafka is a distributed commit log. It is designed for high-throughput, persistent, ordered streams of events. Think: every click on your website, every sensor reading from a factory floor, every trade on an exchange. Kafka stores messages on disk — it can retain them for days, weeks, or forever. Consumers read at their own pace using an offset pointer.

Kafka's superpower is replay. If you deploy a buggy consumer that corrupts data, you can reset the offset and reprocess every message from the beginning. No data is lost. This is why Kafka is the backbone of event-driven architectures at companies like Uber, Swiggy, and Zerodha.

// Producer: your Node.js server drops an event
const { Kafka } = require('kafkajs');

const kafka = new Kafka({ brokers: ['localhost:9092'] });
const producer = kafka.producer();

await producer.connect();
await producer.send({
  topic: 'user-events',
  messages: [{
    key: userId,
    value: JSON.stringify({ type: 'PROFILE_PICTURE_UPLOADED', userId, imageUrl })
  }]
});

// Consumer: a worker processes the event
const consumer = kafka.consumer({ groupId: 'image-processor' });

await consumer.connect();
await consumer.subscribe({ topic: 'user-events', fromBeginning: false });

await consumer.run({
  eachMessage: async ({ message }) => {
    const event = JSON.parse(message.value.toString());
    if (event.type === 'PROFILE_PICTURE_UPLOADED') {
      await resizeAndUpload(event.imageUrl);
    }
  }
});

The groupId is critical. Every consumer in the same group shares the work — each partition is assigned to exactly one consumer in the group. Add more consumers, and you process faster. This is horizontal scaling for your background workers.

RabbitMQ is a message broker built on AMQP. It is designed for routing — complex topologies of exchanges, queues, and bindings. Use RabbitMQ when you need fine-grained control over message delivery: this message goes to these three queues, that message gets retried with exponential backoff, this other message expires after 60 seconds.

SQS is AWS's managed queue. It is simple, it is cheap, and it never goes down. Use SQS when you are on AWS and your requirements are straightforward — a producer, a queue, some consumers. SQS + Lambda is the fastest way to build an async processing pipeline on AWS, and it is the right choice for most Indian startups that do not need Kafka's replay capability.

The rule of thumb: if you need event sourcing, replay, or throughput above 10,000 messages per second, use Kafka. If you need complex routing, use RabbitMQ. If you are on AWS and want zero ops, use SQS.

When to use a message queue: any time a user request triggers work that the user does not need to see complete before getting a response. Image processing, email sending, report generation, data export, push notifications, fraud analysis. If the work can happen 30 seconds later, put it on a queue.

When not to use a message queue: when the work must complete before the response. Payment processing. Password changes. Anything where the user will refresh the page and panic if the result is not there.

---

CAP Theorem: The Trade-off You Cannot Escape

The CAP theorem states that a distributed system can provide at most two of three guarantees: Consistency, Availability, and Partition Tolerance.

The textbook definition: Consistency means every read receives the most recent write. Availability means every request receives a response (not an error). Partition Tolerance means the system continues to operate despite network partitions — messages dropped or delayed between nodes.

The real-world definition: when the network breaks between your database servers, you must choose between returning stale data (sacrificing consistency) and returning an error (sacrificing availability).

Here is the part textbooks get wrong: partition tolerance is not optional. Networks fail. Switches die. Cables get unplugged. In any distributed system, partitions will happen. The choice is not "CP or AP" — it is "when a partition happens, do you choose C or A?"

CP systems (Consistency + Partition Tolerance) choose to return errors rather than serve stale data. This is your bank's database. When the replica is unreachable, the system refuses writes rather than risk showing you a balance that is wrong. MongoDB in its default configuration is CP — the primary accepts writes, and if it cannot reach a majority of nodes, it steps down and refuses writes until a new primary is elected.

AP systems (Availability + Partition Tolerance) choose to serve potentially stale data rather than return errors. This is your Twitter feed. If a partition means you see a tweet 30 seconds late, nobody dies. Cassandra and DynamoDB are AP — they will accept writes and serve reads on any node that is reachable, and sort out consistency later.

The "CA" option does not exist in practice. A system that is both consistent and available during normal operation will face a partition eventually, and at that moment it must choose. There is no CA distributed database.

The practical takeaway: know what your system needs before you pick a database. If you are building a payment system, you need CP. If you are building a social feed, you need AP. If you try to have both, you will have neither when the network fails — and the network will fail.

API Design: The Interface Your System Deserves

Your API is a contract. It is a promise to every frontend engineer, mobile developer, and third-party integrator that this endpoint, with these parameters, will return this shape of data. Break the contract, and you break every client that depends on it.

REST is the default. It maps HTTP methods to CRUD operations, uses resource-oriented URLs, and leans on HTTP status codes for semantics. It is not exciting. It is reliable.

GET    /api/users          → List users
GET    /api/users/42       → Get user 42
POST   /api/users          → Create a user
PUT    /api/users/42       → Replace user 42
PATCH  /api/users/42       → Update user 42 partially
DELETE /api/users/42       → Delete user 42

REST's strength is its predictability. Any engineer who knows HTTP can understand your API in minutes. Its weakness is over-fetching and under-fetching. The /api/users/42 endpoint returns every field — name, email, address, preferences, last 50 orders — when the mobile app only needed the name and avatar. The /api/users endpoint returns a list of IDs and names, but the dashboard needs email and signup date too. You end up building custom endpoints for every client, and your API becomes a mess of one-offs.

GraphQL solves the over-fetching problem by letting the client specify exactly what it needs.

query {
  user(id: "42") {
    name
    avatar
    orders(limit: 5) {
      total
      status
    }
  }
}

The client gets exactly the fields it asked for. No more, no less. The backend resolves each field independently, often from different data sources — the user from PostgreSQL, the orders from a microservice, the avatar from S3.

GraphQL's strength is flexibility. Its weakness is complexity. The N+1 query problem — where resolving each order triggers a separate database query — is the classic GraphQL footgun. The DataLoader pattern solves it by batching and caching requests, but it is one more thing to configure and debug.

// Without DataLoader: N+1 queries
const resolvers = {
  User: {
    orders: (user) => db.query('SELECT * FROM orders WHERE user_id = $1', [user.id])
  }
};
// 1 query for users + N queries for orders = N+1

// With DataLoader: 2 queries total
const orderLoader = new DataLoader(async (userIds) => {
  const orders = await db.query(
    'SELECT * FROM orders WHERE user_id = ANY($1)', [userIds]
  );
  return userIds.map(id => orders.filter(o => o.user_id === id));
});

const resolvers = {
  User: {
    orders: (user) => orderLoader.load(user.id)
  }
};

gRPC is Google's RPC framework built on Protocol Buffers and HTTP/2. It is fast — binary serialization, multiplexed streams, header compression. It is strongly typed — your .proto file is the single source of truth for both client and server. It is the right choice for service-to-service communication in a microservices architecture.

service UserService {
  rpc GetUser (GetUserRequest) returns (User);
  rpc ListUsers (ListUsersRequest) returns (stream User); // server-side streaming
}

message GetUserRequest {
  string user_id = 1;
}

message User {
  string id = 1;
  string name = 2;
  string email = 3;
}

gRPC's strength is performance and type safety. Its weakness is that it is not browser-friendly — you need a gRPC-web proxy, which adds complexity. Use gRPC between your own services. Use REST or GraphQL for your public API.

The decision framework: REST for public APIs and simple CRUD services. GraphQL when your clients have diverse data needs and you want to avoid versioning hell. gRPC for internal service-to-service communication where latency matters. Do not pick one for everything. The right tool depends on the job.

Back-of-the-Envelope Estimation: The Skill That Wins Interviews

Every system design interview includes a moment where the interviewer asks: "How many servers do you need?"

They are not looking for an exact number. They are looking for your ability to reason about scale with rough numbers. This is back-of-the-envelope estimation, and it is the most under-practiced skill in system design.

The framework is simple. For any system, estimate:

1. QPS (Queries Per Second): How many requests hit your system?
2. Storage: How much data do you store, and for how long?
3. Bandwidth: How much data moves in and out per second?
4. Memory: How much RAM do you need for caching?

Let us work through a real example: designing a URL shortener like bit.ly for the Indian market.

Step 1: QPS. Assume 100 million new URLs created per month. That is roughly 40 URLs per second — write QPS. Assume a 100:1 read-to-write ratio (people create URLs once, but they are clicked many times). Read QPS = 40 × 100 = 4,000.

Step 2: Storage. Each shortened URL record: short code (7 chars), original URL (avg 100 chars), user ID (UUID, 36 chars), created_at (timestamp, 8 bytes), expiry (optional). Roughly 500 bytes per record. 100 million records per month × 500 bytes = 50 GB per month. Over 5 years: 50 GB × 60 months = 3 TB.

Step 3: Bandwidth. Write bandwidth: 40 writes/s × 500 bytes = 20 KB/s. Read bandwidth: 4,000 reads/s × 500 bytes = 2 MB/s. Total: roughly 2 MB/s. This is trivial — a single EC2 instance can handle this.

Step 4: Memory (cache). Cache the top 20% of URLs that get 80% of traffic. 20% of 100 million URLs per month, over 5 years: 20% × 6 billion = 1.2 billion URLs. But you only need to cache the hot ones — say, the last month's worth. 20% × 100 million = 20 million URLs × 500 bytes = 10 GB of cache. A single Redis instance with 16 GB RAM handles this comfortably.

The numbers tell you the architecture: this is not a hard problem. A single PostgreSQL instance (with read replicas for the 4,000 QPS reads), a single Redis instance, and a handful of Node.js servers behind a load balancer. You do not need Kafka. You do not need Cassandra. You do not need microservices.

This is the power of estimation. It prevents you from over-engineering a solution to a problem that does not need it. It also prevents you from under-engineering — if the numbers had come out to 100,000 QPS and 50 TB per month, you would know you need sharding, a CDN, and a message queue.

Practice this. For every system you use — WhatsApp, Swiggy, Zerodha, UPI — estimate the numbers. How many messages per second does WhatsApp handle in India? (Answer: roughly 1 million messages per second, given 500 million Indian users sending an average of 50 messages per day.) How many orders per minute does Swiggy process at peak? (Answer: roughly 5,000 orders per minute, given 7 million daily orders and a 3x peak-to-average ratio.) Build the muscle. It will serve you in every interview and every architecture decision.

Microservices: When to Split and When to Stay Together

The most expensive architectural decision you will ever make is whether to build a monolith or microservices. Get it wrong, and you will spend years paying down the complexity debt.

The industry lied to you about microservices. For a decade, conference talks and blog posts told you that monoliths are legacy and microservices are modern. They showed you Netflix's architecture diagram with hundreds of services and said: "This is the goal." They did not show you the 300-person platform team Netflix needed to make that diagram work.

Start with a monolith. Not because microservices are bad. Because you do not yet know where your boundaries are. A monolith lets you move fast while you are still discovering your domain. You can refactor a function call in an afternoon. Refactoring a service boundary takes a quarter.

The Node.js ecosystem makes monoliths natural. A well-structured Express or Fastify application with clear module boundaries — users/, orders/, payments/, notifications/ — is a monolith that is ready to split when the time comes. The key is discipline: modules must not import from each other's internals. If orders/ imports from payments/database.ts instead of calling payments/service.ts, you have already lost the ability to split.

Split when you have a reason. Not "because it's the right way." Not "because Netflix does it." Real reasons:

1. Independent scaling. Your payment service handles 10 QPS. Your feed service handles 10,000 QPS. If they are in the same process, you are running 100 instances of the payment service you do not need. Split them, and you run 2 instances of payments and 100 of feed.

2. Independent deployment. Your team ships twice a day. The payments team ships twice a month. If every deploy requires coordinating with payments, you slow down. Split the services, and each team deploys on their own schedule.

3. Different data models. Your feed needs Cassandra for high writes. Your payments need PostgreSQL for ACID. You can run both databases from a monolith, but at some point the cognitive load of managing two data models in one codebase exceeds the cost of splitting.

4. Team scale. One team of five engineers can manage a monolith. Three teams of five engineers each, working on different features, will step on each other's toes constantly. This is Conway's Law: your architecture will mirror your communication structure. If you have three teams, you will end up with at least three services.

The cost of microservices that nobody talks about:

• Network latency. A function call is nanoseconds. A service call over HTTP is milliseconds. Chain five service calls, and your p99 latency is 50ms before you have done any real work.
• Distributed transactions. In a monolith, a database transaction ensures consistency. In microservices, you need sagas — compensating transactions that undo work when something fails. Sagas are hard to get right and harder to test.
• Debugging. In a monolith, a stack trace tells you what went wrong. In microservices, a failed request might have touched five services, each with its own logs, its own trace ID format, and its own timezone. You need distributed tracing (OpenTelemetry, Jaeger) just to understand what happened.
• Operational complexity. Each service needs its own CI/CD pipeline, its own monitoring, its own alerts, its own on-call rotation. Ten services means ten times the operational surface area.

The pragmatic approach: build a modular monolith first. Enforce module boundaries at the code level. When a module genuinely needs independent scaling, deployment, or a different database, extract it. You will extract 3-5 services, not 50. And those 3-5 services will have clear boundaries because you discovered them through use, not speculation.

This is how Stripe, Shopify, and GitHub scaled — not by starting with microservices, but by splitting their monoliths surgically, one service at a time, when the data proved it was necessary.

The Story of Rohan

Rohan was a senior engineer at a Bangalore fintech startup. Four years of experience. Strong Node.js. Comfortable with React. He had interviewed at three product companies for a Staff Engineer role and been rejected at all three — each time at the system design round.

His problem was not that he did not know the concepts. He knew what a load balancer was. He knew what caching was. His problem was that he could not assemble them into a system. He would freeze when the interviewer said "design a payment gateway" because he had never done it, and he did not have a framework for reasoning about something he had never built.

We worked together for six weeks. We did not study more concepts. We practiced assembly. Every session, I gave him a system to design — a chat app, a ride-hailing service, a stock trading platform — and he had 45 minutes to produce an architecture. The first three were disasters. The fourth was passable. By the tenth, he could walk through any system design problem with confidence.

The framework he learned:

1. Clarify requirements — functional and non-functional. How many users? What latency? What consistency?
2. Estimate scale — QPS, storage, bandwidth. Let the numbers drive the architecture.
3. Design the data model — what entities, what relationships, what database?
4. Design the API — what endpoints, what request/response shapes?
5. Design the high-level architecture — draw boxes: clients, load balancers, services, databases, caches, queues
6. Deep-dive on the critical path — the one flow that must work perfectly
7. Identify bottlenecks and trade-offs — what breaks first, and what do you do about it?

Rohan got the offer. Not because he learned more facts, but because he learned to assemble the facts he already had.

---

Now that you have the building blocks — scaling, caching, databases, message queues, the CAP theorem, API design, and estimation — can you assemble them into WhatsApp?