In April 2012, Facebook agreed to buy Instagram for about $1 billion. At that time Instagram had around 30 million users and only 13 employees. A few months earlier, in December 2011, Instagram's engineering blog said the app was serving 14 million+ users with just 3 engineers.

How does a team that small keep an app that big running? A large part of the answer is one of the most basic ideas in system design: scaling out instead of only scaling up, and doing it in the right places.

In this post I will explain vertical and horizontal scaling in simple English, show how Instagram used them, and give you a clean answer to the interview question: "How do you scale a stateful service versus a stateless one?" I have spent 14+ years in cloud, DevOps and SRE, and this question still comes up in almost every system design round I see.

1. What does "scaling" mean?

Scaling means handling more work (more users, more requests, more data) without the system getting slow or falling over.

There are two basic ways to do it:

• Vertical scaling (scale up): make one machine bigger.

• Horizontal scaling (scale out): add more machines.

Let's look at each.

2. Vertical scaling: a bigger machine

Vertical scaling means giving your server more CPU, more memory, faster disks or a faster network. On AWS, that is changing an instance from a small type to a larger one.

Why people like it:

• Simple. No code changes. The app does not need to know anything changed.

• Good for stateful systems. A single database on a bigger machine is still a single database: one place for data, easy transactions.

• Quick fix. When a server is overloaded today, moving to a bigger size can buy you time.

The problems:

• There is a ceiling. The biggest machine you can buy is still one machine. At some point there is no bigger size.

• Cost grows fast. The largest machines are expensive, and you pay for that size even at quiet hours.

• Single point of failure. If that one big machine fails, everything on it fails.

• Downtime to resize. Changing instance size often means a restart.

Vertical scaling is not bad. It is often the right first step, especially for databases. It just cannot be your only plan.

3. Horizontal scaling: more machines

Horizontal scaling means running many copies of your service on many machines, and putting a load balancer in front to spread requests across them.

Why it is powerful:

• No hard ceiling. Need more capacity? Add more machines.

• Better availability. If one machine dies, the others keep serving. The load balancer stops sending traffic to the dead one.

• Pay for what you use. With auto scaling (for example AWS Auto Scaling groups or Kubernetes Horizontal Pod Autoscaler), you add machines at peak and remove them at night.

The catch: horizontal scaling only works smoothly if any copy can handle any request. That leads to the most important word in this post: stateless.

4. Stateless vs stateful: the key idea

A stateless server keeps no important data in its own memory or disk between requests. Everything it needs comes from the request itself, or from shared systems like a database or a cache.

A stateful server keeps important data locally. Examples: a database server, or an app server that keeps your login session in its own memory.

Why does this matter?

• If app servers are stateless, request 1 can go to server A and request 2 to server C, and nothing breaks. You can add or remove servers at any time.

• If app servers are stateful (for example sessions stored in local memory), then a user must keep going to the same server ("sticky sessions"). If that server dies, the user loses their session. Scaling becomes painful.

The usual fix: move the state out. Store sessions in Redis or a database, files in object storage like S3, and keep the app servers stateless.

Then you have two different scaling problems:

1. Stateless tier (app servers): easy. Scale out horizontally behind a load balancer.

2. Stateful tier (databases): hard. Scale up first, add read replicas, and when that is not enough, shard.

5. How Instagram did it

Instagram's December 2011 engineering post ("What Powers Instagram") described their stack. Their core rules were to keep things very simple, not reinvent the wheel, and use proven technologies. Here is the shape of it, simplified:

The front door: load balancing

Every request first hit load balancers. Instagram used to run two NGINX machines with DNS round robin between them. The problem was that DNS changes take time to spread when a machine had to be removed. So they moved to Amazon's Elastic Load Balancer (ELB), with three NGINX instances behind it. NGINX machines that failed a health check were taken out of rotation automatically. They also terminated SSL at the ELB to save CPU on NGINX, and used Route 53 for DNS.

The app tier: stateless Django servers

Behind that ran Django application servers on AWS EC2 (High-CPU Extra-Large instances at the time). As usage grew, they went from a few app servers to more than 25. The post says this tier was easy to scale horizontally because the servers were stateless. That is the exact pattern from section 4.

The data tier: PostgreSQL, then sharding

Most of Instagram's data (users, photo metadata, tags and so on) lived in PostgreSQL. One database server could not hold it all, so they sharded it.

Their "Sharding & IDs at Instagram" post explains how:

• They created several thousand "logical" shards, mapped in code to far fewer physical database servers.

• Each logical shard was a PostgreSQL schema (a named group of tables inside one database).

• To grow, they could move logical shards from one physical server to another, without re-bucketing their data.

• They generated unique 64-bit IDs inside PostgreSQL itself: time-based bits, the shard ID, and a sequence number. That way IDs were unique across all shards and roughly sorted by time.

They also used streaming replication (replicas in another availability zone) and PgBouncer to pool database connections.

So Instagram used both kinds of scaling:

• Horizontal for the stateless app servers: just add more.

• Vertical, then sharded for the stateful database: big memory-heavy instances, and data split across many of them.

Lesson for us: Instagram did not invent new technology. It used boring, proven parts (load balancers, stateless app servers, PostgreSQL) and scaled each tier in the way that fits that tier.

6. What is sharding, simply?

Sharding means splitting one big dataset into smaller pieces called shards, and putting them on different database servers. Each shard holds a subset of the data, for example all data for users whose ID falls in a certain bucket.

user_id % number_of_logical_shards  → logical shard (schema)
logical shard                       → physical database server

Benefits: each server handles less data and fewer queries, so you can grow almost without limit.

Costs:

• Queries across shards are harder (for example "top photos across all users").

• Transactions across shards are hard.

• You must choose a good shard key. A bad key creates "hot" shards that get most of the traffic.

• Moving data between shards needs planning. Instagram's logical-shard trick exists exactly to make this easier.

That is why the interview line is "shard stateful parts carefully". Sharding is powerful, but it is a one-way door for many designs.

7. The example visuals

The "4 CPU vs 64 CPU" servers and the "App 1–4" and "Shard 1–4" boxes in the Reel are examples to show the idea. They are not Instagram's real instance sizes or counts. The real numbers from the 2011 post were 25+ Django app servers and a main shard cluster of 12 large PostgreSQL instances (plus 12 replicas in another zone).

8. A quick comparison table

9. How to answer this in an interview

Question: "How do you scale a stateful vs a stateless service?"

Strong answer:

"For stateless services, I scale horizontally: run many copies behind a load balancer with health checks, and use auto scaling. To make that work I keep state out of the app servers, with sessions in Redis or a database and files in S3. For stateful services like databases, I scale up first, add read replicas for read traffic, add caching, and when one primary isn't enough, I shard by a good key. Instagram is a classic example: stateless Django app servers that were easy to scale out, behind an ELB, and PostgreSQL sharded into thousands of logical shards on fewer physical servers. That let a very small team run a very large app."

Follow-ups to expect:

• What are sticky sessions and why avoid them? They tie a user to one server, which breaks even load spreading and loses sessions when that server dies.

• Read replicas vs sharding? Replicas copy all the data to scale reads. Shards split the data to scale reads and writes.

• How do you pick a shard key? High cardinality, even spread, and matching your main access pattern (for Instagram, mostly by user).

• When is vertical scaling the right answer? When it's cheap enough, the ceiling is far away, and simplicity matters more than unlimited growth.

Sources

• Los Angeles Times, "Facebook to snap up Instagram photo-sharing app for $1 billion" (10 Apr 2012): 13 employees, about 30 million users: https://www.latimes.com/business/la-xpm-2012-apr-10-la-fi-facebook-instagram-20120410-story.html

• The Atlantic, "Instagram Is Now Worth $77 Million Per Employee" (Apr 2012): 13 employees, 30M+ users: https://www.theatlantic.com/business/archive/2012/04/instagram-is-now-worth-77-million-per-employee/255640/

• Instagram Engineering, "What Powers Instagram: Hundreds of Instances, Dozens of Technologies" (Dec 2011): 14M+ users, 3 engineers, ELB + NGINX, stateless Django app servers, PostgreSQL: https://instagram-engineering.com/what-powers-instagram-hundreds-of-instances-dozens-of-technologies-adf2e22da2ad

• Instagram Engineering, "Sharding & IDs at Instagram": thousands of logical shards as PostgreSQL schemas, 64-bit IDs: https://instagram-engineering.com/sharding-ids-at-instagram-1cf5a71e5a5c

• High Scalability summary of the Instagram architecture (2011): https://highscalability.com/instagram-architecture-14-million-users-terabytes-of-photos/

Preparing for a Cloud, DevOps or SRE interview? Book a Free 15-min Cloud & DevOps Interview Guidance Call with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 https://topmate.io/cloudtechbydevang/2358399

For about ten years, the loudest advice in software was: "Break your monolith into microservices." Then, in March 2023, a team inside Amazon, the company most famous for microservices, published a blog post saying they had moved one service back into a single process and cut its infrastructure cost by over 90%.

The internet went wild. "Microservices are dead!" "Monoliths are back!" Both reactions were wrong.

In this post I will explain, in simple English, what a monolith is, what microservices are, why Amazon split its monolith in the early 2000s, and what the Prime Video team actually did in 2023. Then I'll give you a clear answer for the interview question: "When would you choose a monolith over microservices?" I have spent 14+ years in cloud, DevOps and SRE, and I have seen teams win and lose with both styles. The style is never the point. Fit is.

1. First, the basics

What is a monolith?

A monolith is one application, built and deployed as one unit. All the features live in one codebase: login, product pages, cart, payments, emails. You build it once and deploy it once. Usually it talks to one main database.

Good things about a monolith:

• Simple to start. One repo, one build, one deploy.

• Fast calls. One part of the code calls another as a normal function call, in memory. No network.

• Easy to debug. One process, one log stream, one stack trace.

• Easy transactions. One database means one place to keep data consistent.

Problems as it grows:

• Everyone shares everything. A small change by one team needs a full redeploy of the whole app.

• Scaling is all-or-nothing. If only search is busy, you still scale the whole app.

• One bug can take everything down. A memory leak in one feature can crash the whole process.

• Unclear ownership. When 200 engineers touch one codebase, nobody owns anything fully.

What are microservices?

Microservices split the application into small, separate services. Each service does one job (for example orders, search, payments), owns its own data, and talks to others over the network through an API. Each service is owned by one team and can be deployed on its own.

Good things:

• Independent deploys. The payments team can ship without waiting for the search team.

• Independent scaling. Scale only the busy service.

• Smaller blast radius. If recommendations fail, checkout can still work.

• Clear ownership. One team, one service, end to end.

Costs:

• Network calls everywhere. Calls are slower and can fail. You need timeouts, retries and circuit breakers.

• Harder debugging. One user request may touch ten services. You need distributed tracing.

• Data is split. No single database transaction across services.

• More infrastructure. More pipelines, more monitoring, more things to secure and pay for.

Microservices are a trade. You pay in complexity to buy team independence and independent scaling.

2. Why Amazon split its monolith

In a 2006 interview with ACM Queue, Amazon's CTO Werner Vogels described how Amazon.com started as a monolithic application: a web server talking to a database on the back end. That application held all the business logic and display logic.

He explained that by 2001 it was clear the front-end application could not scale any more. The problem was not only performance. Many complex pieces were combined into one system, so it could not evolve. Parts that needed to scale on their own were tied to shared resources, and there was no isolation and no clear ownership. The shared databases were also hard to scale.

So Amazon moved to a service-oriented architecture. In Amazon's version, each service owns its data together with the business logic, and the only way in is through a published service interface. No other team may read the service's database directly. Over time this grew into hundreds of services. In that interview Vogels said that a single Amazon.com page could call more than 100 services to build it.

The team side mattered just as much as the technology. AWS describes Amazon's two-pizza teams: a team small enough to be fed by two pizzas, ideally fewer than 10 people, with single-threaded ownership of one service. Vogels' famous line from the same interview sums up the culture: "You build it, you run it." The team that writes the service also operates it.

Lesson for us: Amazon did not split its monolith because microservices were fashionable. (Vogels even said they did it before "service-oriented" was a buzzword.) They split because the monolith was blocking team speed and scaling, and they had the scale to justify the cost.

3. Prime Video: when distributed was the wrong choice

Now the 2023 story. It is about one specific tool, not all of Prime Video.

Prime Video's Video Quality Analysis (VQA) team owns a tool that checks video streams for quality problems, for example corrupted frames or audio and video going out of sync. They needed it to monitor thousands of concurrent streams, and to grow beyond that.

The original design

The first version was a distributed, serverless design:

• AWS Step Functions orchestrated the workflow.

• AWS Lambda functions ran the detectors.

• Amazon S3 stored video frames between the steps, so each component could pick them up.

On paper this looks perfect: serverless, independently scalable pieces.

What went wrong

According to the Prime Video Tech blog (Marcin Kolny, 22 March 2023):

• The design hit a hard scaling limit at around 5% of the expected load.

• Orchestration was the main bottleneck. The service did several state transitions for every second of every stream, so it quickly hit account limits. And Step Functions charges per state transition.

• Passing data between components was expensive. Every frame went into S3 and was read back out, which cost a lot at this volume.

The fix

The team moved all the components into a single process. Frames now moved in memory instead of through S3, and the orchestration logic ran inside the same instance. They deployed it on Amazon EC2 and Amazon ECS, and scaled by running more copies of the whole process.

The result, in their words: moving to a monolith cut infrastructure cost by over 90% and increased their ability to scale. They could handle thousands of streams with room to grow, and could also use EC2 compute savings plans to lower costs further.

What it does NOT mean

• It does not mean Amazon or Prime Video dropped microservices. This was one service.

• Some engineers argued the new design is really "one bigger microservice". It still runs as a separate service, just not split into tiny serverless steps. That is a fair point. The real lesson is about where you draw the boundaries.

• It does not mean serverless is bad. Serverless is great for spiky, event-driven work. It was a poor fit here because the workload moved huge amounts of data between steps at high, steady volume.

Lesson for us: if two parts of your system exchange large amounts of data all the time, putting a network (or S3) between them is expensive. Keep tightly connected, data-heavy steps together.

4. Putting the two stories side by side

Both teams made the right call for their problem. That's the whole point.

5. A simple decision guide

These are rules of thumb from my experience, not laws.

Start with a monolith (or a "modular monolith") when:

• You are a small team (one or two teams).

• The product is new and the boundaries are not clear yet.

• Parts of the system share a lot of data and change together.

• You want to move fast with few operational costs.

A modular monolith is a good middle path: one deployable app, but with clean internal modules and clear interfaces between them. If a module later needs to scale or move on its own, you can pull it out into a service.

Move to microservices when:

• Many teams are blocked by each other's releases.

• Different parts have very different scaling needs (for example search vs. account settings).

• You need a smaller blast radius for critical paths like checkout.

• You have the platform maturity: CI/CD, monitoring, tracing, on-call.

Warning signs you split too much:

• One user action needs a long chain of synchronous calls.

• Services always deploy together ("distributed monolith").

• Huge data moves between services on every request.

• Your cloud bill is dominated by data transfer and orchestration, not real work.

6. The example visuals

The service names in the Reel (Orders, Search, Payments, Catalog) are examples to show the idea of splitting an app. They are not an official list of Amazon's services. The Prime Video before/after diagram is a simplified version of the architecture the team described.

7. How to answer this in an interview

Question: "When would you choose a monolith over microservices?"

Strong answer:

"I'd start with a well-structured monolith when the team is small, the domain isn't clear yet, or the parts share a lot of data, because it's simpler, cheaper and faster to change. I'd split out services when teams are blocking each other or parts need to scale differently. Amazon split its monolith in the early 2000s for exactly those reasons, with two-pizza teams owning services. But the split has costs: network calls, data transfer, and operational overhead. Prime Video's video quality team showed that in 2023. Their Step Functions + Lambda + S3 design hit a limit at about 5% of the expected load, and putting it into one process on ECS cut infrastructure cost by over 90%. So I split when teams and scale need it, not by default."

Follow-ups to expect:

• What is a distributed monolith? Services that are split physically but still must deploy together. You get the costs of both styles and the benefits of neither.

• How do you split a monolith safely? The strangler fig pattern: put a router in front, move one feature at a time to a new service, and retire the old code path.

• How do services share data? Through APIs or events, not by reading each other's databases.

• How do you handle a transaction across services? Sagas: a series of local transactions with compensating actions if something fails.

Sources

• Marcin Kolny, Prime Video Tech, "Scaling up the Prime Video audio/video monitoring service and reducing costs by 90%" (22 Mar 2023). Archived copy: https://archive.is/h788m

• The New Stack, "Return of the Monolith: Amazon Dumps Microservices for Video Monitoring" (4 May 2023): https://thenewstack.io/return-of-the-monolith-amazon-dumps-microservices-for-video-monitoring/

• The Stack, "Prime Video service dumps microservices, cuts AWS bill 90%" (May 2023): https://www.thestack.technology/amazon-prime-video-microservices-monolith/

• ACM Queue, "A Conversation with Werner Vogels" (2006): monolith until 2001, service-oriented architecture, 100+ services per page, "you build it, you run it": https://queue.acm.org/detail.cfm?id=1142065

• AWS Executive Insights, "Powering Innovation and Speed with Amazon's Two-Pizza Teams": https://aws.amazon.com/executive-insights/content/amazon-two-pizza-team/

Preparing for a Cloud, DevOps or SRE interview? Book a Free 15-min Cloud & DevOps Interview Guidance Call with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 https://topmate.io/cloudtechbydevang/2358399

You type google.com into your browser and press Enter. Less than a second later, the page is on your screen. It feels like nothing happened. In fact, a small relay race just ran across the internet: your laptop, your router, your internet provider, a DNS resolver, several name servers, and a Google server near you all took part.

"What happens when you type a URL and press Enter?" is one of the most common warm-up questions in cloud, DevOps and backend interviews. It looks simple. But it tests whether you really understand networking, security and caching, the three things that sit under every system you will ever build.

In this post I will walk through the whole journey in simple English, step by step. At the end I will give you a short answer you can say in an interview. I have spent 14+ years in cloud, DevOps and SRE, and almost every production incident I have debugged touched one of these steps: a stale DNS record, an expired certificate, a load balancer health check, or a missing cache.

The big picture first

Here is the whole flow in one line:

DNS → TCP → TLS → HTTP → render, with caching at every step.

1. DNS turns the name google.com into an IP address.

2. TCP opens a reliable connection to that IP address.

3. TLS makes the connection private and proves you are talking to the real Google.

4. HTTP sends your request and brings back the page.

5. Your browser draws the page, and fetches more files it needs.

Modern browsers can also use HTTP/3, which runs over QUIC on UDP instead of TCP. QUIC combines the connection and encryption steps. I will cover that near the end. First, let's take the classic path, step by step.

1. DNS: finding the address

Computers do not talk to names. They talk to IP addresses, such as 142.250.x.x (IPv4) or a longer IPv6 address. DNS (Domain Name System) is the internet's phone book. It turns a name into an address.

Step 1a: check the caches close to you

Before asking anyone on the internet, your computer checks what it already knows:

• Browser cache. Browsers keep recent DNS answers for a short time. (In Chrome you can see this at chrome://net-internals/#dns.)

• Operating system cache. If the browser does not have it, it asks the OS. The part of the OS that does this is called the stub resolver.

Every DNS answer comes with a TTL (time to live). The TTL says how many seconds you may keep that answer before you must ask again. If a cached answer is still within its TTL, the lookup ends right here. That is why the second visit to a site is often faster than the first.

Step 1b: ask the recursive resolver

If there is no cached answer, your OS sends the question to a recursive resolver. This is usually run by your internet provider, or a public resolver like Google Public DNS (8.8.8.8) or Cloudflare (1.1.1.1).

The resolver's job is to find the answer for you. It also has a big cache. Many people use the same resolver, so popular names like google.com are usually already cached there.

Step 1c: the full walk (when nothing is cached)

If the resolver has nothing cached, it walks down the DNS tree:

1. It asks a root name server: "Who handles .com?" The root replies with the addresses of the .com servers.

2. It asks a .com TLD (top-level domain) name server: "Who handles google.com?" The TLD server replies with Google's own name servers.

3. It asks Google's authoritative name server: "What is the IP address of google.com?" This server holds the real record and gives the final answer.

4. The resolver sends the IP address back to your computer, and caches it for next time.

In real life, most lookups skip some of these steps because of caching. Resolvers keep the .com server list and Google's name server list in cache too, so the full walk is rare for popular sites.

One more thing worth knowing: big companies like Google may return different IP addresses to different users, depending on where the user is. This is one way they send you to a nearby server. (I cover this, and anycast, in topic 006 of this series.)

Lesson for us: DNS is a cache-heavy system. When you change a DNS record in production, the old answer can live on in caches until its TTL runs out. Lower the TTL before a planned migration, not during it.

2. TCP: opening a reliable connection

Now your browser knows the IP address. Next, it needs a reliable connection to it. The classic way is TCP (Transmission Control Protocol), on port 443 for HTTPS.

TCP gives you three promises: data arrives in order, without gaps, and without duplicates. To set this up, both sides first agree on starting sequence numbers. They do this with the famous three-way handshake:

1. SYN. Your computer says: "I want to talk. Here is my starting number."

2. SYN-ACK. The server says: "OK. I got your number. Here is mine."

3. ACK. Your computer says: "Got yours. We are connected."

This costs one round trip before any useful data can flow. A round trip (RTT) is the time for a message to go to the server and back. If the server is 100 ms away, the handshake alone costs about 100 ms. That is why physical distance matters so much, and why big companies put servers close to users.

Lesson for us: reuse connections. Keep-alive connections, connection pooling in your apps, and HTTP/2 multiplexing all exist so you don't pay the handshake cost again and again.

3. TLS: making it private and proving who you are talking to

The S in HTTPS means the connection is protected by TLS (Transport Layer Security). TLS does two jobs:

1. Encryption. Nobody in the middle (your Wi-Fi, your ISP, anyone else) can read or change the data.

2. Authentication. The server shows a certificate, signed by a trusted certificate authority. Your browser checks that the certificate is valid and really belongs to google.com.

With TLS 1.3 (RFC 8446, published in 2018), a full handshake takes just one round trip (often called 1-RTT). Older TLS 1.2 handshakes needed two round trips. So TLS 1.3 made every new HTTPS connection faster.

In simple terms, the TLS 1.3 handshake goes like this:

• Your browser sends a "hello" with the encryption options it supports and its share of a key exchange.

• The server replies with its choices, its key share, its certificate and a proof that it owns the certificate.

• Both sides now compute the same secret keys, and encrypted data can flow.

TLS 1.3 also has a 0-RTT mode for repeat visits, where the browser can send some data in its very first message. It is faster, but that early data can be replayed by an attacker, so servers only allow it for safe requests.

Lesson for us: a huge number of real outages are simply expired certificates. Automate renewal (for example with ACM on AWS, or cert-manager on Kubernetes) and alert well before expiry.

4. The request reaches Google: the Google Front End

Now your encrypted connection is open. Who is on the other end? Not the server that builds your search page. You are talking to Google's edge.

Google's own security design paper explains that when a service needs to be on the internet, it registers with an infrastructure service called the Google Front End (GFE). The GFE terminates TLS with the correct certificates and best practices, and it adds protection against denial-of-service attacks. Then it forwards your request to the right backend service inside Google's network.

This pattern is not special to Google. Almost every large website works the same way:

• A front end / edge layer (load balancer, reverse proxy, CDN) that faces the internet.

• Backend services that do the real work, hidden behind it.

On AWS, you would build the same shape with CloudFront and an Application Load Balancer in front of your services. On Google Cloud, the external Application Load Balancer runs on the same front-end idea.

Lesson for us: never expose your app servers directly to the internet. Put a load balancer or reverse proxy in front. It terminates TLS, absorbs attacks, and lets you add or remove servers without users noticing.

5. HTTP: the actual request and response

Inside the encrypted connection, your browser sends an HTTP request:

GET / HTTP/2
Host: www.google.com
User-Agent: ...
Accept: text/html
Cookie: ...

The server sends back an HTTP response: a status code (for example 200 OK, or a 301 redirect from google.com to www.google.com), some headers, and the HTML of the page.

There are three main versions of HTTP in use today:

HTTP/3 is defined in RFC 9114. It uses the QUIC transport, which handles reliability, streams and flow control itself, and has TLS 1.3 built in. That is why with QUIC the connection setup and the encryption setup happen together, saving a round trip compared with TCP + TLS.

Google created the early version of QUIC, and Google's services widely support HTTP/3. Your browser and the server agree on which version to use. Older networks or browsers fall back to HTTP/2 or HTTP/1.1 over TCP.

6. The browser draws the page

The HTML is only the start. The browser reads it and finds more things it needs: CSS, JavaScript, images, fonts. It fetches them too, often from the same connection (thanks to HTTP/2 or HTTP/3) or from CDN servers. Then it builds the page structure (the DOM), applies styles, runs scripts, and paints pixels on your screen.

This part is mostly front-end engineering, but it matters for cloud people too: a slow page is often not a slow server, but too many files, no compression, or no caching headers.

7. Caching at every step

If you remember one thing from this post, remember this. Every step has a cache, and caching is why the web feels fast.

The first visit pays the full price. The second visit is cheaper at almost every layer.

8. A simple diagram

The diagrams in the Reel and here are simplified examples for teaching. Real flows have more steps (for example CNAME records, redirects and CDN layers).

You type google.com + Enter
   │
   ▼
[Browser cache] → [OS cache] → [Recursive resolver]
                                   │ (if not cached)
                                   ├─► Root server  ("ask .com")
                                   ├─► .com server  ("ask Google's name servers")
                                   └─► Google name server ("here is the IP")
   │
   ▼
TCP handshake (SYN, SYN-ACK, ACK)  ── 1 round trip
   │
   ▼
TLS 1.3 handshake                   ── 1 round trip
   │
   ▼
Google Front End (near you) → backend service → HTML
   │
   ▼
Browser renders the page, fetches CSS/JS/images

9. How to answer this in an interview

Short answer (30 seconds):

"First, DNS turns google.com into an IP address. The browser and OS check their caches, then a recursive resolver asks the root, the .com servers and Google's name servers. Then the browser opens a TCP connection with a three-way handshake, and TLS 1.3 sets up encryption in one round trip. The request reaches a Google Front End near me, which terminates TLS and forwards it to a backend. The response comes back over HTTP/2 or HTTP/3, which runs on QUIC, and the browser renders the page. And there's caching at every step: DNS, connections, CDN and the backend."

Follow-up questions to expect:

• What is a TTL and why does it matter? How long a DNS answer can be cached. It controls how fast DNS changes spread.

• Why is TLS 1.3 faster than TLS 1.2? One round trip for a full handshake instead of two.

• Why does HTTP/3 use UDP? QUIC builds its own reliability on UDP, so one lost packet only blocks its own stream, and it combines the transport and TLS handshakes.

• Where would you put a load balancer? At the edge, terminating TLS, in front of stateless app servers.

• How would you debug a slow page? Check DNS time, connect time, TLS time and time to first byte separately (for example with curl -w or browser dev tools).

A senior touch: mention failure points. DNS misconfiguration, an expired certificate, an unhealthy backend behind the load balancer, or a cold cache after a deploy. Interviewers love when you connect the theory to real incidents.

10. Try it yourself

You can see each step on your own machine:

dig google.com                 # DNS answer and TTL
dig +trace google.com          # the full root → .com → Google walk
curl -so /dev/null -w "dns:%{time_namelookup} connect:%{time_connect} tls:%{time_appconnect} ttfb:%{time_starttransfer}\n" https://www.google.com

Run the curl command twice and compare. The difference is caching at work.

Sources

• Cloudflare Learning Center, "What is DNS?" (resolution steps, browser/OS caching, TTL): https://www.cloudflare.com/learning/dns/what-is-dns/

• RFC 9293, Transmission Control Protocol (TCP), section 3.5 "Establishing a Connection" (three-way handshake): https://www.rfc-editor.org/rfc/rfc9293

• RFC 8446, The Transport Layer Security (TLS) Protocol Version 1.3 (1-RTT handshake, 0-RTT data and replay): https://www.rfc-editor.org/rfc/rfc8446

• Google Cloud, "Google infrastructure security design overview" (Google Front End terminates TLS and fronts internet-facing services): https://cloud.google.com/docs/security/infrastructure/design

• RFC 9114, HTTP/3 (HTTP over the QUIC transport): https://www.rfc-editor.org/rfc/rfc9114

• Google Public DNS FAQ (anycast resolver): https://developers.google.com/speed/public-dns/faq

Preparing for a Cloud, DevOps or SRE interview? Book a Free 15-min Cloud & DevOps Interview Guidance Call with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 https://topmate.io/cloudtechbydevang/2358399

You open Netflix on a Friday night. You press Play. The video starts in a second or two, in good quality, with no buffering. Millions of other people around the world are doing the same thing at the same time.

Netflix crossed 325 million paid memberships in late 2025. That is a huge number of people who expect the app to "just work". So how does Netflix stay up?

There is no single magic tool. It is a set of simple ideas, used with a lot of discipline. In this post I will explain these ideas in simple English. At the end, I will show you how to use them in a system design interview. I have spent 14+ years in cloud, DevOps and SRE, and I see the same patterns in every system that scales well, big or small.

1. It started with an outage

In August 2008, Netflix had a major database corruption. For three days, it could not ship DVDs to its members.

For a DVD business, that was painful. For a streaming business, it would have been a disaster. If streaming had been down for three days, nobody could watch anything at all.

Netflix took one big lesson from this: a single, vertically scaled database is a single point of failure. "Vertically scaled" means you make one machine bigger and bigger. When that one machine breaks, everything breaks.

So Netflix decided to move to horizontally scalable, distributed systems in the cloud. "Horizontally scaled" means you add more machines instead of making one machine bigger. If one fails, the others keep working.

They chose AWS (Amazon Web Services). Most customer-facing systems were moved before 2015. In January 2016, after seven years of work, Netflix shut down the last data centre parts used by its streaming service.

The key point: Netflix did not just copy its old systems to the cloud ("lift and shift"). It rebuilt them. In their own words, they moved from a monolithic app to hundreds of microservices, and they changed their data model to use NoSQL databases.

Lesson for us: moving to the cloud does not make you reliable by itself. Changing the design does.

2. Two paths: the "brain" and the "video"

This is the most important idea in the whole post, so let's make it simple.

When you use Netflix, two very different kinds of work happen:

1. The brain (control plane). Logging in, choosing a profile, loading your home screen, showing recommendations, checking your plan and licence, and deciding which server should send you the video. This is lots of small requests. It runs on AWS.

2. The video (data plane). The actual movie file: gigabytes of data flowing to your TV or phone. This comes from Open Connect, Netflix's own content delivery network (CDN).

Here is what happens when you press Play:

1. Your device sends a request to Netflix's front door on AWS. This front door is an API gateway called Zuul. It routes requests, helps with authentication, and can shed load when things go wrong.

2. Zuul sends the request to the right microservices: profiles, recommendations, licensing, playback and more.

3. The playback service looks at which video servers are healthy, which ones have your title, and which ones are closest to you on the network. It sends your device a ranked list of servers to stream from.

4. Your device downloads the video from an Open Connect server, often one that sits inside your own internet provider's network.

Why split it like this? Because the two jobs need very different things. The brain needs flexibility, fast changes and lots of compute. The video needs huge bandwidth, close to the user. Putting each job on the right platform is cheaper and more reliable than forcing both into one place.

3. Open Connect: the video comes from near your home

A CDN (content delivery network) is a group of servers spread around the world that keep copies of content close to users. Most companies rent a CDN. Netflix built its own, called Open Connect, starting around 2012.

As of December 2022, Netflix said it had 18,000 servers in 6,000 locations across 175 countries.

How it works, in simple steps:

• Servers inside ISPs. Netflix gives its Open Connect Appliances (OCAs) free of charge to qualifying internet service providers. Netflix provides the hardware. The ISP provides rack space, power and network connection.

• Servers at internet exchanges. Netflix also places OCAs at internet exchange points, where many networks connect to each other, and peers with ISPs there.

• Filled at night. Netflix predicts what people will watch and copies those files onto the servers during off-peak hours, when the network is quiet. So when a big new show comes out, it is already sitting near you.

• Steered by the control plane. Each OCA reports its health, the routes it has learned (using BGP, the routing protocol of the internet) and the files it has to the Open Connect control plane, which runs in AWS. The control plane uses this data to point your device to the best server, and gives a list of options in case one fails.

The result: the video travels a short distance. It starts faster, buffers less, and puts less load on the wider internet. Netflix estimated that in 2021 alone the program helped ISPs avoid about $1.25 billion in costs.

Lesson for us: move heavy, read-only content as close to users as you can. You may not build your own CDN, but CloudFront, Cloud CDN, Akamai or Cloudflare give you the same idea.

4. Microservices: many small teams, many small services

Netflix runs hundreds of microservices. Each one does one job well: one for profiles, one for viewing history, one for recommendations, and so on.

Why does this help reliability?

• Small blast radius. If the recommendations service has a problem, you might see a simpler home screen, but you can still play a show. One broken part does not take down everything.

• Independent scaling. The playback service can scale up on a Friday night without scaling everything else.

• Independent deployments. Teams can release small changes often, instead of one big risky release.

But microservices bring new problems. When one screen needs many service calls, a slow service can make everything slow. Netflix built (and open-sourced) tools to deal with this:

• Eureka for service discovery (how services find each other).

• Ribbon for client-side load balancing.

• Hystrix for fault tolerance: it isolated calls between services, stopped one failure from spreading to others ("cascading failure"), and gave fallbacks, a simpler answer when a service could not respond.

Netflix later moved to Spring Boot and Spring Cloud as its core Java framework (announced in 2018), and Hystrix is now in maintenance mode. But the ideas are still the base of every good microservice system: timeouts, retries with limits, circuit breakers, and fallbacks. Today you will see them in service meshes like Istio and Envoy, and in libraries like Resilience4j.

A key design rule at Netflix: many services are stateless. They do not keep important data in their own memory. They keep it in caches or databases. This makes it easy to add or remove servers, and even to serve any member from any AWS region.

5. Caching: don't ask the database every time

Your home screen needs a lot of data, very fast: what you watched, what you rated, what rows to show. Hitting a database for every one of these would be slow and expensive.

So Netflix uses EVCache, a caching service built on Memcached. It keeps frequently read data in memory, so reads take milliseconds or less. Netflix's own blog says EVCache is used widely across its microservices and is open source.

Netflix also built global replication for EVCache. Its cloud runs in more than one AWS region (their blog named Northern Virginia, Oregon and Ireland). When data changes in one region, the change is sent to the other regions through a Kafka-based replication system. So if traffic moves to another region, the cache there is already warm.

There are really two layers of caching at Netflix:

1. Data cache (EVCache) for the brain.

2. Content cache (Open Connect) for the video.

Lesson for us: find the data that is read often and changes rarely, and cache it. Think about what happens when the cache is empty ("cold"), and how you keep it fresh.

6. Chaos engineering: break things on purpose

This is the part everyone remembers.

In 2011, Netflix wrote about Chaos Monkey, a tool that randomly shuts down production servers. Why would anyone do that?

Because in the cloud, servers will fail. Netflix's thinking was simple: if failure is going to happen anyway, make it happen often, in a controlled way, while engineers are awake and watching. Then you find weak spots and build automatic recovery. In their words, the goal was that when an instance fails at 3 AM on a Sunday, they won't even notice.

Chaos Monkey ran during business hours, in a carefully monitored environment, with engineers standing by. In July 2012, Netflix open-sourced it under the Apache License.

Chaos Monkey was the first member of the Simian Army:

• Latency Monkey added artificial delays to test slow services.

• Conformity Monkey found servers that did not follow best practices.

• Chaos Gorilla simulated losing a whole AWS Availability Zone.

• Chaos Kong simulated losing a whole AWS region.

Netflix practises region evacuation: moving all traffic out of one AWS region into the others. With a project called Nimble, which keeps extra capacity ready, Netflix said it cut evacuation time from about 50 minutes to about 8 minutes.

Lesson for us: you don't need to start with production. Start with a game day in staging. Kill a pod. Block a dependency. Watch your alerts. Write down what surprised you. Make the blast radius small and grow it slowly. Today, tools like AWS Fault Injection Service, Chaos Mesh and LitmusChaos make this easier.

7. Putting it all together

Here is the full picture in one place:

None of these ideas is new on its own. What makes Netflix strong is that they use all of them together, and they practise failure as a normal part of work.

8. How to use this in a system design interview

A very common interview question is: "Design a video streaming platform like Netflix."

Here is a simple structure you can follow:

1. Clarify. How many users? Which regions? Live or on-demand? Which devices? What matters most: start time, quality, cost?

2. Split control plane and data plane. APIs (login, catalogue, recommendations, playback decisions) on the cloud. Video delivery on a CDN. Say this early. It shows you understand the problem.

3. Content pipeline. Upload, encode into many bitrates and formats, store in object storage (like S3), push popular content to the CDN before peak hours.

4. Playback flow. Client asks the playback service, which returns a ranked list of CDN servers. The client uses adaptive bitrate streaming, so quality goes down instead of stopping when the network is slow.

5. Scale and reliability. Stateless microservices with auto-scaling. Caching for hot data. Multi-AZ always, multi-region for critical services. Timeouts, retries and circuit breakers between services.

6. Prove it. Monitoring on key metrics (stream starts per second is a great business metric), alerts, and chaos testing.

7. Trade-offs. Own CDN vs rented CDN. Active-active multi-region (more cost, faster recovery) vs active-passive (cheaper, slower recovery). Strong vs eventual consistency for things like viewing history.

Senior tip: interviewers listen for trade-offs and for real experience. If you can, add one line from your own work, for example: "In my last project we put static content behind a CDN and cut origin load by X%." Numbers and your role make the answer believable.

9. Key takeaways

• Remove single points of failure. Scale out, not just up.

• Split the brain (control plane) from the heavy data (data plane).

• Put content close to users with a CDN.

• Keep services small and stateless, with timeouts and fallbacks.

• Cache what is read often.

• Practise failure on purpose, starting small.

If this helped, save it, share it with a friend who is preparing for interviews, and follow @cloudtechbydevang for one cloud and DevOps topic a day.

Sources

• Netflix Q4 2025 shareholder letter (Jan 2026): "crossed the 325M paid memberships milestone". https://ir.netflix.net/files/doc\_financials/2025/q4/FINAL-Q4-25-Shareholder-Letter.pdf · CNBC coverage: https://www.cnbc.com/2026/01/20/netflix-nflx-earnings-q4-2025.html

• Variety (Apr 2024), Netflix stops quarterly subscriber reporting from Q1 2025: https://variety.com/2024/tv/news/netflix-stop-reporting-subscriber-numbers-starting-2025-1235975341/

• Netflix, "Completing the Netflix Cloud Migration" (Feb 2016): https://about.netflix.com/news/completing-the-netflix-cloud-migration

• Netflix, "Open Connect: Celebrating a Decade" (Dec 2022): https://about.netflix.com/news/open-connect-celebrating-a-decade-of-smooth-and-efficient-streaming

• Open Connect overview and partner docs: https://openconnect.netflix.com/en · https://openconnect.netflix.com/Open-Connect-Overview.pdf

• Netflix Tech Blog, "The Netflix Simian Army" (Jul 2011): https://netflixtechblog.com/the-netflix-simian-army-16e57fbab116

• Ars Technica, Chaos Monkey open-sourced (Jul 2012): https://arstechnica.com/information-technology/2012/07/netflix-attacks-own-network-with-chaos-monkey-and-now-you-can-too/

• Netflix Tech Blog, "Open Sourcing Zuul 2": https://netflixtechblog.com/open-sourcing-zuul-2-82ea476cb2b3

• Netflix Tech Blog, "Introducing Hystrix for Resilience Engineering": https://netflixtechblog.com/introducing-hystrix-for-resilience-engineering-13531c1ab362

• Netflix Tech Blog, "Netflix OSS and Spring Boot — Coming Full Circle": https://netflixtechblog.com/netflix-oss-and-spring-boot-coming-full-circle-4855947713a0

• Netflix Tech Blog, "Caching for a Global Netflix" (EVCache): https://netflixtechblog.com/caching-for-a-global-netflix-7bcc457012f1

• Netflix Tech Blog, "Project Nimble: Region Evacuation Reimagined": https://netflixtechblog.com/project-nimble-region-evacuation-reimagined-d0d0568254d4

Preparing for a Cloud, DevOps or SRE interview? Book a Free 15-min Cloud & DevOps Interview Guidance Call with me: what to study, how to answer system-design and on-call questions, plus a CV and roadmap check 👉 https://topmate.io/cloudtechbydevang/2358399