Designing a YouTube-Scale Video Platform

Every minute, thousands of hours of video are uploaded to platforms like YouTube, while millions of viewers expect playback to begin almost instantly, regardless of their device or network conditions. Building a system that satisfies those expectations is far more than a storage problem. In this article, we'll design a YouTube-scale video platform from first principles, introducing each architectural component only when the previous design can no longer meet the system's requirements.

PS: This article designs a hypothetical large-scale video platform inspired by YouTube. It focuses on the core upload and playback architecture rather than YouTube's proprietary implementation.

The Problem

Users upload videos, sometimes several gigabytes at a time, and a global audience watches them on whatever device and connection they happen to have. Playback needs to start within a second or two and stay smooth as available bandwidth rises and falls. The scope here is upload and delivery. Browsing, search, and recommendations are a separate ranking problem, and they're set aside.

A few questions are worth settling before anything gets designed. Is this video-on-demand or live? Pre-recorded video gets transcoded once after upload, while live video has to be transcoded in real time with much tighter latency, and that's a genuinely different system. This design assumes video-on-demand unless stated otherwise.

Views outnumber uploads by orders of magnitude, and that gap is exactly what makes CDN and delivery costs the central concern rather than a side detail. The mix of devices and networks matters too. A phone on cellular data and a television on fiber can't possibly share the same file, and that gap is what determines how many rungs the quality ladder needs and how much adaptive streaming actually matters. Minutes of delay between upload and a watchable video are acceptable for on-demand content, which is what allows transcoding to happen asynchronously, in batches, rather than under time pressure. Assume a global audience, which pushes per-region storage and CDN placement to the front of the list. And assume recommendations and search stay out of scope, since they're really a separate problem in ranking.

Why Video Isn't Just Another Storage Problem

The read path here looks like most media platforms at first glance: files sitting in blob storage, served through a content delivery network. Video adds two things that shape everything downstream. One is the encoding ladder, meaning multiple bitrates produced from a single upload. The other is adaptive bitrate streaming, where the player itself picks a quality for each few-second segment based on how the network is behaving right now.

The cost profile looks different too. Storage is cheap. Getting the bytes out to a large audience is what actually costs money, so a lot of the decisions later in this design exist purely to keep origin traffic as close to zero as possible. Data coming into the system, meaning uploads, is called ingress. Data leaving it, meaning the bytes served out to viewers, is called egress. For video, egress dwarfs ingress, because every upload gets watched many times over.

What actually defines this problem, then, isn't storage and retrieval. It's delivery cost and the encoding ladder.

The Concepts This Design Rests On

Video playback works as a loop. The player downloads a small manifest that lists the available quality levels and the URLs of the segments making up the video. It then fetches segments a few seconds at a time into a buffer, a short queue of already-downloaded video waiting to be shown, and plays from that buffer while continuing to download ahead of playback. A brief dip in the network just drains the buffer rather than stopping anything, and at each segment boundary the player is free to switch to a higher or lower quality.

Renditions

A rendition is the same video encoded at one resolution and one bitrate, a single rung on the ladder. A lower bitrate means a smaller file and lower quality. A higher bitrate means the opposite. Bitrate is measured in megabits per second, the rate at which the stream delivers data. Since eight bits make a byte, 3 Mbps works out to roughly 0.38 MB per second, or around 225 MB for a ten-minute clip. For reference, typical renditions on a platform like this run around 1 Mbps at 360p, 2.5 Mbps at 720p, 4.5 Mbps at 1080p, and upward of 15 Mbps at 4K. To play without stalling, the viewer's connection has to sustain at least the bitrate of whatever rendition is playing.

Codecs

A codec compresses each rendition. It stores one full keyframe and then only the differences between subsequent frames, which is where nearly all of the compression comes from. A more efficient codec gets the same visual quality at a lower bitrate, which is the whole game when it comes to controlling delivery cost.

The encoding ladder

The encoding ladder is the full set of renditions produced from a single upload, something like 240p all the way up to 4K, one rung for each rough class of device and network. The same video exists at every rung simultaneously, so whatever device is watching, the player can always find a quality it can actually sustain.

Segments and the manifest

A rendition is never fetched as one single file. Each one gets divided into short segments, typically a few seconds long, and that segment is the actual unit that gets requested and cached. A manifest lists every rendition and, for each one, the ordered list of segment URLs. Two manifest formats dominate in practice: HLS, Apple's protocol using .m3u8 files, and DASH, the open standard using .mpd files. Both follow the same basic model of a manifest plus segments, and the player reads the manifest once and then fetches segments in order from whichever rendition it's currently using.

Adaptive bitrate streaming

The player, not the server, decides which quality to request. It measures the bandwidth it's actually getting and, at every segment boundary, asks for the next segment at the highest rendition it believes it can sustain, stepping down when the signal weakens and back up once it recovers. The server's job here is simple: serve cacheable segments and nothing more. All of the adaptive logic lives entirely on the client.

Rebuffering

If the buffer runs empty, playback stalls until enough new data arrives, the familiar spinning-wheel pause. It happens when the network can't deliver the current quality fast enough to keep the buffer ahead of what's being played. Preventing exactly this is the entire reason adaptive bitrate streaming exists in the first place: a fixed 1080p stream will stall the moment bandwidth dips, while an adaptive stream drops down to whatever quality the network can actually sustain, keeps the buffer full, and playback never stops.

The idea holding all of this together: playback is a loop driven entirely by the client. Read the manifest, fill the buffer, and switch renditions at segment boundaries as conditions change.

Figure 1. Video playback lifecycle.

Requirements

The problem statement names two actions directly: users upload videos, and an audience watches them. Discovery, meaning browsing, search, and subscriptions, is a separate ranking-and-search problem and stays out of scope. That leaves two functional requirements: uploading a video, after which it gets transcoded and becomes watchable, and watching a video with adaptive streaming, meaning smooth playback across different connections with seeking supported.

The non-functional requirements come from the qualities the problem statement demands. Starting playback "within a second or two" and staying smooth "as bandwidth changes" sets both startup latency and continuous playback quality, both of which adaptive streaming addresses directly. Serving "a global audience on any device and any connection" sets an availability bar and, at that scale, makes delivery cost the binding constraint on the whole design. And the unstated but obvious expectation that an uploaded video is never simply lost sets a durability requirement on the source file.

Two things are worth separating clearly here: the property that absolutely cannot be compromised, and the constraint the rest of the architecture organizes itself around. Durability is the property. If the source master is lost, that video can never be recovered, full stop. Egress is the constraint. Storage is inexpensive and metadata is small, so nearly every later decision, tiered caching, immutable segment URLs, lazy transcoding of rarely-watched content, per-region replication, all serves one purpose: pushing origin egress as close to zero as it can get.

Sizing the System

The point of estimating numbers here isn't precision. It's establishing, with actual figures, that egress dominates every other cost by an enormous margin. Say there are roughly a billion watch-hours across the platform every day, coming from about a billion daily viewers each watching around an hour. Say roughly 500 hours of video get uploaded every minute, from a much smaller set of creators each uploading a few minutes a day. Say the average stream runs about 3 Mbps, a typical mid-ladder quality somewhere around 720p to 1080p averaged across viewers and devices. And say a source video runs about 10 MB per minute of footage, with the full encoding ladder adding roughly two to three times that once every rendition is accounted for.

Ingest works out to something on the order of 12.5 GB of new stored bytes every second, or around 1.08 petabytes a day. That's a real number, but storage is cheap and this data only gets written once. Egress is a different story entirely: at a billion watch-hours a day and 3 Mbps average, the math comes to roughly 1,350 petabytes a day, or about 40.5 exabytes a month. Egress moves something like 1,250 times more bytes per day than storage grows, and unlike storage, it recurs on every single view.

Video metadata, meanwhile, barely registers. Records run around a kilobyte each, easily handled by an ordinary sharded database, and media outweighs metadata by roughly a thousand to one. That gap alone is reason enough to keep metadata and media in entirely separate systems.

The Interface

Picture a multi-gigabyte upload over a flaky mobile connection that drops partway through. The upload path has to support recovering from that without starting over, which means splitting the file into chunks under a single session and re-sending only the pieces that didn't make it, rather than retrying the whole transfer from scratch.

The API splits into two halves, and neither one looks like an ordinary CRUD interface for small objects. Getting bytes in means handling a multi-gigabyte file over a connection that might drop at any point. Getting bytes out means serving cacheable media to a player that changes quality mid-playback.

Opening an upload session. A single request carrying the entire file can't be resumed if it fails, so the first call carries no bytes at all. It just opens a session: the server allocates an id, returns a short-lived, pre-authorized link (a pre-signed URL, the kind Amazon S3 or Google Cloud Storage provide) that lets the client write bytes directly to blob storage without passing through application servers, and fixes the chunk size the client is expected to use.

Uploading the chunks. With a session open, the client sends the file as fixed-size chunks, each one identified by its index, so a failure only affects a single chunk rather than the whole file. The server just tracks which parts have arrived. If the connection drops, recovery means re-sending only the missing chunks.

Committing the upload. At this point the bytes are sitting in blob storage, but nothing about the video is watchable yet, and transcoding shouldn't start until every chunk has actually arrived. A final call commits the session: it verifies all the parts are present, writes the video's metadata record, and fires off the event that kicks transcoding into motion. Keeping this as a separate step from the chunk uploads guarantees the record gets created and transcoding gets triggered exactly once, and only after every byte has landed.

Figure 2. Resumable upload pipeline.

Fetching the manifest. On the read side, when a viewer opens a video, the player first needs the list of available qualities and where their segments live, the manifest. That's a small metadata read, nothing to do with actual media.

Fetching the segments. Finally, the media itself. Each segment is a plain, immutable file with its own URL, served straight from the CDN edge rather than any application endpoint. Because the URL never changes, the CDN can cache it indefinitely, and application servers never sit anywhere on the actual playback path.

The Data Model

Rather than starting from a finished schema, it's worth building it up one gap at a time.

The video itself

The uploaded item has a title, an uploader, a duration, and a status that moves through uploading, then transcoding, then ready. One row per upload: a video id, uploader id, title, status, duration in seconds, and a creation timestamp. That's enough to list a video and track its progress, but it says nothing about how the video actually plays.

The rendition

A phone on cellular needs a low-quality version. A television on fiber wants the top of the ladder. Each of these is a separately encoded file with its own codec and bitrate, part of the encoding ladder described earlier. That's a one-to-many relationship a single column can't hold, so it becomes its own entity: a rendition id, the video id it belongs to, a quality label, bitrate in kbps, codec, and its own manifest URL. Each rendition is one rung of the ladder, a resolution, bitrate, and codec bundled with a manifest, and the same video exists at every rung so the player can always pick something it can sustain.

The segment

Even a rendition can't be fetched as one file, because adaptive streaming changes quality mid-playback. The player fetches a few seconds at a time, and the CDN needs something addressable at that same granularity to cache. So each rendition gets divided into short segments: a segment id, the rendition it belongs to, its sequence number, its URL, and its duration. The manifest is what reassembles these into something playable, listing them in order for the player to walk through.

Where each piece actually lives

Metadata, meaning the video and rendition records, lives in a sharded database partitioned by video id: small, structured, and read on every single watch. The large data, meaning the source master, the renditions, and the segments, lives in blob storage behind immutable, cache-forever URLs, with the source retained indefinitely in case it needs to be re-transcoded to a new codec down the line. The status field on the video record governs availability directly: a video becomes watchable as soon as its baseline renditions exist, and higher-quality rungs can keep encoding in the background afterward.

One video fans out to many renditions, and each rendition fans out to many segments. Metadata and media never share a store.

Building the Architecture, One Failure at a Time

The naive version

Start with one app server. A creator uploads to it, it writes the file to disk, and on playback it streams the bytes straight back out. Under real video traffic this fails in three distinct ways. The multi-gigabyte upload passes through the app server over a connection that might drop at any point. One stored file can't serve both a phone and a television equally well. And every single viewer streams bytes through that same server, which makes origin egress prohibitively expensive the moment the audience grows past a handful of people.

Fix one: get the upload off the app server

Picture a 4 GB upload from a phone, routed through the app server, dropping at ninety percent complete. A naive retry restarts from byte zero. The actual fix changes where the bytes get written in the first place. They go directly to a blob store using a resumable, pre-signed URL, so a dropped connection only costs one chunk's worth of retry, and the app server ends up writing nothing but a small metadata record. Large opaque files simply don't belong on an app server's local disk.

Fix two: transcode into a ladder, off the request path

The bytes are stored now, but a raw source still can't play everywhere, so it needs to be transcoded into the full ladder of renditions and segments. Doing that inside the original upload request seems like the obvious approach, but a full ladder takes minutes of CPU time, which means the request would simply time out, and upload availability would end up hostage to however much encoding capacity happened to be free at that moment.

Instead, a completed upload places an event on a queue, and a fleet of workers runs the ladder asynchronously: encoding every rendition, slicing everything into segments, writing renditions and segments to blob storage, and finally flipping the video's status to ready. Because segments are independent of each other, this work spreads cleanly across the whole fleet.

The upload-to-watchable flow, start to finish: the creator uploads chunks resumably, an upload-complete event fires, a worker picks up the transcode job from the queue, encodes the ladder and slices it into segments, writes the renditions and segments to blob storage, and finally sets the video's status to ready.

Fix three: serve playback from the CDN, not the origin

Imagine a brand-new video suddenly goes viral. Within the same second, thousands of edge caches around the world all miss on the same not-yet-cached segment and turn to the origin simultaneously. The fix isn't provisioning the origin to survive that burst. It's putting a shield in front of the origin that merges all of those simultaneous misses into a single fetch.

This is the costly failure of the three. If every viewer read straight from the origin, origin egress would equal the platform's entire viewing bandwidth, the exabyte-scale figure from the earlier estimate. Nearly every byte has to be served from the edge instead. The viewer fetches the small manifest from the metadata store, then fetches immutable segments straight from the CDN, switching renditions per segment through adaptive streaming. The origin only gets touched on a cache miss, and those misses pass through an origin shield, a single intermediate cache sitting between the many edge locations and the origin itself. Because every edge in a region routes its misses through that one shield rather than hitting the origin directly, a cold segment gets read from the origin exactly once and then served to every edge that wants it afterward. That coalescing is precisely what keeps a newly viral video from taking the origin down.

A playback request, cache hit versus cache miss: the viewer asks for the manifest and gets back rendition and segment URLs, then asks for a segment. On a hit, it's served straight from the edge. On a miss, the request goes through the shield to the origin, gets served back, and gets cached at the edge for next time.

The composed design

Put the three fixes together and the full system emerges. Metadata lives in its own database. The source, written once at upload time, lives in one blob store. The renditions, written later by the transcode workers, live in another. A queue connects the upload path to the transcode fleet. The CDN edge and origin shield sit between viewers and the renditions store. Each of these pieces answers one specific failure of the original single-server design: direct-to-blob upload solves the fragile transfer, asynchronous transcoding solves the request timeout, and CDN delivery with an origin shield solves the cost and overload problem of serving a global audience.

Figure 3. Architecture evolution.

Figure 4. Final system architecture.

A Closer Look at a Few Parts

The transcode pipeline

One uploaded video needs to become five or more separate copies, because no single source can play acceptably on every device. That's the encoding ladder. Producing it is five or more times the encoding work of a single pass, and a two-hour video obviously can't be allowed to take two hours to transcode. The fix is segmentation and parallelism. Because segments are independent, the source gets split into chunks, those chunks get transcoded concurrently across the whole worker fleet, and the results get reassembled afterward. The useful side effect is that transcode time stops being tied to video length at all. Given enough workers, a two-hour video and a ten-minute video both finish in roughly the time it takes to encode a single chunk.

Two refinements round this out. Producing separate HLS and DASH manifests for every rendition from scratch would duplicate a lot of storage, so instead the source gets packaged once into a shared segment format both protocols can reference, and the packager generates both manifests from those same underlying segments, roughly halving segment storage in the process. And since a job spread across a whole fleet can fail partway through, each unit of work gets keyed by video, rendition, and segment together, so a single failed segment retries on its own rather than restarting the entire transcode. Because the source master is always retained, a failure only ever delays completion. It never loses data outright.

Adaptive delivery and the CDN

The player decides which quality to request, and it decides fresh at every single segment boundary. It measures its own download throughput and how much buffered video it has left, and at each boundary requests the next segment at the highest rendition those two signals can sustain, dropping to 480p instead of 1080p when bandwidth falls and climbing back up once it recovers. The server does nothing adaptive at all and holds no state per viewer. It just serves cacheable segments. All the intelligence sits at the very edge of the system, on the client itself, which keeps the actual delivery path a dumb, cacheable read.

The manifest is the contract that makes this possible. It lists every rendition and its segment URLs, and the player just picks a path through that list, switching renditions between segments without ever re-downloading anything it already has. Because segment URLs never change, the CDN can cache each one forever.

The CDN hierarchy carries most of the actual load. Edge points of presence, meaning a CDN's local cache clusters placed close to users across many cities, serve the overwhelming majority of requests for popular segments, while regional caches and the origin shield push the cumulative hit rate up toward nearly complete. A newly popular video only goes cold once per point of presence, and the shield collapses all the simultaneous misses that follow into a small handful of actual origin fetches. For a scheduled premiere or a drop known in advance to go viral, segments can even get pushed out to the edges ahead of demand, so the very first viewers never pay for a cold miss at all.

Delivery economics and storage tiering

Video is unusual among most systems in that egress bandwidth, not compute or storage, dominates the cost. That reduces most of this discussion to a single equation and its consequences: origin egress equals total traffic multiplied by one minus the cache hit rate. At exabyte scale, even a cache hit rate of 99% still leaves a substantial amount of origin egress, so the whole design leans on long cache lifetimes, immutable URLs, shield coalescing, and pre-positioning to push that hit rate as close to perfect as it can get.

Popularity itself follows a power law: a small set of videos accounts for most of the views, while a very long tail gets watched rarely if ever. Storage gets tiered accordingly, with popular renditions sitting on fast storage close to the CDN, and older or rarely-watched content moved onto cheaper archival storage that trades away fast first-byte latency. The same reasoning applies to which renditions even get produced up front. Eagerly generating a 4K rendition for a video that's almost never watched at that quality wastes both transcode time and storage, so common rungs get generated right away while rare or very high-quality rungs get generated on demand instead. Source masters and popular renditions get replicated across regions so the CDN's origin stays local to viewers, while cold content sits in fewer regions.

Variants Worth Naming

For live streaming, transcoding happens in real time as the stream arrives, segments get produced continuously, and the manifest keeps growing for the duration of the broadcast. The added constraint is latency, since viewers expect to be only seconds behind the actual live moment, which usually means shorter segments or low-latency chunked transfer. Delivery still runs through the CDN, but cache lifetimes shrink and the origin ends up receiving a continuous stream of brand-new segments rather than a fixed, already-finished set.

At ten times the scale described here, egress and storage both climb into multiple exabytes, and the same measures stop being optional and become mandatory: aggressive cold-tiering, on-demand transcoding for long-tail content, per-region CDN origins, and pushing cache hit rate as high as it will go. Transcode fleet cost becomes significant enough on its own that codec efficiency, choosing something like AV1 over H.264, becomes a real tradeoff between transcode CPU time and egress savings.

Recommendations and discovery, the watch-next feed, are their own separate ranking problem, built on a fan-out of subscriptions, and they're named here only to be set aside rather than designed.

The Pattern Underneath It All

A video platform is really the same metadata-and-media split found everywhere else, just applied at a scale where delivery, not storage, becomes the actual problem. One immutable file turns into a whole set of immutable files, renditions multiplied by segments. The read path turns into a cache-hit-rate optimization problem precisely because egress dominates cost. And the client, not the server, does the adapting, which keeps the delivery path itself a simple, cacheable read.

That same shape, precompute variants, segment everything for cacheability, push adaptation onto the client, and serve from the CDN, shows up anywhere large media reaches a global audience: live sports broadcasts, podcasts, game asset delivery, software distribution. Recognizing video as simply large media delivered at planetary scale reduces the whole problem down to an encoding ladder and a hit-rate budget.

The Short Version

Given thirty seconds to summarize the whole design, it comes down to five decisions. Upload once and transcode to a ladder, turning one source video into multiple renditions, each sliced into short segments, produced asynchronously behind a queue. Deliver adaptively, letting the client read a manifest, fetch segments, and change quality per segment as bandwidth shifts, keeping playback smooth across every kind of connection. Lean on the CDN, because video is fundamentally an egress-bound workload, and nearly every byte has to be served from the edge or both bandwidth cost and origin load become unmanageable. Keep metadata and media apart, with small records in a database and large segments in blob storage behind the CDN. And tier storage by popularity, since a small set of videos drives most of the views and the long tail belongs on cheaper storage.