HomeDesign a Rate Limiter

Medium

Design a Rate Limiter

Overview

MediumBackend

Design a Rate Limiter

Build a distributed rate limiting service to protect APIs from abuse by limiting requests per user/IP across multiple microservices.

What is this system?

A Rate Limiter (RL) is a critical security layer that prevents users from making too many requests to an API. It protects the system from abuse, DDoS attacks, and resource exhaustion. Rate limiters are used everywhere: YouTube limits video uploads per user, Twitter limits tweets per day, Stripe limits API calls per second.

Target Architecture

api_client

User / Client

Millions of user requests

⚡ 10000 QPS0

load_balancer

Load Balancer (Nginx)

Routes requests to multiple RL instances

⚡ 10000 QPS⏱️ 2ms

web_server

Rate Limiter Instance #1

Node.js service running RL logic

⚡ 5000 QPS⏱️ 10ms

web_server

Rate Limiter Instance #2

Redundant RL instance for failover

⚡ 5000 QPS⏱️ 10ms

cache

Redis (Cluster Mode)

Distributed token bucket storage with replication

⚡ 100000 QPS⏱️ 2ms

database

PostgreSQL (Rules)

Rate limit rules per service/user

⚡ 1000 QPS⏱️ 15ms

microservice

Backend Services (YouTube, Comments, etc)

Protected by rate limiter

⚡ 5000 QPS⏱️ 100ms

React Flow

100%

Key Insight

Rate limiters sit at the gateway level and enforce rules dynamically based on user identity, service type, and API endpoint. Different services can have different rules - a user might post 100 comments but upload only 20 videos in 24 hours.

Real-World Context

Every major tech company (Google, Amazon, Netflix, Stripe) runs rate limiters. They handle millions of requests per second while maintaining sub-millisecond latency.

What you'll learn

1.Gateway Rate Limiting Setup
2.Database rules check choking
3.Adding Redis State Cache
4.Stateless Scaling and Redundancy
5.Choosing Token Bucket Algorithm
6.In-Memory Data Structures

TARGET ARCHITECTURE — READ ONLY

api_client

User / Client

Millions of user requests

⚡ 10000 QPS0

load_balancer

Load Balancer (Nginx)

Routes requests to multiple RL instances

⚡ 10000 QPS⏱️ 2ms

web_server

Rate Limiter Instance #1

Node.js service running RL logic

⚡ 5000 QPS⏱️ 10ms

web_server

Rate Limiter Instance #2

Redundant RL instance for failover

⚡ 5000 QPS⏱️ 10ms

cache

Redis (Cluster Mode)

Distributed token bucket storage with replication

⚡ 100000 QPS⏱️ 2ms

database

PostgreSQL (Rules)

Rate limit rules per service/user

⚡ 1000 QPS⏱️ 15ms

microservice

Backend Services (YouTube, Comments, etc)

Protected by rate limiter

⚡ 5000 QPS⏱️ 100ms

React Flow

100%

Loading architecture...

The System Design Expert

HomeDesign a Rate Limiter

Medium

Design a Rate Limiter

Overview

MediumBackend

Design a Rate Limiter

Build a distributed rate limiting service to protect APIs from abuse by limiting requests per user/IP across multiple microservices.

What is this system?

Target Architecture

api_client

User / Client

Millions of user requests

⚡ 10000 QPS0

load_balancer

Load Balancer (Nginx)

Routes requests to multiple RL instances

⚡ 10000 QPS⏱️ 2ms

web_server

Rate Limiter Instance #1

Node.js service running RL logic

⚡ 5000 QPS⏱️ 10ms

web_server

Rate Limiter Instance #2

Redundant RL instance for failover

⚡ 5000 QPS⏱️ 10ms

cache

Redis (Cluster Mode)

Distributed token bucket storage with replication

⚡ 100000 QPS⏱️ 2ms

database

PostgreSQL (Rules)

Rate limit rules per service/user

⚡ 1000 QPS⏱️ 15ms

microservice

Backend Services (YouTube, Comments, etc)

Protected by rate limiter

⚡ 5000 QPS⏱️ 100ms

React Flow

100%

Key Insight

Real-World Context

Every major tech company (Google, Amazon, Netflix, Stripe) runs rate limiters. They handle millions of requests per second while maintaining sub-millisecond latency.

What you'll learn

1.Gateway Rate Limiting Setup
2.Database rules check choking
3.Adding Redis State Cache
4.Stateless Scaling and Redundancy
5.Choosing Token Bucket Algorithm
6.In-Memory Data Structures

TARGET ARCHITECTURE — READ ONLY

api_client

User / Client

Millions of user requests

⚡ 10000 QPS0

load_balancer

Load Balancer (Nginx)

Routes requests to multiple RL instances

⚡ 10000 QPS⏱️ 2ms

web_server

Rate Limiter Instance #1

Node.js service running RL logic

⚡ 5000 QPS⏱️ 10ms

web_server

Rate Limiter Instance #2

Redundant RL instance for failover

⚡ 5000 QPS⏱️ 10ms

cache

Redis (Cluster Mode)

Distributed token bucket storage with replication

⚡ 100000 QPS⏱️ 2ms

database

PostgreSQL (Rules)

Rate limit rules per service/user

⚡ 1000 QPS⏱️ 15ms

microservice

Backend Services (YouTube, Comments, etc)

Protected by rate limiter

⚡ 5000 QPS⏱️ 100ms

React Flow

100%