SysPad › Examples › Chat system
Chat system
An NLB (not ALB - connections are long-lived and stateful) fronts WebSocket chat servers, with a Redis presence tracker, a Cassandra-style message store, and push notifications for offline users.
Why chat needs an L4 load balancer, and message persistence kept separate from presence tracking so a message send both durably stores and routes in one hop.
- Load: 10,000 req/sec
- First to saturate: Offline Push
- Estimated cost: $61k/mo
How it works
- NLB
- An L4 network load balancer, not an L7 ALB: chat connections are long-lived, stateful WebSockets, so you balance connections once at connect time rather than per request.
- Chat Servers
- Hold the WebSocket. On each message they persist it and check the recipient’s presence in parallel, delivering over an open socket if online or handing to push if not.
- Presence
- Who is online right now, updated by heartbeats. Kept separate from the message store because it is tiny, hot, and ephemeral - the opposite access pattern to durable messages.
Request flows
Send a message
- Client
- DNS
- NLB
- Chat Servers
- Message Store
- Presence
- Offline Push
Presence heartbeat
- Client
- DNS
- NLB
- Chat Servers
- Presence
Manage a group
- Client
- DNS
- NLB
- Chat Servers
- Group Management
Lessons for this architecture
- Modelling users and traffic entry points
- How Route 53 works
- How Network Load Balancer works
- How Amazon API Gateway works
- How Amazon ElastiCache works
- How Keyspaces (Cassandra) works
- How AWS Lambda works
- How SNS works
Worked step by step
The same interview question, built up one twist at a time, with a simulation at each stage.