Design a notification system

Do the numbers before the boxes

An interviewer gives a scale, not a load. If we draw before we do the math, we cannot say why any box has its size.

Each of our 10 million daily users gets 3.2 pushes a day: replies, messages and order updates.

Each also gets 0.75 emails a day, such as receipts and a share of a weekly digest.

And 0.05 SMS: one user in 20 signs in on a new device each day. In all, that is 4.0 notifications per user per day.

Per day: 10 million x 4.0 = 40 million. Per second on average: 40,000,000 / 86,400 = 463.

Traffic is not flat. Notifications follow the activity that causes them, and activity peaks in the evening.

We take a peak factor of 2.2: the busiest hour carries 2.2 times the average. 463 x 2.2 = 1,019, so we plan for 1,000.

We measure load in requests per second. Each request asks us to send one notification.

By channel at peak, push is 1,000 x 3.2 / 4 = 800. Email is 1,000 x 0.75 / 4 = 187.5, and SMS is 1,000 x 0.05 / 4 = 12.5.

Each push goes to one device, the one the user used most recently. Their other devices are fallbacks.

So push is 80% of messages, email 18.75% and SMS 1.25%. We size each part where it first appears.

Check in: Close the page and derive 1,000 requests per second from 10 million daily users. Then split it by channel.

Accept fast, send behind a queue

A producer calls us inside its own request. If we call a provider before we answer, every producer waits for Google, AWS or a phone carrier.

So we split the work in two. The Notification API accepts a request, and a Sender pool delivers it later.

The API checks the request, reads the user's settings, renders the template and puts one message on a queue.

A queue holds each message until a sender takes it. The API then answers 202 Accepted: we have the request, and we will finish it later.

Per task: each copy of the API is a task with one vCPU, a virtual processor.

A request costs about 1 ms of CPU. That is 0.2 ms to check it, 0.5 ms to render the template and 0.3 ms to build two calls.

One vCPU has 1,000 ms each second, so one task serves 1,000 requests per second.

The API uses auto-scaling: it adds tasks as load grows, and keeps each near 70% busy.

1,000 / (1,000 x 0.7) = 1.4, so its floor of 2 tasks runs, at 50%.

User settings is a DynamoDB table with one item per user. The item holds opt-ins per type and channel, email, phone and up to 3 devices.

That is 200 bytes of opt-ins, 100 of contact details, 3 devices of 180 bytes, 60 of attribute names and 100 of overhead: 1,000 bytes.

We buy provisioned capacity in read capacity units (RCU) per second. We pay for them, used or not.

A settings read is eventually consistent: it may miss a change made in the last second. A read of up to 4 KB costs 0.5 RCU.

Reads: 1,000 x 0.5 = 500 RCU. At 70% use, 500 / 0.7 = 714, so we buy 720 RCU.

Stage 1 runs this design in SysPad at 1,000 requests per second. Each card is a part of the system, and each arrow carries requests.

Read the four tiles. "First to saturate" reads "Nothing at this load". Its note names the busiest part: User settings, at 69%.

A part saturates when it gets more requests than it can serve.

The latency tile shows 3ms. The tile shows the slowest path, which here is the accept path: 1 ms in the API, then 2 ms to read settings.

That 3 ms is the accept time, what a producer waits for.

Delivery is not on that path. The queue cuts it off, so the producer has its 202 before any provider is called.

So every stage on this page shows 3ms, however slow a provider gets. We judge delivery by the queues and the senders.

On Stages 2 and 6 the tile reads "≥ 3ms", a lower limit, because the settings read is saturated there.

Press 1,500. User settings saturates: 1,500 x 0.5 = 750 RCU, against 720.

The 70% target buys 43% of headroom, because 1 / 0.7 = 1.43. The table runs out at 720 / 0.5 = 1,440 requests per second.

The trade-off:

A 202 is a promise. Once we accept, the queue must keep the message until a sender delivers it.

We pay for every read unit we buy. At peak, 30% of them sit idle, and that is our headroom.

Stage 1: Accept fast, one queue, one sender pool, 1,000 a second

• Load: 1,000 req/sec
• First to saturate: Nothing at this load
• Estimated cost: $458k/mo

• SMS provider (Twilio): SMS is billed per message: about $409,240/month at 12.5 msg/s and $0.0126 a message.

Check in: Say how you got 720 read units, starting from 1,000 requests per second. Then say where the table runs out.

Three providers, three ceilings, one bill

Each provider limits how fast we may send, and each bills in its own way. Before we grow, we need both numbers for each one.

The Sender pool takes messages off the queue and calls the provider for its channel. Each call mostly waits on the provider.

Push goes through FCM, Firebase Cloud Messaging from Google. iOS pushes go through it to Apple.

We assume an FCM call takes 50 ms from us-east-1, because Google publishes no figure.

We take 40 ms for an email call to SES, the AWS email service, and 100 ms for an SMS call to Twilio.

Average send: 0.8 x 50 + 0.1875 x 40 + 0.0125 x 100 = 48.75 ms of waiting. With 0.5 ms of CPU, we call it 50 ms.

Each sender task keeps 50 sends in flight: 50 slots, each waiting on one call. So a task sends 50 / 0.05 s = 1,000 a second.

That uses 1,000 x 0.5 ms = half a vCPU, so the slots set the limit, not the CPU. 2 tasks run, at 50%.

Google publishes that FCM accepts 600,000 messages a minute per project. 600,000 / 60 = 10,000 a second, and we use 800, or 8%.

SES sends at a rate that AWS grants per account and Region, called a quota. We ask for twice the peak: 187.5 x 2 = 375, so 400.

Why twice: a quota increase costs nothing, but AWS takes up to 24 hours to answer. So we ask early, with room.

SMS goes out on a short code, a short number that carriers approve for volume. One sends 100 messages a second and costs $1,000 a month.

We use 12.5 of its 100, or 12.5%. A toll-free number allows only 3 a second, which is too few.

Push is free. An email costs $0.10 per 1,000. An SMS costs $0.0083 at Twilio plus a carrier fee of about $0.0043: $0.0126.

So one SMS costs as much as 126 emails.

Read the stage's warnings. The only one says: "SMS is billed per message: about $409,240/month at 12.5 msg/s and $0.0126 a message."

The cost tile shows $458k/mo. SMS is 1.25% of messages and 89% of the bill.

Every cost tile prices the peak rate as if it ran all month. Messages bill per send, and the average rate is the peak / 2.2.

So an average month of Stage 1 costs about $209k, and SMS is still 89% of it. Comparisons between stages hold either way.

The trade-off:

We pay for reach. SMS reaches every phone, and it costs 126 times an email. So SMS carries sign-in codes and nothing else.

Stage 1: Accept fast, one queue, one sender pool, 1,000 a second

• Load: 1,000 req/sec
• First to saturate: Nothing at this load
• Estimated cost: $458k/mo

• SMS provider (Twilio): SMS is billed per message: about $409,240/month at 12.5 msg/s and $0.0126 a message.

Check in: Which provider runs out first as load grows? Work it out from each quota and its share of the messages.

Ten times the traffic, three walls

The interviewer asks: Traffic grows ten times, to 10,000 requests per second. What breaks first?

We now have 100 million daily users. At peak, that is 8,000 pushes, 1,875 emails and 125 SMS a second.

Press 1,000 on Stage 2, then press 10,000. The design is the same. Only the load changes.

At 10,000, three cards turn red: User settings, the Email provider (SES) and the SMS provider (Twilio).

"First to saturate" names one of them, and its note says "and 2 more".

The latency tile now shows "≥ 3ms", a lower limit. The settings read on the accept path is saturated.

Read the stage's warnings. Each red card fails for its own reason.

User settings needs 10,000 x 0.5 = 5,000 RCU and has 720. That is money: we buy read units, and they take effect in minutes.

SES gets 1,875 emails a second against a quota of 400. The fix is free: we ask AWS, and it answers within 24 hours.

Twilio gets 125 codes a second against 100. The fix is a second short code: $1,000 a month, and 6 to 10 weeks for carriers to approve it.

So the three walls look alike on the canvas, but their fixes take minutes, up to a day, and up to ten weeks. We call that the lead time of a fix.

The Push provider (FCM) card shows 80%. It is not red yet, and it is the one wall that money does not move.

Three design facts we reason about. A throttled settings read fails the accept, so producers get errors, not a slow 202.

SES answers a throttled send with an error, so the sender must back off and try again.

Twilio holds messages past 100 a second for up to 10 hours. A sign-in code that waits more than a few minutes has expired.

The cost tile shows $3,375k/mo. Providers bill only what they send, so throttled emails and held codes cost nothing yet.

Step through the figure to see the three walls and their lead times.

The trade-off:

A quota has a lead time, and traffic does not wait for it. We ask for each one before we need it.

Three walls, three lead times

1. At 10,000 a second, three parts saturate. Each one is short of something different.
2. The fixes differ by how long they take. Start the slowest one first.
3. FCM is not red yet. It is the one wall that money does not move.

Stage 2: The same design at 10,000 a second

• Load: 10,000 req/sec
• First to saturate: Email provider (SES)
• Estimated cost: $3,375k/mo

• Latency past saturation is a lower bound, not an estimate. User settings is saturated on the critical path, where the queueing model stops tracking a real queue - the true figure is higher by an amount this model cannot know. Relieve the bottleneck to get a latency number worth reading.
• User settings: Read capacity 694% used (provisioned): 5000 of 720 RCU/s at 10000 RPS.
• Email provider (SES): Send rate 1,875/s exceeds the 400/s sending-rate quota: 1,475/s are throttled.
• SMS provider (Twilio): Carrier throughput exceeded: 125 msg/s against 100/s (1 short code at 100/s each). The excess queues at Twilio, for up to 10 hours.
• SMS provider (Twilio): SMS is billed per message: about $3,266,920/month at 100 msg/s and $0.0126 a message.

Check in: Name the red card you can fix today, the one you can fix tomorrow, and the one that takes a quarter. Give each fix.

Buy units, ask for a quota, order a short code

Each wall needs its own fix. We size each one from the new load, with the same 70% target.

User settings: 10,000 x 0.5 = 5,000 RCU. 5,000 / 0.7 = 7,143, so we buy 7,200 RCU.

Why not cache settings? 7,200 RCU cost $674 a month, 0.03% of an average month. A cache for the hot set costs about 6 times that.

A cache could also ignore an opt-out for up to 5 minutes. Each item is read about 4 times a day, so a cache does not pay.

SES: we ask for twice the peak again. 1,875 x 2 = 3,750, so we ask for 4,000 emails a second.

SMS: 125 / 0.7 = 179 codes a second, more than one short code sends. So we use 2 short codes: 200 a second, 62.5% used.

We order the second short code 6 to 10 weeks before we need it. Growth over months gives us that time, if we watch.

The API and the Sender pool each scale to 10,000 / (1,000 x 0.7) = 14.3, so 15 tasks.

Press 10,000 on Stage 3. Nothing saturates. The note on "First to saturate" reads "Busiest: Push provider (FCM) at 80%".

FCM gets 8,000 of its 10,000 a second. Google raises that quota only on request: 25% more from its console, and more through Firebase Support.

Google does not promise to grant either. So FCM is the wall we watch from here on.

Say it out loud: sending each push to every device multiplies FCM load. At 1.2 devices per user, push is 9,600 a second, 96% of the quota.

Press 12,600. Only the Push provider (FCM) saturates: 0.8 x 12,600 = 10,080 pushes a second, against 10,000.

User settings runs out at 7,200 / 0.5 = 14,400. SMS runs out at 200 / 0.0125 = 16,000, and SES at 4,000 / 0.1875 = 21,333.

The cost tile shows $4,576k/mo. An average month is about $2.08M, and SMS is still 89% of it.

Say it out loud: "About $2 million a month at 10x, nearly 90% of it SMS."

The trade-off:

The fixes are cheap. User settings goes from $127 to $1,267 a month, and the second short code costs $1,000.

The bill grows by about $1,201k a month because we now send every email and code. That is the price of the traffic, not of the fixes.

Stage 3: More read units, a bigger email quota, a second short code

• Load: 10,000 req/sec
• First to saturate: Nothing at this load
• Estimated cost: $4,576k/mo

• SMS provider (Twilio): SMS is billed per message: about $4,084,400/month at 125 msg/s and $0.0126 a message.

Check in: Traffic grows another 25%. Press 12,600, name the part that breaks, and say how you would get more of it.

One slow provider, one shared pool

The interviewer asks: The push provider is having a bad day. What happens to sign-in codes?

We assume FCM still answers, but a call takes 1 second instead of 50 ms.

We do not give up on a call after 50 ms, so a slow answer holds its slot for the whole second.

Stage 4 is Stage 3 with two numbers changed. The FCM card now adds 1,000 ms, and the Sender pool's send time grows to match.

Average send: 0.8 x 1,000 + 0.1875 x 40 + 0.0125 x 100 + 0.5 = 809.25 ms, so about 810 ms.

A task now sends 50 / 0.81 = 61.7 a second. Auto-scaling stops at 40 tasks, so the pool sends at most 2,469 a second.

Press 10,000 on Stage 4. The Sender pool saturates.

Push is 80% of messages, but it now takes 800 of every 810 ms of the pool's time. That is 99%.

Email and SMS are healthy, yet only a quarter of their traffic gets through. Sign-in codes fall from 125 to 31 a second, and emails from 1,875 to 463.

The latency tile still shows 3ms. Producers notice nothing, because the accept never waits for a provider.

Read the stage's warnings. The Send queue warning says the backlog grows by 7.5k/s.

The cost tile shows $1,136k/mo. That is not a saving: providers bill what they send, and they now send a quarter as much.

We reason about time by hand. Say the slowdown lasts 10 minutes, and the queue runs roughly first in, first out.

The backlog grows by 10,000 - 2,469 = 7,531 a second. After 10 minutes, 7,531 x 600 = 4.5 million wait: 3.6 million pushes, 847,000 emails and 56,000 codes.

Every channel moves at 2,469 / 10,000 = 24.7% of its rate. A message sent t seconds in waits 7,531 / 2,469 x t = 3.05 t.

That holds while it goes out before the slowdown ends, so t + 3.05 t <= 600, and t <= 148 seconds.

At 148 seconds, 1.1 million are ahead, and 1.1 million / 2,469 = 451 seconds. That is the longest wait, about 7.5 minutes.

Sign-in codes usually expire in 5 to 10 minutes, so many arrive dead.

When FCM recovers, the pool has room, but each provider still sets the pace for its own channel.

Pushes clear at the 10,000 - 8,000 = 2,000 a second FCM has spare: 3.6 million / 2,000 = 1,800 seconds, about 30 minutes.

Emails clear at 4,000 - 1,875 = 2,125 a second spare, in about 7 minutes. Most waiting codes are past their 5-minute validity, so we drop them.

Press 2,000. Nothing saturates: the shared pool keeps up only while the load is under 2,469.

Step through the figure to see the slow pushes fill the slots.

The trade-off:

A shared pool is cheap on a normal day. On a bad day, the slowest provider sets the pace for every channel.

One slow provider, one shared pool

1. On a normal day, a send takes about 50 ms. One task with 50 slots sends 1,000 a second.
2. Push is 80% of messages, but it now takes 99% of the pool's time.
3. The pool sends 2,469 a second, so codes and email get a quarter through. After 10 minutes, 4.5 million messages wait.
4. With one lane per channel, slow pushes hold only push slots. Codes and email do not wait.

Stage 4: Push calls take 1 second, one shared sender pool

• Load: 10,000 req/sec
• First to saturate: Sender pool
• Estimated cost: $1,136k/mo

• Send queue: Sender pool drains 2.5k/s of the 10.0k/s offered, so the backlog here grows by 7.5k/s - about 27.1M messages after an hour.
• Sender pool: Scaled to the 40-task ceiling and 100% used at 10000 RPS.
• SMS provider (Twilio): SMS is billed per message: about $1,010,000/month at 30.9 msg/s and $0.0126 a message.

Check in: Push is 80% of messages. Say what share of the pool's time it takes now, and show the working.

One queue and one sender per channel

A user waits at a login screen for each sign-in code. Codes must not wait behind slow pushes.

The API reads settings, picks the channel, and puts the message on that channel's queue. Each queue has its own sender, and we call each pair a lane.

Each lane is sized to its provider. The Push sender has 50 slots per task and a 50 ms send on a normal day: 1,000 a second per task.

It runs at most 10 tasks: 10 x 1,000 = 10,000, the FCM quota. The lane can never offer FCM more than FCM accepts.

Today a push takes 1 second, so 10 tasks send 10 x 50 / 1 = 500 a second.

The Email sender has 50 slots and a 40 ms send: 1,250 a second per task. 1,875 / (1,250 x 0.7) = 2.1, so 3 tasks run.

The SMS sender is paced to the short codes: 2 tasks x 10 slots / 0.1 s = 200 a second. That is exactly what two short codes accept.

When more than 200 codes arrive a second, they wait in our queue, where we can see them and drop expired ones.

Its auto-scaling is off. More tasks would only send faster than the codes accept.

Press 10,000 on Stage 5. The Push sender saturates, and its card is red. That is the design working: the push lane waits, and the others do not.

Email and SMS now get all their traffic through. Push is late either way, and nothing else is.

Read the stage's warnings. The backlog now grows in the Push queue only, by 7.5k/s.

The cost tile shows $4,576k/mo, the same as Stage 3 to within a dollar. Isolation costs almost nothing.

Press 500. Nothing saturates. Today the push lane keeps up only while the whole load is under 625, because 0.8 x 625 = 500.

The trade-off:

A capped lane drains slowly. After a 10-minute slowdown, 4.5 million pushes wait, and FCM has 2,000 a second spare: 37.5 minutes.

That is longer than the shared pool's 30 minutes. The lane sent 500 pushes a second, the pool 1,975, so its backlog is bigger: 4.5 million against 3.6 million.

The trade is deliberate: pushes recover 7.5 minutes later, and codes and email are never late.

A time-to-live on each push drops the ones that are no longer worth sending.

A collapse key lets a phone show one "3 new messages" instead of three pushes. We also run three queues and three senders instead of one.

Stage 5: One queue and one sender per channel

• Load: 10,000 req/sec
• First to saturate: Push sender
• Estimated cost: $4,576k/mo

• Push queue: Push sender drains 500/s of the 8.0k/s offered, so the backlog here grows by 7.5k/s - about 27.0M messages after an hour.
• Push sender: Scaled to the 10-task ceiling and 100% used at 8000 RPS.
• SMS provider (Twilio): SMS is billed per message: about $4,084,400/month at 125 msg/s and $0.0126 a message.

Check in: Why do sign-in codes still arrive on time? Name the part that protects them.

A campaign in the everyday lane

The interviewer asks: Marketing wants to send one push to every user who allows it. What breaks?

We assume 10 million users allow marketing pushes, 10% of our 100 million. FCM is healthy again.

A Campaign job reads that list. For each user, it reads settings, then puts one push on the Push queue.

Run naively, it goes as fast as everyday traffic: 10,000 users a second. It ends after 10,000,000 / 10,000 = 1,000 seconds, about 17 minutes.

Why so fast? A job with no rate limit runs as fast as its reads come back. Its author aimed for 10 million users in under 20 minutes.

The exact rate does not matter. The push lane has 10,000 - 8,000 = 2,000 a second spare, and User settings has 14,400 - 10,000 = 4,400 reads spare.

So any naive rate over 2,000 builds a push backlog, and over 4,400 it also throttles settings.

The job spends about 0.25 ms of CPU per user, so a task handles 4,000 a second. 4 tasks handle 16,000, and run at 62.5%.

Press 10,000 on Stage 6. The Push sender and User settings both saturate. The FCM card is red at 100%, but the result does not name it.

That FCM card is the lane from Stage 5 doing its job: it gives FCM exactly its quota. The cost of it is the queue.

User settings now gets 20,000 reads a second. That needs 20,000 x 0.5 = 10,000 RCU, against 7,200.

The API needs those reads to accept anything. So while the job runs, producer requests can fail, and the latency tile shows "≥ 3ms".

The Push queue gets 18,000 a second, and the Push sender sends 10,000.

Read the stage's warnings. The Push queue backlog grows by 8.0k/s.

The warning gives the backlog after an hour at this rate. Our job stops after 1,000 seconds, so we work out the peak by hand.

The backlog peaks at 8,000 x 1,000 = 8 million pushes. A reply push sent as the job ends waits 8,000,000 / 10,000 = 800 seconds, about 13 minutes.

After the job, the lane has 2,000 a second to spare, so 8 million take 4,000 seconds to clear.

So everyday pushes run late for 1,000 + 4,000 = 5,000 seconds, about 83 minutes.

Press 6,000. A preset scales the whole load, campaign included. The push lane breaks first: 1.8 x 6,000 = 10,800 pushes, against 10,000.

User settings holds at 6,000: 12,000 reads need 6,000 RCU, 83% of 7,200.

The trade-off:

The campaign costs almost nothing in money, because push is free. It costs time: everyday pushes run late, and accepts can fail.

Stage 6: A campaign to 10 million users shares the push lane

• Load: 10,000 req/sec
• First to saturate: Push sender
• Estimated cost: $4,579k/mo

• Latency past saturation is a lower bound, not an estimate. User settings is saturated on the critical path, where the queueing model stops tracking a real queue - the true figure is higher by an amount this model cannot know. Relieve the bottleneck to get a latency number worth reading.
• User settings: Read capacity 139% used (provisioned): 10000 of 7200 RCU/s at 20000 RPS.
• Push queue: Push sender drains 10.0k/s of the 18.0k/s offered, so the backlog here grows by 8.0k/s - about 28.8M messages after an hour.
• Push sender: Scaled to the 10-task ceiling and 100% used at 18000 RPS.
• SMS provider (Twilio): SMS is billed per message: about $4,084,400/month at 125 msg/s and $0.0126 a message.

Check in: The FCM card shows 100%, but the result does not name it. Where does the campaign wait, and who waits with it?

Give the campaign its own paced lane

Everyday pushes must not wait behind marketing, and the API must keep its settings reads. The campaign can wait for hours.

The Campaign job now writes to its own Marketing queue and reads nothing. Writing is cheap, so it still ends in about 17 minutes.

A Marketing sender reads each user's settings at send time, then calls FCM. So we check an opt-out when it is freshest.

We pace the lane from the quota that remains. FCM takes 10,000 a second, and it counts the quota per minute.

We keep 15% free for bursts of everyday pushes: 10,000 x 0.85 = 8,500. Everyday push uses 8,000 at peak, which leaves 500 a second.

A marketing send takes 52 ms: a 2 ms settings read, then a 50 ms FCM call. With 13 slots, a task sends 13 / 0.052 = 250 a second.

So 2 tasks send 500 a second. Auto-scaling is off, because the pace is the point.

Press 10,000 on Stage 7. The Marketing sender saturates, and its card is red. That is the design working: the marketing lane waits, and the others do not.

The latency tile shows 3ms again, with no "≥". FCM gets 8,500 a second, 85% of its quota, and the Push queue does not grow.

Read the stage's warnings. The Marketing queue backlog grows by 9.5k/s, and again the job stops after 1,000 seconds.

So the backlog peaks at 9.5 million. At 500 a second, the campaign takes 10,000,000 / 500 = 20,000 seconds, about 5.6 hours.

At an average hour, everyday push is 8,000 / 2.2 = 3,636 a second. So 8,500 - 3,636 = 4,864 a second is free.

The campaign then takes 10,000,000 / 4,864 = 2,056 seconds, about 34 minutes. So we run campaigns at night, by time zone.

For a big launch, we ask Firebase Support for a temporary quota increase at least 15 days ahead. Google allows two a year.

FCM also asks senders to ramp up over at least 60 seconds, and not to start on the hour or the quarter hour.

Press 12,000. FCM saturates too: everyday push grows to 9,600, plus the fixed 500. That is 10,100 against 10,000.

A fixed pace must follow what remains of the quota. When everyday push grows, the marketing pace must shrink.

Step through the figure to follow the campaign through its lane.

The trade-off:

The campaign now takes hours, not 17 minutes. Nobody is waiting for it, so we accept that.

We add one queue and one sender. A fixed pace also needs a person, or a job, to lower it as everyday traffic grows.

The campaign in its own paced lane

1. FCM takes 10,000 a second. We keep 15% free, and everyday push uses 8,000, so 500 remains.
2. The job enqueues 10 million messages in about 17 minutes. They wait in their own queue, and everyday pushes do not.
3. At peak the campaign takes 5.6 hours. At an average hour, about 4,860 a second is free, and it takes 34 minutes.

Stage 7: The campaign in its own paced lane

• Load: 10,000 req/sec
• First to saturate: Marketing sender
• Estimated cost: $4,579k/mo

• Marketing queue: Marketing sender drains 500/s of the 10.0k/s offered, so the backlog here grows by 9.5k/s - about 34.2M messages after an hour.
• Marketing sender: Fargate 1vCPU-2GB ×2 at 100% of a fixed 2-task ceiling.
• SMS provider (Twilio): SMS is billed per message: about $4,084,400/month at 125 msg/s and $0.0126 a message.

Check in: How long does the campaign take at peak? Give two ways to make it shorter, with a number for each.

The API and the data model

The interviewer asks: Walk me through the API and the data model.

None of these parts breaks under load, so no stage is about them. An interviewer asks about each one, so we answer each in a line or two.

Send: POST /v1/notifications with the user, the type, the template, its data and a priority. It returns 202 Accepted with a notification ID.

The status in that answer is "queued", or "suppressed" if the user opted out.

The producer sends an idempotency key in a header: one ID per notification, sent again on every retry.

A retry with the same key gets the first result back, not a second notification. The same key with a different body gets 409.

Send also returns 400 for bad template data, 404 for an unknown user, and 429 when a producer goes over its own quota.

A batch call takes up to 100 notifications and returns 207 Multi-Status, with one result each.

Status: GET /v1/notifications/{id} returns queued, sent, delivered, failed, suppressed or expired, with the attempts and times.

Settings: GET and PUT /v1/users/{id}/preferences hold opt-ins per type and channel, quiet hours and a time zone.

Devices: PUT /v1/users/{id}/devices/{device_id} stores a platform and a push token, and DELETE removes it.

Campaigns: POST /v1/campaigns with a segment, a template and a send window returns 202 with a campaign ID.

Data: User settings is keyed by user ID. Its item holds opt-ins, quiet hours, time zone, locale, email, phone and devices.

The Notification log is keyed by the producer and its idempotency key. It keeps user, type, channel, status, attempts and times for 30 days.

Each notification makes three writes to it: accept, claim and result. Each is a conditional write, which succeeds only if the item is in the state we expect.

At 10,000 a second, that is 3 x 10,000 / 0.7 = 42,857 write capacity units, about $20,000 a month. That is more than every server on this page.

It is also past the 40,000 units a table gets by default, so it is one more quota to ask AWS for.

A cheaper choice keeps only the idempotency item: one write with a 24-hour expiry, about $6,700 a month. Status events then go to a separate event log.

A dead token: FCM answers 404 UNREGISTERED. We delete the token and do not retry.

Templates have versions and a locale each. 1,000 templates x 5 locales x 5 KB = 25 MB, so each API task holds them in memory.

We render at accept time, so a template change never alters a message already in a queue.

The trade-off:

Each choice has a price. Three conditional writes per notification cost more than all our servers, so we say which writes we would keep.

Check in: A producer times out and sends the same request again. Say what stops a second notification, and what a reused key with a new body gets.

Say this in 45 minutes

The interviewer asks: What fails in this design, and what does a user see?

A design that works on a good day is half an answer. The interviewer asks what fails, what a user sees, and how late a message can be.

Durability: the API answers 202 only after the queue confirms it has the message. The queue stores it in several Availability Zones before it confirms.

Retries: the queue is the retry timer. The sender hides a failed message for 10, 20, 40, 80, then 160 seconds, which is exponential backoff.

Each wait gets random jitter of plus or minus 50%. After 5 tries, about 5 minutes, the message moves to a dead-letter queue for failures.

Without backoff, a 10-minute FCM outage piles up 8,000 x 600 = 4.8 million pushes. Retried every 30 seconds, they ask for 4,800,000 / 30 = 160,000 calls a second against 10,000.

FCM's 429 means wait for its retry-after time, or 60 seconds. A 400 or a 404 means never retry.

Duplicates: the sender claims the log item with a conditional write, so a second copy fails and is deleted. Exactly once to a phone is not possible.

Status: SES publishes delivery events, and Twilio calls us back per message. FCM only confirms it accepted a push, so "sent" is the last honest status, unless the app reports an open.

Per user: at most 10 everyday pushes an hour, then one collapsed push. At most 1 marketing push a day, and 5 codes per phone an hour.

The code limit stops SMS pumping, where attackers request codes to premium numbers and we pay. Quiet hours run 22:00 to 08:00 in the user's time zone, except codes and security alerts.

Failover: a second SMS provider needs its own 2 short codes, $2,000 a month. Email moves to SES in a second Region, and push holds its queue.

Scheduled sends: if 1 million users pick 9:00, a scheduler that fires 1,000 a second sends the last one about 17 minutes late.

At 100,000 a second, every wall is a quota: FCM 80,000, SES 18,750, 50,000 read units, and 13 short codes. We ask for each one months ahead.

Minutes 0 to 5: requirements. Channels, settings, campaigns and status. Ask the scale, and which messages are urgent.

Minutes 5 to 10: the numbers. 4 notifications a user a day gives 1,000 a second: 80% push, 18.75% email, 1.25% SMS.

Minutes 10 to 15: the API with 202 and an idempotency key. The data model: settings, the log and tokens.

Minutes 15 to 22: accept, queue, send. Give each provider's quota, and say that SMS is the bill.

Minutes 22 to 28: ten times the load. Three walls, three lead times, and FCM at 80% next.

Minutes 28 to 34: a slow provider. One queue and one sender per channel, each capped at its quota.

Minutes 34 to 40: a campaign. Its own lane, paced from the quota that remains, with opt-out checked at send time.

Minutes 40 to 45: retries and backoff, duplicates, and failover.

The trade-off:

Each answer to a failure has a price. The second SMS provider costs $2,000 a month for traffic it may never carry.

Backoff makes a failed message later, and a time-to-live drops some late pushes. We say both before the interviewer asks.

Check in: Set a timer for 45 minutes. Give the whole answer out loud, and check it against the pacing plan.