SysPad › Examples › Serverless data pipeline
Serverless data pipeline
IoT-style ingestion: HTTP API to a buffering Lambda, a Kinesis stream, a processor Lambda writing to DynamoDB + an S3 lake, with Athena queries and a Glue catalog.
A fully serverless, event-driven pipeline with async boundaries and the analytics tier.
- Throughput: 25,000 req/sec
- First to saturate: Ingest API
- Estimated cost: $363k/mo
How it works
- Event Stream
- The ingest API returns once events land here; every step after runs async off the stream.
Request flows
Device event ingestion
- Ingest API
- Ingest Lambda
- Event Stream
- Processor
- Device State
- Data Lake
Scheduled crawl & query
- Data Lake
- Glue Catalog
- Athena Queries