How Apache Pinot works

Built for the other kind of analytics

A warehouse is built for a few heavy queries from analysts. Pinot is built for the opposite: thousands of small queries per second from a product feature, answered in tens of milliseconds, over data that arrived from Kafka seconds ago. Think "who viewed your profile this week" rendered inside the app, for every user at once.

Brokers scatter, servers scan

Queries hit a broker. The broker works out which servers hold the segments the query needs, scatters the request to them, and merges what comes back. The servers hold segments on local disk and do the actual scanning. Controllers and Zookeeper keep the map of who holds what.

The published sizing guidance is roughly 1,000 queries per second for a 16 core broker on standard analytical queries, and about the same for a 16 core server in the good case. So you scale brokers with QPS and servers with QPS, data size and ingestion together.

Replica groups: ask fewer servers

Replica group routing arranges servers into groups where each group holds a complete copy of the table. A query is then routed to one group instead of the whole cluster: fan-out drops, the per-server overhead a query pays drops with it, and fewer nodes get the chance to be the slow one.

⚓ On the SysPad node, raise Replica groups and watch throughput climb while latency climbs too. If both moved the same direction, the model would be lying to you.

What else narrows fan-out besides replica groups?

Partitioning the table on the same key the stream is partitioned by lets the broker prune to the servers that can possibly hold matching rows, so a query filtered by that key touches a fraction of the group. Combined with replica groups, a per-user lookup can go from "all 40 servers" to "one server", which is how Pinot reaches the QPS numbers it advertises.

Ingestion runs on the same machines

Real-time tables consume Kafka on the servers themselves: consuming partitions, building segments in memory, then committing them. Those cores are the same cores answering queries, and every replica consumes independently, so a replication factor of 3 means the stream is read and indexed three times over.

⚓ Push Stream ingestion up on the SysPad node and the query ceiling falls even though the server count never changed. A Pinot cluster can be CPU rich and query poor.