How Anthropic (Claude) works

Stateless, token-billed (the LLM basics)

Like any LLM API, Claude is stateless: it remembers nothing between calls, so your app resends the conversation each time. Text is split into tokens, and you’re billed per input token plus per outputtoken, with output priced higher than input.

The context window caps the tokens per request. Claude’s flagship models offer very large windows (up to ~1M tokens), so you can feed in big documents, but more tokens still mean more cost and latency.

Choosing a model tier

Claude comes in tiers that trade capability against cost and speed: Haiku (fastest, cheapest), Sonnet (balanced), Opus (most capable of the everyday tiers), and the frontier family for the hardest reasoning and long-horizon agentic work. Higher tiers cost more per token and are a bit slower.

⚓ Chat assistants, RAG over your data, extraction/classification, coding agents, and tool-using workflows.

Prompt caching: the cost lever

Many apps send a large, unchanging chunk every call, a long system prompt, instructions, retrieved documents, few-shot examples. Prompt caching lets you mark that stable prefix so it’s processed once and reused: subsequent calls read it at roughly a tenth of the input price, and faster.

How would you verify caching is actually working?

The API response reports cache usage: cache_read_input_tokens (served from cache, cheap) versus cache_creation_input_tokens (written) and plain input_tokens (full price). If cache-read stays zero across identical-prefix requests, something is silently changing the prefix, often a timestamp or non-deterministic JSON ordering.

Tool use and streaming

Claude can call tools you define: it returns a structured request to run a function, your app executes it and passes the result back, and the model continues. This is how an LLM reaches your APIs, databases, and the outside world, the basis of agents.

Responses generate token by token, so you stream them to show output immediately and to avoid request timeouts on long generations.

A dependency with rate limits

The API enforces rate limits (requests and tokens per minute) and returns HTTP 429 when exceeded; it can also be slow or briefly overloaded. Treat it like any remote dependency: retry with backoff, set timeouts, stream long responses, cache where you can, and often process asynchronously so a spike doesn’t block users.