Building production AI applications requires moving beyond "back-of-the-envelope" math. Between asymmetrical input/output pricing, system prompt caching, and context window explosion, estimating API costs accurately is vital for unit economics.
1. The Core LLM Token Pricing Equation
Every LLM API call breaks down into input tokens (prompt context, system instructions, retrieval context) and output tokens (completion generated by the model). Output tokens typically cost 3x to 5x more than input tokens due to the autoregressive compute overhead.
Total Cost = (Input Tokens · Rate_in) + (Cached Tokens · Rate_cache) + (Output Tokens · Rate_out)
Where:
• Rate_in = Cost per 1M prompt tokens
• Rate_cache = Cost per 1M cached prompt tokens (typically 75-90% discount)
• Rate_out = Cost per 1M completion tokens2. Heuristics: Converting Words and Code to Tokens
When budgeting before telemetry is gathered, use these verified token density multipliers across standard tokenizers (cl100k_base / o200k_base):
| Content Type | Word-to-Token Multiplier | Rule of Thumb |
|---|---|---|
| Standard English Text | 1 word ≈ 1.33 tokens | 750 words ≈ 1,000 tokens |
| Technical Docs / Math | 1 word ≈ 1.6 to 1.8 tokens | 550 words ≈ 1,000 tokens |
| Source Code (Python/JS) | 1 line ≈ 10 to 14 tokens | 100 lines ≈ 1,200 tokens |
| Structured JSON Payloads | 1 JSON key-value ≈ 4 to 8 tokens | Heavy delimiter overhead |
3. Prompt Caching Economics and Impact
Modern providers (Anthropic, DeepSeek, OpenAI) offer prompt caching that discounts recurring system prompts by up to 90%. For RAG applications with a static 20k token document corpus and 500 daily queries, prompt caching can slash monthly API bills from $450 down to $72.
Frequently Asked Questions
- Why are output tokens more expensive than input tokens?
- Input tokens are processed in parallel through matrix multiplication in the GPU tensor cores, whereas output tokens must be generated sequentially one token at a time (autoregressive decoding), consuming dedicated memory bandwidth for every step.
- How does prompt caching affect latency?
- Prompt caching not only slashes costs by up to 90% but also cuts Time-to-First-Token (TTFT) by up to 80% because the model does not need to re-encode the cached prefix.