How to Reduce AI API Costs: A Builder's Practical Guide

To reduce AI API costs, do four things in this order: measure what each call actually costs, cache the part of your prompt that never changes, send anything that isn't urgent through a batch endpoint, and stop using your biggest model for easy jobs. None of it is exotic. Most of it is a few lines of config.

The AI editor that wrote your app didn't think about your bill. It will happily stuff the whole codebase into every request. This guide covers what to change once the app is live and the invoices are real.

Step 1: Find out where the money goes

You can't trim what you can't see. Before you touch a prompt, log the token counts from every response. Every major API returns a usage object, and that's your ground truth.

Log at least these per request:

• Input tokens, output tokens, and cached tokens (if the provider reports them)
• Which model and which feature made the call
• A user or session ID, so one noisy user doesn't hide in the average

Then sort by total cost per feature. You'll usually find one or two endpoints eating most of the budget. Fix those first and ignore the rest.

One thing worth knowing: output tokens usually cost more than input tokens, and for reasoning models the hidden "thinking" tokens count as output. One overview of OpenAI API pricing lists output tokens (reasoning included), model choice, tool calls and very long context as the biggest levers. Check the current price sheet for your model rather than trusting a blog's numbers, mine included.

Step 2: Turn on prompt caching

If your requests start with the same long block (system prompt, tool definitions, a big reference doc), you're paying full price to re-process it every time. Prompt caching fixes that.

OpenAI: mostly automatic

OpenAI's prompt caching guide says caching reuses work when requests share the same prompt prefix. The minimum cacheable prefix is 1,024 tokens. Cache reads are billed at 0.1× the uncached input price on the models the table lists, and some are lower. You don't change your code to get the discount. You do need to structure your prompts so the stable part comes first.

Anthropic: opt in with cache_control

Claude's caching is explicit. You mark the stable chunk with a cache_control breakpoint. According to the Claude pricing docs, a 5-minute cache write costs 1.25× the base input price, and a 1-hour write costs 2×. A cache hit costs 0.1× on most models, and some models are cheaper still. The same page says the 5-minute cache pays for itself after one read, and the 1-hour cache after two.

That write premium is why caching can backfire on sporadic traffic. If a prompt gets called once an hour, a 5-minute cache just charges you extra to write something nobody reads. Caching works when the same prefix is hit repeatedly within the window.

The rule that matters most: stable first, changing last

Caches match on the beginning of the prompt. So put things in this order:

1. System instructions
2. Tool definitions
3. Reference docs or examples
4. Conversation history
5. The new user message

A timestamp or user name at the top of your system prompt breaks the cache for everything after it. AI-generated code does this all the time. Move that stuff to the bottom.

Then check it works. Look for the cached-token field in the usage data. If it's always zero, something upstream of your prompt is changing between calls.

Step 3: Use batch for anything that can wait

Not everything needs an answer in two seconds. Nightly summaries, tagging a backlog, generating descriptions, running evals: all of that can wait.

OpenAI's Batch API offers 50% lower costs than the synchronous API, a separate pool of higher rate limits, and a 24-hour turnaround. Anthropic's Message Batches API advertises the same 50% discount, according to this comparison, which also covers Google's Gemini batch mode. Verify the model and region you use, since the comparison itself says to.

The trade-off is that results can take up to 24 hours. Don't put a user-facing chat behind it. But if it's a cron job, it should probably be a batch.

Anthropic's docs also say caching multipliers stack with the batch discount, so a batch job with a shared prompt prefix can get both.

Step 4: Match the model to the job

This is the biggest saving, and the one people skip because the big model "just works."

Split your calls into tiers:

• Cheap and small: classification, extraction, routing, reformatting, short rewrites
• Mid-tier: most chat, summarizing, standard code edits
• Top model: hard reasoning, long agent runs, anything where a wrong answer is expensive

Test it properly. Pull 30 to 50 real inputs from your logs, run them through the smaller model, and read the outputs side by side. If you can't tell the difference on the task, you've found free money. If it fails on 1 in 10, keep the big model for that path, or add a fallback: try the small model first and escalate only when a simple check fails.

Step 5: Shrink what you send and what you get back

Once the structure is right, trim the fat.

• Cap output. Set max_tokens (or your provider's equivalent) to a sane limit for each endpoint. A runaway response costs real money.
• Ask for short formats. "Return JSON with these three fields" is cheaper than a polite paragraph around the same data.
• Trim history. Don't resend 40 turns of chat. Keep the last few turns and a short summary. OpenAI's caching guide notes that compacting can still save money even when your cache-hit rate drops, since fewer input tokens is cheaper by itself.
• Retrieve, don't dump. Send the three relevant chunks, not the whole document.
• Limit tool output. A tool that returns a 10,000-line log into the model context is a quiet budget killer.

Also look at your retry logic. An agent loop that retries on every error, with the full context each time, can multiply a single failure into a pile of charges.

Step 6: Put guardrails around spend

Efficiency won't save you from a bug or a bot hammering your endpoint. Add basic limits:

• Per-user daily request or token caps in your own backend
• Rate limiting on any public endpoint that calls a model
• A spending limit or usage alert in your provider's dashboard, if it offers one. Check what yours supports.
• Never ship an API key in client-side code. Call the model from a server route.

These matter most for apps built fast with AI tools, where the generated code may call the API straight from the browser and have no limits at all.

Quick comparison

Lever	What it does	Effort	Best for
Prompt caching	Discounts repeated prompt prefixes	Low (reorder prompt; add a marker on Claude)	Long system prompts, agents, chat
Batch API	Half-price, up to 24h turnaround	Medium (async flow)	Backlogs, evals, nightly jobs
Smaller model	Lower per-token price	Medium (needs testing)	Classification, extraction, routing
Output caps and trimming	Fewer tokens in and out	Low	Everything

A prompt to paste into your editor

Drop this into Cursor, Claude Code or whichever agent you use, with your repo open:

Audit this codebase for LLM API cost problems. Do not change code yet.

1. List every place we call a model API (file, function, model used).
2. For each call, tell me: is the stable content (system prompt, tools, docs)
   at the START of the prompt, or is something that changes per request
   (timestamps, user data) breaking prefix caching?
3. Flag calls with no max output token limit.
4. Flag calls that resend full chat history or whole documents.
5. Flag calls that could use a smaller model or a batch endpoint.
6. Flag any API key used in client-side code.

Output a table sorted by likely cost impact, then propose the three
smallest changes with the biggest savings.

Read the answer critically. The agent can spot patterns, but it can't see your real traffic. Check its guesses against your logs from Step 1.

A sensible order of attack

1. Add logging for tokens, cost and feature.
2. Reorder prompts so the stable part leads, then confirm cached tokens show up.
3. Cap output length everywhere.
4. Move non-urgent jobs to batch.
5. Test a smaller model on your cheapest-to-fail tasks.
6. Add per-user limits and a dashboard alert.

Rerun your logs after each step. If a change doesn't move the number, undo it and move on.

FAQ

What's the fastest way to reduce AI API costs?

Fix prompt order and turn on caching. If your app has a long system prompt, you probably pay full price for it on every call. Put the stable content first, check that cached tokens show up in your usage data, and you'll see the effect on the next invoice.

Does prompt caching hurt quality?

No. Caching reuses the processing of text the provider has already seen, so it changes cost and speed, not the model's answer. The risk is financial: on Claude, a cache write costs more than a normal read of the same input, so caching a prompt you rarely reuse can cost more than skipping it.

Should I switch to a cheaper model for everything?

No. Switch task by task and test on real inputs from your logs. Simple, repetitive jobs like tagging and extraction are good candidates. Keep the stronger model for work where mistakes are costly, and consider a "try cheap, escalate if it fails" setup for the gray area.

Cover photo: Christopher Gower on Unsplash