← All news and guides
Engineering

AI Spend Audit: Finding Waste in LLM, GPU and AI SaaS Bills

How to audit AI spend: inventory every bill, tie cost to features, cut LLM waste, right-size GPUs, trim seats and review vector stores and logs.

Mono Team ·

A dim data hall with rows of dark server racks, a single rack in the middle distance glowing with amber indicator lights, shallow depth of field.

Short answer: an AI spend audit lists every place your company pays for AI, ties each charge to a feature, team or customer, and then checks four things: what you pay per unit, how much of what you bought is used, what runs when nobody needs it, and who owns each line. The waste usually sits in a few places: model calls that resend context or loop, a premium model doing easy work, GPUs left on around the clock, AI seats nobody opens, and logs and indexes nobody reads. You find it with an inventory, a cost-per-feature view and a short list of checks. The fixes come after, in order of money saved against risk.

This guide covers the whole AI bill, not only token costs. It adds GPUs, AI software seats and the supporting infrastructure (vector stores, tracing and test environments). For the mechanics of token growth in one product, read LLM bill shock. Prices below are public list prices checked on 20 November 2026, in US dollars per 1 million tokens unless stated, ex-tax. They change often, so recheck before you act. Worked numbers are illustrations, not benchmarks.

What is an AI spend audit, and what should it cover?

An AI spend audit is a structured review of everything you pay for that involves AI. Most companies have four separate bills, owned by four different people, and nobody sees the total.

BillWhat it containsWho usually owns itTypical blind spot
Model APIsWhat it contains: Tokens billed by OpenAI, Anthropic, Google or a cloud marketplaceWho usually owns it: EngineeringTypical blind spot: No split by feature or customer, so one invoice line hides many products
GPUs and private modelsWhat it contains: Rented or owned GPU servers running open-weight modelsWho usually owns it: Platform or ML teamTypical blind spot: Hours billed against hours used
AI software seatsWhat it contains: Copilot, coding assistants, meeting tools, writing tools, chat subscriptionsWho usually owns it: IT, finance or each team leadTypical blind spot: Cards on expense reports that IT never sees
Supporting infrastructureWhat it contains: Vector databases, embedding jobs, tracing tools, logs, staging environmentsWho usually owns it: EngineeringTypical blind spot: Treated as part of general cloud spend, never reviewed

An AI infrastructure assessment is the technical half of the same exercise. It looks at the same four bills from the system side: how the GPUs are used, how the vector store is sized, what the tracing tool captures and how requests reach the models.

Step 1: How do you inventory every AI bill?

A neat stack of unopened plain white envelopes held by a steel clip on a wooden desk beside a pair of reading glasses, soft window light, shallow depth of field.

Start with money, not code. Pull twelve months of charges from five sources and put them in one sheet.

  • Model vendor invoices and dashboards. Include every account, not only the company one. Developers often start with a personal key and a company card.
  • Cloud invoices. Models bought through a cloud marketplace appear on the cloud bill, not the model vendor's. GPU instances, managed vector databases and storage sit here too.
  • Card and expense data. Search for the names of AI tools and for small recurring charges. Link what you find to your shadow AI audit, which covers the data side of the same problem.
  • Your identity provider. The list of apps your staff sign in to shows paid seats that finance may not know about.
  • Contracts and renewals. Note minimum commitments, committed-use discounts and the date each one renews.

Give each line an owner, a monthly cost, a unit (tokens, GPU hours, seats, GB stored) and a renewal date. Any line with no owner is your first finding.

Step 2: How do you tie AI cost to a feature, team or customer?

An invoice gives you a total. It cannot tell you which feature or customer caused it. You need your own record of each model call, and it should come from one place in your code so that nothing skips it. Log these fields:

  • feature or workflow name, and the environment (production, staging, test)
  • customer or tenant ID, and a hashed user ID if you need one
  • provider, model and model version
  • input tokens, output tokens and cached input tokens, as the provider reports them
  • latency, retry count, and whether the call was part of an agent loop
  • a request ID that links calls belonging to one user action

The OpenTelemetry generative AI conventions (open-telemetry/semantic-conventions-genai) already name most of these fields, such as `gen_ai.request.model` and `gen_ai.usage.input_tokens`. Treat them as an emerging standard: their status is Development, not stable. Prompt and response content is opt-in in those conventions, which is a sound default for a cost audit too: record usage and leave the text out.

Then price the tokens and build three views: cost per feature, cost per customer and cost per completed task. The last one matters most. A cheap model that fails and retries can cost more per finished task than an expensive one that succeeds first time. Our guide to LLM observability setup covers the tooling choices.

Step 3: Where is the waste in LLM API spend?

Use this table to match what the data shows to a cause.

Waste patternWhat it looks like in the dataFirst fix
Stable prefix billed in full every timeWhat it looks like in the data: Cached input tokens near zero while the system prompt and tool definitions are identical across callsFirst fix: Turn on prompt caching and put fixed content first
Premium model on easy workWhat it looks like in the data: Classification, extraction or routing calls on your largest modelFirst fix: Test a smaller model on a sample and compare cost per completed task
Oversized contextWhat it looks like in the data: Input tokens flat and large, even for short questionsFirst fix: Retrieve fewer chunks, rerank, summarise old chat turns
Agent loopsWhat it looks like in the data: Many calls per user action, with a long tail of very long runsFirst fix: Cap steps and token budget per task
RetriesWhat it looks like in the data: The same prompt billed more than onceFirst fix: Fix the cause of the failure and retry only the failed step
Real-time calls for batch workWhat it looks like in the data: Overnight reports and backfills running through the live endpointFirst fix: Move them to a batch API
Idle and test trafficWhat it looks like in the data: Staging, evaluation and developer calls billed on production keysFirst fix: Separate keys per environment, with budgets

Three price facts shape the order of fixes. Check them against your own provider's page, because they differ by model.

  • Output costs more than input. On every model in the OpenAI and Anthropic lists we checked on 20 November 2026, output is priced at five times input. A prompt change that lengthens answers can cost more than one that lengthens the prompt.
  • Model tier matters more than almost anything else. Anthropic's list shows Haiku 5.5 at $0.10 in and $0.50 out, Sonnet 5.5 at $2 and $10, and Opus 5.5 at $4 and $20. Sonnet is 20 times the price of Haiku, and Opus is 40 times. OpenAI's three tiers (GPT-6 Luna at $0.10 and $0.50, GPT-6.2 Sol at $2 and $10, GPT-6.2 Astra at $8 and $40) follow a similar pattern. Only your own evaluation can say whether a cheaper tier is good enough.
  • Discounts stack. Anthropic lists cache writes at 1.25 times base input for a 5-minute cache and 2 times for a 1-hour cache. Cache reads cost 0.1 times on most models and 0.05 times on Opus 5.5 and Sonnet 5.5. These stack with the Batch discount, which is 50% off at OpenAI and Anthropic for work that can wait. OpenAI also discounts cached input. US-only inference at Anthropic costs 1.1 times, so check whether a region setting is adding a premium you do not need.

A made-up example shows the size of the cache effect. Say a feature makes 10,000 calls a day, each with a 3,000-token system prompt and tool list that never changes. That is 30 million input tokens a day. At $2 per million, a Sonnet 5.5 price, the stable part costs $60 a day, or about $1,800 a month. If nearly every call reads that prefix from the cache at 0.05 times, it costs about $3 a day, or about $90 a month, plus the occasional cache write. Your real saving depends on your hit rate, so measure it. A prefix that is rarely reused can cost more with caching, because the write costs more than a normal read of the same tokens.

Price changes are part of the audit too. Google lists Gemini 4 Carbon at $2 in and $10 out through 30 November 2026, and $4 and $20 from 1 December 2026. Put each price change and renewal date on a calendar.

Step 4: Are your GPUs doing useful work, or just running?

A GPU bills for every hour it is switched on, whether or not it serves a request. That makes utilisation the first number to check on a private model. The public rates below are from E2E Networks, an Indian cloud provider (e2enetworks.com/pricing, checked 20 November 2026, per hour, ex-GST). The monthly figures assume 730 hours.

GPURate per hourPer month if left on (730 hours)
NVIDIA H100 80GB, on-demandRate per hour: ₹379.05Per month if left on (730 hours): about ₹2.77 lakh
NVIDIA H100 80GB, committed (3+ months)Rate per hour: ₹347.70Per month if left on (730 hours): about ₹2.54 lakh
NVIDIA H100 80GB, spotRate per hour: ₹70Per month if left on (730 hours): about ₹51,100
NVIDIA A100 80GBRate per hour: ₹189Per month if left on (730 hours): about ₹1.38 lakh

Three checks catch most GPU waste.

  • Hours on against hours used. A development H100 used for 10 hours on 22 working days is 220 hours, about ₹83,400 on demand. Left on all month, the same card costs about ₹2.77 lakh. Schedule non-production GPUs to stop outside working hours.
  • Real utilisation, not the headline number. Many tools report a GPU as busy whenever any work is running on it, even if most of the card sits unused. Look at requests served per hour, queue depth and memory use as well, and size the fleet against peak load, not average load.
  • The commitment trap. In this list, the committed rate is about 8% below on-demand. If a commitment bills every hour, it only beats on-demand when the card is busy more than about 92% of the time. Read your own contract before you commit.

India's IndiaAI compute programme offers subsidised rates to eligible entities. We have not verified the current rates, so check eligibility and prices on its portal. If you are weighing a private model against an API, private LLM vs API cost in rupees gives the full break-even model, and our Own Your AI service builds the private setup itself.

Step 5: Which AI software seats should you cut?

Rows of identical empty desks in a bright open-plan office, chairs pushed in, closed laptops on a few of them, morning light across the floor.

Seat waste is the easiest to find and the most often ignored. Pull usage from each vendor's admin console and from your identity provider, and ask three questions of each product.

  • Who has logged in, and when? A seat unused for 60 or 90 days is a candidate to remove. Pick the window before you look at the data.
  • Do two tools do the same job? Chat assistants, meeting recorders and writing tools overlap a lot.
  • Does the contract auto-renew, and at what price? Put the notice date in your calendar.

Low use is not always a reason to cut. Sometimes the tool is fine and people were never shown how to use it. If you bought Copilot seats and few people open them, read our 90-day adoption plan before you remove them. The sensible order is to measure real usage, fix permissions and training, then decide seat numbers at a set review date.

Seats are a data flow too. GitHub said on 25 March 2026 that from 24 April 2026, interaction data from Copilot Free, Pro and Pro+ users trains models unless the user opts out, and that Copilot Business and Enterprise users are not affected. Check each vendor's current terms before you assume a plan is safe for work data.

What else should an AI infrastructure assessment check?

The supporting systems often cost more than expected, and no one owns them. Check each one against a simple question.

ComponentWhat to checkCommon waste
Vector storeWhat to check: Size of the index, replica count, how often it is rebuilt, and whether stale documents are removedCommon waste: Re-embedding the whole corpus on every deploy; replicas sized for a peak that never comes; deleted documents still indexed
Tracing and logsWhat to check: What is captured per call, how long it is kept, and the pricing unitCommon waste: Full prompts and responses stored for every call, for longer than anyone reads them
Test environmentsWhat to check: Do staging and evaluation setups run all day and share production keys?Common waste: Idle staging servers and evaluation runs billed to production

Tracing tools have their own bills, and most price by volume of events or units captured, so capture settings drive the cost. Hosted plans also limit how long data stays available, and self-hosting swaps a subscription for servers, storage and staff time. Compare tools on your own call volume. Our LLM observability setup guide walks through one public price list and the hosted versus self-hosted choice.

Logs raise a data protection point as well as a cost point. A prompt log that holds personal data is personal data, and it needs a retention and erasure plan. Capture less by default, and see our guide to DPDP erasure automation for how to handle deletion across stores.

Step 6: How do you rank the findings and set owners?

Do not try to fix everything. Score each finding on three things: monthly saving, effort and quality risk. Fix the high-saving, low-risk items first.

Finding typeTypical effortQuality riskDo it
Remove unused seats and duplicate toolsTypical effort: LowQuality risk: Low; tell users firstDo it: First
Move offline work to a batch APITypical effort: Low to mediumQuality risk: None, if results can waitDo it: First
Turn on prompt caching and reorder promptsTypical effort: MediumQuality risk: LowDo it: Early
Cap agent steps, retries and output lengthTypical effort: MediumQuality risk: Medium; test with real tasksDo it: Early
Route easy tasks to a smaller modelTypical effort: Medium to highQuality risk: Medium; needs an evaluation setDo it: After you can measure quality

For each fix, write down the saving you expect, the quality check that proves nothing broke, and the person who owns it. Before you change a model, build a small evaluation set from real requests and compare cost per completed task, not price per token.

How do you keep AI spend under control after the audit?

  • Budgets and alerts. Set a monthly budget per feature and per environment, with an alert at a chosen percentage.
  • Per-customer limits. Give each tenant a token or request quota so one customer or script cannot run away with the bill.
  • A monthly review. Check cost per feature, cost per completed task, GPU utilisation and seat usage when the invoice lands.
  • A price calendar. Record vendor price changes, discount expiry and contract renewal dates.

If you would rather have an outside team do the first pass, Mono's Tech Audit reviews cloud and software spend with read-only access to your cloud accounts and billing, and flags idle resources, duplicate tools and unused seats as part of the same review. Mono does not set your budgets or certify your costs.

Sources: OpenAI API pricing (openai.com/api/pricing), Anthropic pricing (anthropic.com/pricing and platform.claude.com/docs/en/about-claude/pricing) and Google Gemini API pricing (ai.google.dev/gemini-api/docs/pricing), all checked 20 November 2026. E2E Networks GPU price list (e2enetworks.com/pricing), checked 20 November 2026. GitHub blog, "Updates to GitHub Copilot interaction data usage policy", 25 March 2026. OpenTelemetry generative AI semantic conventions (github.com/open-telemetry/semantic-conventions-genai), status Development. Last reviewed 20 November 2026.

FAQ

What is an AI spend audit?

An AI spend audit is a review of everything a company pays for that involves AI: model API tokens, GPU servers, AI software seats and the supporting systems such as vector stores, tracing tools and test environments. It ties each charge to an owner and a feature, then checks price per unit, utilisation, idle time and renewal terms to find waste.

Where does most LLM API waste come from?

The common causes are resending a large unchanged prompt on every call without caching, using a premium model for easy tasks, oversized retrieved context, agent loops and retries, long outputs, and real-time calls for work that could run in a batch. Output tokens cost more than input tokens, so long answers matter. The audit shows which cause applies to you by logging tokens per feature.

How do I know if my GPUs are underused?

Compare the hours a GPU is switched on with the hours it serves requests. Then look at requests per hour, queue depth and memory use, not only a single utilisation figure, and size capacity against peak load. A committed GPU rate that is only a few percent below on-demand pays off only if the card is busy almost all the time, so check the contract.

Should I cut AI seats that few people use?

Measure first. Pull login and feature data for a fixed window, such as 60 or 90 days, and ask whether low use comes from a poor tool or from missing training and permissions. Remove seats nobody has used and duplicate tools. For tools that may be worth keeping, run an adoption plan and set a review date before you decide seat numbers.

Does an AI spend audit need access to prompts and customer data?

No. Cost analysis needs usage records: model, token counts, feature, tenant ID, latency and retries. It does not need prompt or response text. Capture content only when you debug a problem, and keep it for as short a time as you can, because logs that hold personal data need their own retention and erasure plan.

Find out where your AI money is going.