Short answer: an architecture review of an AI product asks 30 questions in six groups: scope and ownership, data flow, the model layer, quality and observability, security, and cost and operations. A good answer to each is a written document, a config setting, a test result or a log, not a verbal reassurance. If the team cannot show evidence, mark the question red. Run the review before launch, before a big customer's security review, before a funding round, and whenever the model, the provider or the data changes.
This guide is for CTOs, engineering leads and founders who need to check an AI feature or product, their own or one they are about to buy or invest in. It is a checklist built from public standards and common failure points, not a certification, and it is not legal advice. The last section covers AI technical debt: how to find it and how to put a price on it.
How do you run an AI architecture review?
Keep it to one working session per group. For each question, ask the team to show the evidence, then score it:
- Green: the evidence exists, is current, and someone other than the author can find it.
- Amber: the practice exists but is undocumented, partial, or older than the last major change.
- Red: no evidence, or the answer is "we think so".
Record the owner and a date for every amber and red. Treat reds in the data flow and security groups as blockers for any enterprise sale. The tables below give each question and what a good answer looks like.
Is the scope clear, and who owns the feature? (questions 1 to 5)
| # | Question | What a good answer looks like |
|---|---|---|
| 1 | Question: What task or decision does the AI own, and what does a human still decide? | What a good answer looks like: A one-line scope in writing. High-impact outputs have a named human step. |
| 2 | Question: What is the worst realistic wrong output, and who is hurt by it? | What a good answer looks like: A short list of failure modes with severity, each linked to a control. |
| 3 | Question: Who is the named owner in production? | What a good answer looks like: One person or team accountable for quality, cost and incidents. |
| 4 | Question: What happens when the model is down, slow or wrong? | What a good answer looks like: Timeouts, a fallback path and a non-AI route, all tested. |
| 5 | Question: Is there a list of every AI feature, model and provider in use? | What a good answer looks like: An inventory that includes tools staff added without telling IT. A shadow AI audit is the usual way to find them. |
Where does data go, and can you delete it? (questions 6 to 10)

| # | Question | What a good answer looks like |
|---|---|---|
| 6 | Question: Can you draw every place one user's data goes in a single request? | What a good answer looks like: A diagram covering the prompt, chat history, retrieval store, cache, logs and every vendor. See our guide to DPDP compliant AI architecture. |
| 7 | Question: Is personal data masked or minimised before it reaches the model? | What a good answer looks like: Redaction runs before the call. Aadhaar numbers are masked to the last 4 digits. Recall is tested on your own data. |
| 8 | Question: Do provider settings match what you tell customers? | What a good answer looks like: Dated screenshots or contract terms for training, retention and zero-retention options, per provider and per plan. |
| 9 | Question: Can you delete one person's data from every store? | What a good answer looks like: A tested path across vector indexes, caches, logs and backups, with a documented backup expiry. |
| 10 | Question: Where is data hosted, and what if transfer rules change? | What a good answer looks like: Regions listed per vendor. A plan for restrictions under DPDP Section 16 or sector rules. |
On question 9, unless a law requires retention, DPDP Section 8(7) asks a Data Fiduciary to erase personal data when consent is withdrawn or the purpose is no longer served, whichever is earlier, and to make its processors do the same. Section 12 gives people a right to ask for erasure. These core duties start on 13 May 2027. The Rules do not spell out backup erasure, so write down when your backups expire.
On question 10, DPDP does not require data to stay in India. Section 16 lets the government restrict transfers to notified countries, and as of November 2026 none have been notified. Hosting in India avoids future restrictions and sector localisation rules. Whether your setup is fine for your sector is a question for counsel.
How is the model layer built? (questions 11 to 15)
| # | Question | What a good answer looks like |
|---|---|---|
| 11 | Question: Are the model, prompt and parameters version-pinned? | What a good answer looks like: Each release names the exact model version and prompt version. A change is a reviewed commit, not an edit in a dashboard. |
| 12 | Question: Can you switch model or provider without rewriting the product? | What a good answer looks like: Provider calls sit behind one interface, and a test set shows the new model's results before the switch. |
| 13 | Question: Are prompts treated as code? | What a good answer looks like: Prompts live in version control with review and tests, not scattered across files and notebooks. |
| 14 | Question: Does retrieval enforce each user's permissions? | What a good answer looks like: Documents are filtered by the caller's access at query time, not after the model has seen them. See permission-aware RAG. |
| 15 | Question: What limits an agent's tools? | What a good answer looks like: Least-privilege credentials per tool, an allow-list, and human approval for anything irreversible. |
On question 15, the Model Context Protocol specification says there should always be a human in the loop with the ability to deny tool invocations, and that clients must treat tool annotations as untrusted unless they come from trusted servers. The spec recommends the first and requires the second.
How do you know it works, and can you trace an answer? (questions 16 to 20)
| # | Question | What a good answer looks like |
|---|---|---|
| 16 | Question: Is there an evaluation set built from real cases, and does it run before each change? | What a good answer looks like: A versioned test set with hard and edge cases, run in CI on every prompt, model or retrieval change. |
| 17 | Question: What is the pass bar, and who agreed it? | What a good answer looks like: A written threshold per task, signed off by the product owner, not by whoever built it. |
| 18 | Question: Can you trace one answer to its prompt, model version, retrieved documents and tool calls? | What a good answer looks like: A trace ID on every response, searchable in your logs. |
| 19 | Question: How do you spot regressions and drift in production? | What a good answer looks like: A sampled review queue, user feedback signals and alerts on quality scores. |
| 20 | Question: What do you log, and for how long? | What a good answer looks like: A logging policy that balances debugging with privacy, with retention set per log type. |
On question 18, the OpenTelemetry GenAI semantic conventions are an emerging standard. They define attributes such as `gen_ai.request.model` and `gen_ai.usage.input_tokens`. Their status is Development, not stable, so expect changes. Prompt and response content is opt-in, which is the right default when prompts hold personal data.
On question 20, Rule 8(3) of the DPDP Rules requires a Data Fiduciary to keep personal data, traffic data and processing logs for at least one year for the purposes in the Seventh Schedule, then erase them. This applies from 13 May 2027. Prompt logs that contain personal data are likely to be caught, so decide what you store and where, and confirm the position with counsel.
How secure is the system? (questions 21 to 25)

| # | Question | What a good answer looks like |
|---|---|---|
| 21 | Question: Which OWASP LLM Top 10 risks have you tested, and how? | What a good answer looks like: A test record mapped to the list, at least for prompt injection, sensitive information disclosure, improper output handling and excessive agency. |
| 22 | Question: What happens when a retrieved page or document contains instructions? | What a good answer looks like: Retrieved content is treated as untrusted data. Injection tests are in the eval set. |
| 23 | Question: Is model output treated as untrusted before it reaches a database, shell, browser or API? | What a good answer looks like: Output is validated, escaped or schema-checked before any downstream use. |
| 24 | Question: Where do secrets live, and can the model see them? | What a good answer looks like: Keys sit in a secrets manager, are scoped and rotated, and never appear in prompts or logs. |
| 25 | Question: Which packages, MCP servers and plugins does it use, and are they pinned and reviewed? | What a good answer looks like: A pinned dependency list with a review step for every new server or plugin. |
The OWASP Top 10 for LLM Applications 2025 lists prompt injection (LLM01), sensitive information disclosure (LLM02), supply chain (LLM03), improper output handling (LLM05), excessive agency (LLM06) and unbounded consumption (LLM10), among others. Say your tests are "mapped to the OWASP LLM Top 10". Do not say "OWASP compliant", because there is nothing to be compliant with.
Question 25 matters because tooling around AI has had serious flaws. CVE-2025-6514 was a command injection in the mcp-remote package (versions 0.0.5 to 0.1.15), rated 9.6 under CVSS 3.1. CVE-2025-49596 was an unauthenticated remote code execution flaw in MCP Inspector below version 0.14.1, rated 9.4 under CVSS 4.0. Pin versions and patch them like any other dependency. Our vibe coding security checklist covers the same ground for AI-written code, and our guide to technical due diligence for vibe-coded startups covers what investors and buyers check.
What does it cost, and who runs it? (questions 26 to 30)
| # | Question | What a good answer looks like |
|---|---|---|
| 26 | Question: What does one request cost, and what would the bill be at 10 times the usage? | What a good answer looks like: A cost-per-request figure from real traces, with a simple model for volume, context length and retries. |
| 27 | Question: Are there spend limits, rate limits and loop caps? | What a good answer looks like: Per-user and per-key budgets, a maximum number of agent steps, and alerts before the limit. |
| 28 | Question: Who is alerted when cost, latency or error rate moves? | What a good answer looks like: Named on-call owners and dashboards that show tokens and spend per feature. |
| 29 | Question: What is the plan for an AI incident? | What a good answer looks like: A runbook for a wrong answer reaching a customer, a leak or a runaway bill, with a kill switch and contacts. |
| 30 | Question: Can a new engineer run, test and deploy it from the docs? | What a good answer looks like: A README, a runbook and a working local setup. No single person holds all the knowledge. |
On question 26, price per token is only part of the bill. Output tokens cost more than input tokens. Long context, retries and agent loops add up. Caching can cut cost: on Anthropic's public list prices, checked 20 November 2026, a cache read costs 0.1 times the base input price on most models and 0.05 times on Opus 5.5 and Sonnet 5.5, while a five-minute cache write costs 1.25 times. Prices change, so check the current page. Our post on LLM bill shock walks through the main cost drivers.
On question 29, if an incident exposes personal data, Rule 7 of the DPDP Rules sets a duty, from 13 May 2027, to notify affected people and the Board without delay, with a detailed report to the Board within 72 hours. CERT-In separately requires reportable cyber incidents to be reported within 6 hours. Write the runbook before you need it.
What is AI technical debt, and how do you price it?
AI technical debt is the shortcut that works in the demo and costs you later. It is often hidden, because the feature appears to work. Common forms are below.
| Debt item | How to spot it | The interest you pay |
|---|---|---|
| No evaluation set | How to spot it: Prompt changes ship after a few manual checks | The interest you pay: Regressions reach users. Every model upgrade is a gamble. |
| Prompt sprawl | How to spot it: Similar prompts are copied across files and nobody knows which is live | The interest you pay: Slow fixes and inconsistent answers. |
| Provider lock-in | How to spot it: SDK calls are spread through the code base | The interest you pay: A price rise or deprecation forces a rewrite. |
| Unowned vector stores | How to spot it: Indexes were built once and no one refreshes or deletes from them | The interest you pay: Stale answers and a failed erasure request. |
| Untracked spend | How to spot it: One shared API key and no per-feature cost | The interest you pay: Surprise bills and no way to find the cause. |
| Undocumented fine-tunes | How to spot it: No record of training data, base model or settings | The interest you pay: You cannot reproduce, audit or retire the model. |
To price a debt item, use your own numbers for three things:
- Principal: the engineer-days needed to fix it, at your loaded daily cost.
- Interest per month: engineer hours lost to the workaround, plus extra token spend, plus the expected cost of an incident (its likelihood multiplied by its impact).
- Payback: principal divided by monthly interest. A fix that pays back in a few months usually goes first.
Add a trigger to each item, such as "before the next enterprise security review" or "before we pass 10,000 users". That turns a vague worry into a dated decision. A tech audit can run this review for you if your team is too close to the code to score it.
What should you do with the answers?
- Fix the reds in data flow and security first. They block enterprise deals and carry legal exposure.
- Write down the ambers. Most are missing documents, not missing systems.
- Turn the 30 questions into a spreadsheet. Copy the tables, add owner, evidence link and date columns, and review it each quarter.
- Re-run the review after big changes: a new model, a new provider, a new data source or a new agent tool.
Sources: OWASP Top 10 for LLM Applications 2025 (genai.owasp.org/llm-top-10). Model Context Protocol specification, server tools and authorization (modelcontextprotocol.io). CVE records CVE-2025-6514 and CVE-2025-49596 (cveawg.mitre.org). OpenTelemetry GenAI semantic conventions (github.com/open-telemetry/semantic-conventions-genai). Anthropic API pricing (platform.claude.com/docs/en/about-claude/pricing), checked 20 November 2026. Digital Personal Data Protection Act, 2023, Sections 8, 12 and 16, and DPDP Rules, 2025, Rules 6, 7 and 8. Last reviewed 20 November 2026.
FAQ
What is an AI architecture review?
An AI architecture review is a structured check of how an AI feature or product is built. It covers scope and ownership, data flow, the model layer, evaluation and observability, security, and cost and operations. Each question needs evidence such as a document, a setting or a test result, so the review gives a scored list of gaps rather than a general opinion.
When should you run an architecture review for an AI product?
Run it before launch, before a large customer's security review, before a funding round or acquisition, and after any big change to the model, provider, data sources or agent tools. A short quarterly re-check keeps the answers current, because AI systems change faster than most software.
What are the most important questions to ask about an AI product?
Start with data flow and security. Ask where each user's data goes, whether you can delete it, what the model can access, and how prompt injection and unsafe output are handled. Then ask whether an evaluation set runs before each change. Failures there are the ones that block enterprise deals.
What is technical debt in AI features?
It is the shortcuts in an AI feature that work now and cost more later. Examples are no evaluation set, scattered prompts, provider calls spread through the code, vector stores nobody owns, and untracked spend. You price each item as principal (the effort to fix it) against monthly interest (the time, spend and incident risk it adds).
Does an architecture review make a product compliant or certified?
No. A review finds gaps and produces evidence, but it does not certify a product. Accredited certification against standards such as ISO/IEC 42001 comes only from accredited certification bodies, not from consultants or reviewers, Mono included, and legal compliance questions, including under the DPDP Act, should go to counsel.
