Short answer: you add AI to an existing SaaS product as a separate, narrow service that sits beside your app, not as a rewrite of it. The AI service calls a model through a gateway, reads customer data only through the permission checks your product already has, and ships behind feature flags with a per-customer switch. Your database, your auth and your core screens stay as they are. The real work is in four places: data access, tenant isolation, testing and rollout.
This post is for founders and engineering leads with a live product, paying customers and a roadmap that now says "AI". It is engineering guidance, not legal advice. Contract and privacy wording goes to your counsel.
What does "without a rebuild" actually mean?
It means your product keeps its current architecture, and you add five thin pieces around it. Nothing in the list below needs a new database, a new front end framework or a new auth system.
| Piece | What you add | What stays the same |
|---|---|---|
| AI service | What you add: A small service or module that owns prompts, model calls and tool calls | What stays the same: Your main application and its release process |
| Data access | What you add: A read path from the AI service to your data, through your existing permission checks | What stays the same: Your tables, your roles, your tenant model |
| Model gateway | What you add: One internal endpoint that calls models, with keys, budgets and fallbacks | What stays the same: Nothing in the app talks to a model provider directly |
| Controls | What you add: Feature flags, a per-tenant AI setting, a kill switch and rate limits | What stays the same: Your plan and entitlement logic, extended by one setting |
| Evidence | What you add: Logs, traces, a test set and a cost figure for each AI call | What stays the same: Your current logging and monitoring stack |
Which AI features should you add first?
Start with features where a wrong answer is cheap to spot and cheap to undo. Leave actions that change customer data for later. A short ranking helps, and our AI use case matrix goes deeper on scoring.
| Feature type | Example in a SaaS product | Main risk | Good first step? |
|---|---|---|---|
| Summarise | Example in a SaaS product: Summary of a ticket thread, a call note or a long record | Main risk: Missed or invented detail | Good first step?: Yes. The user can see the source beside the summary |
| Draft | Example in a SaaS product: A first reply, a description or a report the user edits | Main risk: Tone or fact errors | Good first step?: Yes, if the user must press send |
| Classify or extract | Example in a SaaS product: Tag incoming items, pull fields from documents | Main risk: Silent wrong labels | Good first step?: Yes, with a confidence threshold and a review queue |
| Search and answer | Example in a SaaS product: Ask a question over the customer's own records and files | Main risk: Showing one user another user's data | Good first step?: Yes, but build permission-aware retrieval first |
| Act (agent) | Example in a SaaS product: Update records, send messages, run workflows | Main risk: Wrong actions at speed | Good first step?: Last. Start read-only, then add actions one at a time with approval |
Pick one or two features that map to a task your users already do many times a week.
Where should the AI code live in your architecture?

You have three options. For most products the second is the right default.
| Option | How it works | When it fits | Watch out for |
|---|---|---|---|
| Inside the monolith | How it works: Model calls and prompts live in your existing codebase | When it fits: A very small team and one low-risk feature | Watch out for: Slow model calls tie up your request threads, and a prompt change needs a full release |
| Separate AI service | How it works: A small service your app calls over an internal API, with its own deploys and its own config | When it fits: Most products | Watch out for: One more thing to run, and a clear interface you must keep narrow |
| Third-party widget | How it works: A vendor's chat or copilot embedded in your UI | When it fits: A quick test of demand | Watch out for: Your customer data goes to another processor, and you control little of the behaviour |
A separate service keeps three things apart that should not be tangled. Prompts change weekly and need quick releases. Model calls are slow and can fail, so they need timeouts, retries and streaming. Your core product needs to stay stable while both of those change.
Keep the interface small. Your app sends a request with the tenant ID, the user ID, the feature name and the inputs. The AI service returns a result or a stream. Long jobs should use the job queue you already have. Do not let the browser call a model provider directly. The key would sit in the page, and you would have no place to apply limits or logging.
How do you give the model customer data without leaking it across tenants?
This is where most existing SaaS products get hurt. Your app already knows who can see what. An AI feature that reads data through a side door, such as one shared search index with no filter, can show a user someone else's records. The rule is simple: the AI service sees only what the signed-in user could already see in the product.
- Pass the user's identity through. The AI service receives the tenant ID and user ID with every request and uses them on every query. A service account with access to everything is a leak waiting to happen.
- Filter inside the database where you can. PostgreSQL row level security lets a policy restrict which rows a user can return. With RLS on and no policy, all access is denied. Superusers and roles with `BYPASSRLS` always bypass it, and table owners bypass it unless you set `FORCE ROW LEVEL SECURITY`. Check which role the AI service connects as.
- Know how your vector search filters. The pgvector documentation says that with approximate indexes, filtering is applied after the index is scanned. At the default `hnsw.ef_search` of 40 and a filter that matches 10% of rows, a query returns about 4 rows. Iterative scans (version 0.8.0 and later), partial indexes or partitioning fix this. Without them, a tenant filter can silently return too few results.
- Do not create one collection per tenant by default. Qdrant's guidance is that a separate collection for each tenant is rarely the most efficient approach. It suggests one collection with a tenant payload index, and separate collections only for a limited number of tenants that need strict isolation.
- Expect synced permissions to lag. Microsoft's Azure AI Search documentation on document-level access says permission refresh does not provide immediate revocation, and that ADLS ACLs refresh about once a day. If a user loses access, a copied permission list can still show the old state. For instant revocation, check the source system at query time before you show a result.
Our post on permission-aware RAG walks through each of these in more detail, with tests you can run.
Which model should you use, and how do you avoid lock-in?
Do not wire one provider's SDK into every feature. Put a gateway between your AI service and the models. The gateway gives you one place for keys, spend limits, fallbacks and logging, and it lets you change models without touching feature code. Our post on multi-model architecture covers the design.
Two examples show what a gateway does. LiteLLM describes itself as an open source AI gateway that calls more than 100 providers in the OpenAI format. It ships as an SDK and as a proxy with virtual keys, spend tracking, guardrails and load balancing, under an MIT licence except for its `litellm/enterprise/` folder. Its router supports ordered fallbacks, plus separate fallbacks for context window and content policy errors. OpenRouter is a hosted router. Its `models` array is tried in order on errors, and the request is priced using the model that was finally used.
A hosted router is another processor that sees your prompts. For sensitive customer data, a self-hosted gateway in front of India-hosted or private models is the safer design. If you later move some features to an open-weight model, a server such as vLLM exposes an OpenAI-compatible API, so the gateway can route to it with little change in your code. Check the licence on each model card first, because open-weight licences differ by model and some limit commercial use. See our guide to open-weight licences.
Whatever you choose, keep the model name, prompt version and gateway route in config, not in code. That is what makes a switch a config change.
How do you handle personal data and customer commitments?
Existing customers signed contracts before your AI feature existed. Check three things before launch.
- Is the AI provider a new sub-processor? If your customers have a sub-processor list or notice period in the contract, a new model provider may need to be added to it first.
- What does the provider do with the data? Read the provider's terms on retention and training for the plan you actually use. Customers will ask whether you train on their data, so have a documented answer.
- Who can switch it off? Enterprise buyers expect a tenant-level control. Our post on enterprise readiness lists the AI settings they ask for, and why the setting must be enforced in code, not only shown in a settings page.
On personal data, in most B2B SaaS set-ups your customer is the Data Fiduciary under the DPDP Act and you act as its Data Processor, but confirm the roles for your contracts with counsel. Section 8(2) needs a valid contract for processing on a fiduciary's behalf, and Section 8(1) keeps the fiduciary responsible. The core duties apply from 13 May 2027. Every place personal data goes in an AI feature, such as prompts, retrieval indexes, logs and caches, should be on your data map, with an owner and a deletion path.
The DPDP Act does not require data to stay in India or models to be self-hosted. Section 16 lets the Government restrict transfers to notified countries, and as of November 2026 none is notified. Sector rules and customer contracts may still require India hosting, and the main cloud providers have regions there: AWS in Mumbai and Hyderabad, Azure in Pune, Hyderabad, Chennai and Mumbai, and Google Cloud in Mumbai and Delhi. GPU types differ by region and zone, so check before you plan. A masking step before the prompt also reduces what leaves your system, as our guide to PII redaction for LLMs explains.
How do you test an AI feature when the answers change every time?
You test it the way you test search: with a fixed set of real cases and a score, not with a single run that looks good. A basic setup has four parts.
- A golden set. Collect 50 to 100 real inputs with the answer a good human would give. Include the hard and the odd cases, such as empty records, long threads, mixed languages and attempts to trick the model.
- A pass rule per feature. For a summary, the rule might be that it names the same key facts as the human answer and invents none. For extraction, you can score field by field.
- A run on every change. Run the set when you change the prompt, the model or the retrieval. A change that raises one score and drops another should be visible before it ships.
- Tenant safety tests. Ask as user A for data that only user B may see, and check that the answer, the sources and the logs show nothing from B.
Log every call with the model, prompt version, token counts, latency, status, tenant and cost, so you can trace a bad answer back to its cause. Our post on LLM observability for small teams shows the log fields and the cheap quality signals. The same logs feed your answer to a customer who asks what the AI did with their record.
How do you roll an AI feature out safely?

Treat the first release as an experiment you can stop in a minute.
- Dark launch. Run the feature on real traffic with the output hidden, and compare it to what your users did. You find failure cases without exposing anyone.
- Internal, then a few tenants. Turn it on for your own team, then for two or three customers who asked for it and know it is new.
- Per-tenant opt-in or opt-out. Store the setting on the tenant, check it in the AI service on every call, and write the change to your audit log.
- A kill switch. One flag that turns off all model calls or a single feature, without a deploy, and a clear message in the UI when it is off.
- Rate limits and budget caps. Limit calls per user and set a spend cap per tenant and per feature. A loop in your code or a heavy user should hit a limit, not a bill.
- Show sources and allow feedback. Link each answer to the records it used, and give the user a way to flag a bad one. Those flags become new test cases.
Add a human step wherever the AI proposes a change to customer data. The user sees the proposed change and approves it. Remove the step for a given action only after the logs show it is reliable.
What will it cost, and how do you estimate it?
Model cost is tokens in, times the input price, plus tokens out, times the output price. Estimate it per request, then multiply by your expected volume. The worked example below uses a public list price, checked on 20 November 2026, of $2 per 1 million input tokens and $10 per 1 million output tokens (Anthropic's listed rate for Sonnet 5.5, ex-tax). It is an illustration, not a quote for your product, and prices change.
| Step | Illustrative figure | Result |
|---|---|---|
| Input per request | Illustrative figure: 3,000 tokens (instructions, retrieved records, the question) | Result: 3,000 x $2 / 1,000,000 = $0.006 |
| Output per request | Illustrative figure: 500 tokens | Result: 500 x $10 / 1,000,000 = $0.005 |
| Cost per request | Illustrative figure: Sum of the two | Result: $0.011 |
| Cost per 100,000 requests | Illustrative figure: 100,000 x $0.011 | Result: $1,100 |
Then check where the cost can be cut. Smaller models for easy steps, caching repeated prompts, shorter retrieved context and batch pricing for jobs that can wait (listed at 50% off by both OpenAI and Anthropic) all help. Our AI spend audit guide shows how to tie each charge to a feature and a tenant.
What are the common mistakes?
| Mistake | What goes wrong | Fix |
|---|---|---|
| One shared index for all customers | What goes wrong: A search returns another tenant's record | Fix: Filter by tenant and user on every query, and test it |
| No per-tenant switch | What goes wrong: An enterprise customer cannot turn it off, and the deal stalls | Fix: Add the setting and enforce it in the AI service |
| Judging quality by a demo | What goes wrong: The feature fails on real messy data | Fix: Build the golden set before launch |
Where Mono helps
Mono is an AI-first product studio. We design and build AI features into live products as part of AI product development: the architecture, the permission-aware data path, the gateway, the test set and the rollout controls. Mono has shipped full platforms, such as Prosprr and Maple Drive, and staffed design and frontend work for Otto's AI product MVP. If you want a second opinion before you build, our tech audit reviews an existing architecture against the points above. Mono is not a law firm or a certification body, so legal wording and any certification stay with your counsel and an accredited auditor.
Sources: PostgreSQL documentation, "Row Security Policies" (postgresql.org/docs/current/ddl-rowsecurity.html). pgvector README, version 0.8.7 (github.com/pgvector/pgvector). Qdrant documentation, "Multitenancy" (qdrant.tech/documentation/guides/multiple-partitions/). Microsoft Learn, "Document-Level Access Control - Foundry IQ (Azure AI Search)", dated 9 November 2026. LiteLLM repository and routing documentation (github.com/BerriAI/litellm, docs.litellm.ai/docs/routing). OpenRouter documentation on model fallbacks. vLLM documentation on OpenAI-compatible serving. Digital Personal Data Protection Act, 2023, Sections 8 and 16; DPDP Rules, 2025. AWS, Microsoft Azure and Google Cloud region documentation. Anthropic and OpenAI public pricing pages, checked 20 November 2026. Last reviewed 20 November 2026.
FAQ
Can you add AI to an existing SaaS product without rebuilding it?
Yes. Add a separate AI service beside your app, a model gateway, a permission-aware data path, feature flags and logging. Your database, auth and core screens stay as they are. A rebuild is only needed if your data has no reliable tenant ID or can only be read by the monolith.
Where should the AI code live in a SaaS architecture?
For most products, in a small separate service that your app calls over an internal API. It keeps prompt changes, slow model calls and retries away from the core product. A very small team can start inside the monolith for one low-risk feature, but should keep the model calls behind one internal interface.
How do you stop an AI feature showing one customer's data to another?
Pass the tenant and user identity into every AI request, and filter every retrieval by it. Use database row level security where you can, check how your vector search applies filters, and test by asking as one user for another user's data. For instant revocation, check the source system at query time.
Which AI features should a SaaS product build first?
Start with summaries, drafts, and classification or extraction, where the user can see the source and approve the result. Add search over customer data once permission-aware retrieval works. Leave recommendations and agents that change data until you have usage data, a test set and an approval step.
Do you need a model gateway when you only use one provider?
It is still worth having. A gateway gives you one place for keys, spend caps, logging and fallbacks, and it lets you change models or providers without editing every feature. It also keeps provider keys out of the browser and the main application.
Does the DPDP Act stop a SaaS company from using US AI providers?
Not as the law stands in November 2026. Section 16 lets the Government restrict transfers to notified countries, and none is notified. Your customer contracts, sector rules and sub-processor terms may still limit where data goes, so check them before you choose a provider or a region.
