Swasthya Sarathi
A healthcare-companion platform for Swasthya Sarathi
A website and mobile app development project for a multi-service healthcare companion — It helps peo...
Read itLLM systems, RAG and applied ML
AI & ML Development
AI features built with care, not just for demos. Tests before launch, answers based on real sources with citations, guards against made-up answers, and a cost-per-request budget you can watch.
Last updated
- An evaluation harness in CI before a single feature reaches a customer
- Every answer based on a fetched source, with a citation the user can open
- Refusal and hand-over paths. The system says "I do not know" instead of making things up
- Cost per request and p95 delay budgeted, watched and alerted on
What this is
The demo is the easy part.
The most valuable thing we do on an AI engagement is sometimes to tell you not to build one.
A regex will beat an LLM at extracting a GSTIN. A trained classifier will beat an LLM at sending a support ticket to the right place, at a thousandth of the cost and a hundredth of the delay. A SQL query will beat an LLM at "how many orders shipped late last month" — and it will be right, which the LLM will not always be. If the task has one fixed answer, a fixed-rule system should give it. LLMs are for language: summarising, drafting, pulling facts from messy text, talking with people, and working through documents where the rules cannot be written down.
When it is truly the right tool, we build it with care, not just as a demo. That means an evaluation harness before anything goes live. It means guards that catch a made-up answer before your customer reads it. It means a retrieval layer that bases every answer on a source you can cite. And it means a cost-per-request budget you really watch. The demo is the easy part — anyone can produce an impressive one in a weekend. Making it correct on the four hundredth query, at a cost per query you can justify, is the real work.
Nothing ships without an evaluation harness.
If you cannot measure whether your AI feature got better or worse, you are not engineering. You are gambling.
So the first thing we make on every AI project is not a prompt. It is an evaluation set: 150 to 400 real inputs from your own field, each with a known-good answer, put together with your experts. It costs a week and it is the most valuable week in the project.
The harness runs that set on every change, such as a new prompt, a new model version or a new way of fetching text. It scores each one. Where there is a right answer, it checks for an exact match. Where there is not, it checks how closely the answer follows the fetched source. A stronger model also grades answers against a rubric, with human spot-checks. It also records delay at p50 and p95 and cost per request. It runs in CI, just like a test suite, because that is what it is.
The value shows up the first time somebody says "let us just tweak the prompt". Without a harness, that is a guess. With one, you find out in four minutes that the tweak fixed three cases and quietly broke eleven. We have watched more than one team ship a "small prompt improvement" that dropped accuracy nine points. They only found out from customer complaints two weeks later.
Where AI projects die
The demo worked. Then it met production.
We have been called in to fix all six of these. Five of them can be avoided in week one.
-
Nobody can say whether it is getting better
No eval set, no starting score, no pass mark. Every prompt change is a guess, and a "small improvement" quietly dropped accuracy nine points two weeks ago.
-
It invents policy with total confidence
Asked about a refund window that does not exist, it makes one up in smooth, believable English. There is no way to refuse, so refusing was never an option for it.
-
The inference bill is the new cloud bill
Twenty fetched chunks stuffed into every prompt, the top model used on every request including "hi", and no saving of answers. ₹3 lakh a month for something a reranker and a router would have run for ₹40,000.
-
It takes nine seconds to answer
Three model calls one after another and a slow vector search. Users leave the chat and open a support ticket, which was exactly what this was meant to prevent.
-
The retrieval is bad, so the answer is bad
Documents cut blindly every 500 characters, splitting tables and clauses in half. The model is not hallucinating. It is faithfully summarising garbage.
-
Customer data went somewhere nobody checked
PII in prompts, sent to a consumer API tier, with no data processing agreement and nothing hidden. The DPDP question arrives and there is no honest answer.
The first decision
Prompt, retrieve, fine-tune, or do not use a model at all.
| Classic ML / rules | Prompt + RAG | Fine-tuning | |
|---|---|---|---|
| Answer lives in your documents | No | Best | Poor — and goes stale |
| Task is deterministic (extract, route, count) | Best | Wasteful | Wasteful |
| Needs a consistent tone or output format | No | Workable | Best |
| Cost per request | Negligible | Moderate | Low, after training |
| Answers must be citable | n/a | Yes | No |
| Updates when your data changes | Retrain | Instantly | Retrain |
| Typical time to production | 3–6 weeks | 6–12 weeks | 10–16 weeks |
What you get
The support that makes it safe to launch.
None of this shows in the demo. All of it is what separates a feature you can stand behind from one you will have to apologise for.
An evaluation harness in CI
150–400 real inputs with known-good answers, scored on accuracy, following the source, correct refusals, delay and cost. Runs on every prompt and model change. If results get worse, the build fails.
Hallucination guardrails
Answers based only on the fetched text. Claims checked against the source chunks. A clear "I do not know" path that passes the question to a human, because a system that cannot refuse will always make things up.
Retrieval that actually retrieves
Cutting text by meaning so tables and clauses stay whole, search by both keyword and vector, a reranker, and citations the user can open. Most "hallucination" is a retrieval bug in disguise.
A cost and latency budget
Routing by difficulty: a cheap model for the easy 80%, and the top model only where it is worth it. Prompt caching, context sizing, streaming. Cost per request is tracked with an alert for odd spikes.
A data path you can defend
PII hidden before data leaves, business API tiers that keep your data out of training, full logs of prompts and answers, and open-weight models in your own VPC where DPDP or a contract demands it.
Seeing what the system does, even when answers vary
Every request is traced with its fetched text, tokens, cost and delay. Thumbs-down feedback goes into the eval set, so live failures become tomorrow's tests.
How we build
Evaluate, then build. Never the other way round.
The order is the whole method. Teams that build first and measure later launch systems that are wrong with full confidence.
-
01
Feasibility, and the honest no
1 weekWhat is the task, what does success look like in numbers, what does a wrong answer cost, and is a model even the right tool? This is where we tell you a classifier or a SQL query does the job for a hundredth of the price. Several of our best AI projects ended here, and those clients came back.
-
02
Build the evaluation set
1–2 weeks150 to 400 real inputs from your field, each with a known-good answer, built with your experts. Then the harness that scores against it and the pass marks that decide a release. This is the most valuable two weeks in the project and it happens before any feature is written.
-
03
Retrieval and data pipeline
2–4 weeksLoading the documents, cutting them by meaning so structure stays whole, embeddings, search by keyword and vector, and reranking. We measure retrieval quality on its own, with recall at k, before the model writes anything. You cannot fix a writing problem that is really a retrieval problem.
-
04
The system, with guardrails
3–6 weeksPrompt design, tool calling, set-format outputs checked by fixed rules, forced citations, refusal paths and handing over to a human. Everything is improved against the harness, so every change gets a score and is not just argued about.
-
05
Cost, latency and shadow launch
1–2 weeksRouting by difficulty, prompt caching, context sizing, and streaming to make p95 feel faster than it is. Then we run it quietly on real traffic with humans reviewing before anything reaches a customer. We open it up only when the numbers hold.
-
06
Production and the feedback loop
OngoingTraced requests, alerts for odd costs, and thumbs-down feedback sent straight back into the eval set. Every live failure becomes a permanent test, so the same mistake cannot ship twice. Models change under you. The harness is what lets you cope with that.
The budgets we hold
Numbers, not adjectives.
Every AI system we build has a clear budget for correctness, speed and cost. We agree it with you before the build and use it as a pass mark for release. "It seems pretty good" is not a launch rule we accept.
Talk about your use case
400
Eval cases in the harness
Real inputs from your field, labelled by experts
1.8s
p95 time to first token
Streaming, with retrieval in the path
68%
Typical cost reduction
Routing by difficulty, reranking, caching
100%
Answers with a citation
Or a clear, honest refusal
What we actually build
Applications, not experiments.
Document intelligence over your own corpus
Contracts, policies, tenders, technical manuals, claim files. Answers to questions based on the documents, with citations the reader can open. Clauses pulled into set fields, and comparison across versions. This is where RAG truly pays off, and where careless chunking quietly ruins everything by cutting a table in half.
Support assistants that escalate
Answers based on your help centre and order data, with a way to refuse and a clean handover to a human. Fewer tickets is measured honestly, against problems solved, not against "did not open a ticket".
Classification and extraction
Ticket routing, invoice and KYC extraction, lead scoring, defect detection. Often a small trained model, not an LLM. It is faster, cheaper and more accurate, and we will say so.
Forecasting and recommendation
Demand forecasting, spotting customers likely to leave, changing prices with demand, and suggesting the next best step. Real ML tested on past data, not a chatbot wearing a lab coat.
Agent-style workflows with a tight scope
Tool-calling systems that read a document, query a database, draft an action, and stop for a human where money or risk changes hands. We keep them narrow on purpose. A broad agent that acts alone is a demo. A narrow one that matches three hundred invoices a day and flags the eleven it is unsure about is a product.
The stack
Not tied to one model, on purpose.
Top models change every three months. The harness, the retrieval layer and the guards are what last. So we build in a way that lets you swap the model in an afternoon.
Claude API
OpenAI API
Gemini
Llama / Mistral (self-hosted)
Hugging Face
pgvector
Qdrant
LlamaIndex
LangGraph
Rerankers
Ragas
Custom eval harness
FastAPI
PyTorch
scikit-learn
Langfuse
Proof
All case studies
AI in production, with the numbers.
Systems that are measured, not just shown in demos.
Turant Logistics
Logistics & Supply Chain
Cutting order-to-dispatch by 62% for a 400-truck 3PL
A fleet, ePOD and route-planning platform that works offline first, for drivers on roads with poor signal. The two ready-made TMS products t...
62% faster
Faster order-to-dispatch on the ops floor
91% shorter
Shorter POD-to-invoice cycle (8.4 days to 19 hours)
FealDeal
E-commerce — Ethnic Fashion
A high-catalogue ethnic-wear storefront for FealDeal
A Shopify store for a Surat ethnic-wear label. It has 200+ SKUs of lehengas, sarees and suits, sorted by type, colour, fabric and work. Shop...
Shopify
Platform
200+ SKUs
Catalogue
Collection by Rehan
E-commerce — Watches & Accessories
A brand-led watch and eyewear store for Collection by Rehan
A Shopify storefront for a Surat watch and eyewear retailer — where shoppers can browse by gender, brand, style and combo. It has search tha...
Shopify
Platform
Brand & style
Browse by
Whenever the task has a deterministic answer. Finding a GSTIN or a pincode in text is a regex job. Sending a ticket to one of nine queues is a classifier job, which is cheaper, faster and more accurate. "How many orders shipped late last month" is a SQL query, and the LLM will sometimes get it wrong while sounding completely sure. Keep LLMs for language: summarising, drafting, pulling facts from messy text, talking with people, and working through documents where the rules cannot be written down.
RAG in about nine cases out of ten. RAG changes what the model knows. Fine-tuning changes how it behaves. If the answer is in your documents, policies or database, which it usually is, fetch it, give it to the model, and cite it. Fine-tuning is the right choice for a steady output format, a set tone, or a narrow sorting task where you have thousands of labelled examples. It is a poor and costly way to teach a model facts, and the facts go out of date the moment your documents change.
You cannot remove it fully, so we build around it. Base every answer on the text we fetched, and refuse to answer outside it. Require citations and check that each claim matches a fetched piece of text. Drop the claims that do not. Give the system a clear "I do not know" path that passes the question to a human. Most systems hallucinate because refusing was never made available to them. Add fixed-rule checks for anything with a set format. Then measure the rate that is left on your eval set and make it a release gate.
That is a design question and we answer it before we build, not after. We size the context. The difference between stuffing in twenty fetched chunks and reranking to the best four is often 60% of your bill. We route by difficulty. The easy 80% of requests go to a small cheap model, and only what needs it goes to the top model. We save and reuse answers a lot. A support assistant handling 50,000 queries a month typically lands between ₹25,000 and ₹90,000 in inference cost, monitored per request.
We plan the data path first. What leaves your network, to which provider, for how long it is kept, and under what business agreement. The API tiers that keep your data out of training are not the default and must be switched on. PII is hidden before it reaches an outside model where the use case allows. Where a law or a customer contract needs it, we run open-weight models in your own VPC or on your own servers. And we log every prompt and answer, so an auditor gets an honest reply.
An evaluation set of 150 to 400 real inputs from your field, each with a known-good answer, built with your experts in the first week. It runs in CI on every prompt, model and retrieval change. It scores accuracy, how closely answers follow the source, correct refusals, p95 latency and cost per request. We agree the pass marks with you at the start. If a change drops accuracy below the mark, it does not go live, however good the demo looked.
Tell us the use case. We will tell you if it needs a model.
Sometimes the answer is a two-week classifier instead of a six-month AI programme. You will get that answer straight, and for free, in the first conversation.
Proof
Shipped, measured, still running.
MV Tech Education
A course and admissions platform for MV Tech Education
A website development project for a Bihar vocational institute. It is a course list and admissions s...
Read itPatna Taxi
A booking-focused taxi website for Patna Taxi across Bihar
A website development project for a Patna and Muzaffarpur cab operator. It is a fast taxi-booking si...
Read itRelated services
Web Development
Live web applications built on Laravel, Next.js and Postgres. They are typed, tested, easy...
ExploreMobile App Development
Flutter, React Native and native apps that keep working offline, on old phones and on a me...
ExploreEcommerce Development
Commerce platforms where the checkout turns visits into sales and the back office numbers...
ExploreLet's talk ai & ml development
Bring us the hard version of the problem.
A senior engineer reads every enquiry. You'll get a real answer — scope, risk and a number — within one business day.