What mentoring Avanti Fellows team taught me about shipping trustworthy and affordable AI
This blog is written by Poorva Malviya. Link to the original blog is here.
Last month, I sat in a room in Bangalore watching 7 enthusiastic organizations demo what they’d built over a 3-month period at the closing ceremony of AI Cohort 2.0, hosted by Project Tech4Dev. I mentored the Avanti Fellows team, which demoed an AI system that writes progress reports for teachers mentoring students for IIT-JEE. Poojitha presented it beautifully, and the room was impressed. I was too, but not by the part everyone was looking at.
To my engineer’s eye, the interesting part was the machinery behind it that told the team, every single day, on every single run, whether that report could be trusted.
Here’s the one lesson from this cohort I keep repeating to anyone who will listen: in the social sector, the evaluation layer isn’t overhead. It’s the product. It needs to be a first-class citizen in the planning.
Avanti’s first pilot, in an earlier cohort, had a problem. Only about half of the AI-generated student summaries were fully free of hallucination.
A summary tells a teacher how a specific child is doing: where they’re struggling, what to do next, and how to frame the conversation. A fabricated number in that report doesn’t produce a bad user experience; it changes how an adult treats a child.
The team didn’t get uncomfortable or defensive. They got systematic. We aligned early on one principle: the ability to measure quality is the core system, and AI generation is a component sitting on top of it. The prompts and the model are what everyone demos. Hot take: they’re also the part that matters least. The eval layer is what makes the system safe to run, cheap to run, and faster iterative improvements.
What we changed
Move one: get the math out of the model. The first order of business for this project was pulling the deterministic parts of report creation out of the LLM’s purview. The biggest accuracy jump came from this move, not a better prompt, not a bigger model. Accuracy on the team’s benchmark went from roughly 80% to 100%.
This is not a new idea. It’s separation of concerns, a very old principle that still holds true in this ever-changing space 🙂. LLMs are a language layer, not a calculation layer, so let the model explain the numbers, but never let it produce them.
Move two: build a golden dataset before scaling anything. The team created reference reports for a small set of benchmark students, reviewed and approved by human experts, along with a handful of scoring criteria. Every time the prompt, the pipeline, or the model changed, the system was re-scored against this golden set.
In software engineering terms, it’s a regression test suite applied to a system whose outputs are prose. It converts “the new version feels better” into “the new version scores better on the 6 things we care about.”
Move three: instrument everything. The team used observability tooling to trace what the model did, so failures were diagnosable.
I’ll admit to one reversal. I advised the team to set up a Langfuse evaluation layer for LLM tracing plus LLM-as-a-judge scoring. Then I looked closely at the golden dataset and how complete their human-approved scoring criteria already were, and I changed my mind. They didn’t need a model judging a model; they needed to keep running the rubric they had. I withdrew my own advice when I saw that the system is already better and complete.
Safety and affordability are linked
This was a welcome surprise for me. Because Avanti could measure quality precisely, they could confidently run on an inexpensive model: DeepSeek v4 Pro. Their cost landed at roughly ₹1–2 per student report. Without an eval pipeline, choosing a cheap model is a gamble we’re making with students’ data. With one, it’s just an engineering decision; we know exactly what we traded and what we didn’t.
The eval layer didn’t only make the system trustworthy. It made it affordable.
What I’d tell a team on day zero
Working backwards from Avanti’s experience:
- Build your golden dataset before you write your first prompt. If you can’t say what a good output looks like, no model can produce one.
- Decide what the model must never compute. Numbers, scores, dates, numeric comparisons, anything that belongs in deterministic code. It’s not only about accuracy; it simplifies the architecture too.
- Choose your quality criteria with the domain expert, not the engineer or the product manager. The teacher or the nurse knows what a dangerous error looks like. The engineer doesn’t(yet).
- Budget evaluation as build time, not QA time. It’s not the thing you do after the product works. It’s how you find out whether it works.
At the ceremony, the applause was for the team’s work. It should have been for the 6 benchmark students, the scoring rubric, and the pipeline that reruns them on every change.
If you fund AI in this sector, ask to see the eval pipeline before you ask to see the demo. If you build in this sector, know that the unglamorous layer is the one that will keep your users safe and your costs manageable.
And if you get the chance to mentor a team like Avanti’s: please take it. I had an excellent time mentoring and learning with the team.
The numbers here come from the Avanti Fellows team’s own work — read Poojitha’s reflections. Thanks to the Avanti Fellows team and Project Tech4Dev.
