A caring bot isn’t automatically a safe one.

We tell chatbots things we wouldn’t tell a stranger.

The reply comes in seconds, sounds warm, and remembers what we asked last week. So we keep typing: our name, our phone number, our symptoms, the worry that kept us up at 2 a.m. Every message feels like a private conversation. Every message is also data, handed to an AI system.

For most of us, that means oversharing about shopping habits. For someone using a health, farming or welfare service, it can mean a medical history, an ID number and a home address, all in one WhatsApp message.

At the start of last quarter, we focused our efforts on building responsible AI into Glific? In September, we worked on Evaluations 2.0 in Glific so NGOs can check whether their bot is right. Now we have added guardrails so every message is checked to see whether it is safe. 

Where we started

Glific and Kaapi. Glific is an open-source WhatsApp platform that NGOs use in health, agriculture, education and livelihoods. Kaapi is the AI layer behind it, connecting those bots to AI providers

The first half of the work: Evals 2.0 launched in Glific in early September 2026. An AI judge scores a bot’s answers against a golden dataset, the NGO’s own documents and its config, then suggests what to fix.
Read more: The long way home: how Evals 2.0 came back to Glific

NGOs Uptake: In six weeks, 20 NGOs ran 220 evaluations, and 19 came back to run more. They found bots inventing application links, giving wrong clinic hours and contradicting their own documents.
Read more: Your AI assistant sounds sure. But is it accurate?

What evals can’t cover. Evals test a bot before and between releases. They can’t stop one specific live message from one specific user. That is the job of a guardrail: a check that runs on every message, before it reaches the model and before the model’s answer reaches the user.

Our running example. SNEHA runs a maternal health chatbot on Glific, where expecting mothers in Mumbai ask about pregnancy and newborn care. It is a good illustration of what is at stake. The users are vulnerable, the topics are sensitive, and the messages arrive in Hindi, Hinglish, and English. In our first iteration of Guardrails, most of our guardrails were built on English models, but most of our users don’t write in English. Here are the three risks we set out to fix and how we tested each fix.

Risk 1: People overshare, and the model shouldn’t see everything

When a bot feels personal, people share names, phone numbers, Aadhaar numbers, and addresses without thinking. A mother asking SNEHA’s bot about a missed check-up might include her full name and number in the same message. That data shouldn’t be sent to a third-party LLM.

Our PII guardrail caught English names fairly well. Hindi, Marathi, Kannada and Telugu names were often missed.

The root cause was the name-detection model inside our PII tool (Presidio with spaCy en_core_web_lg), which is English-only. Pattern-based checks for phone, email, PAN and Aadhaar were unaffected, because patterns don’t depend on language.

What we tried. We compared alternatives and chose IndicBERTv2 fine-tuned on Naamapadam, a dataset of named entities in Indian languages. It supports Indian languages, runs on a CPU, fits Presidio’s existing setup, and needs no new dependency or API key.

How we tested it

• We wrote 550 synthetic WhatsApp-style NGO messages: 500 containing PII and 50 clean. Together they cover all 15 entity types we support, including names, locations, phone numbers, email, PAN, Aadhaar, passport, vehicle registration and voter ID.
• We used real Indian names and rewrote every message in six languages: English, Hindi, Hinglish, Marathi, Kannada and Telugu. Names and places were in native script. That gave 3,300 rows.
• Both models ran on exactly the same rows. We then checked the results on a second, independent set of 419 messages (300 English, 119 Hindi).

What we found

	English model (before)	IndicBERT (now)
Hindi names detected	7%	75%
Marathi / Kannada / Telugu names detected	9–11%	58–75%
Places detected (Indian languages)	5–26%	~100%
False alarms on clean messages	72	13
Hindi accuracy (second dataset)	0.78	0.97

• Row by row, the new model was better on 722 messages and the old one on 167. Of the 13 remaining false alarms, 11 were celebrity names, like Bill Gates, that the dataset labels as non-PII.
• What it costs: English and Hinglish name recall dropped a little (75% → 66% and 73% → 57%), and each check takes about 20 ms instead of 6 ms. We accepted that trade, because detecting 7% of Hindi names meant names in Indian languages were effectively unprotected.

Risk 2: A helpful model will answer questions it shouldn’t

Evals showed bots answering confidently beyond what their NGO had approved: unapproved clinical claims, invented forms, made-up camps. An LLM is trained to be helpful. It doesn’t know an NGO’s mandate, audience or legal limits.

A maternal health bot must refuse to predict a baby’s sex, because sex determination is illegal in India. SNEHA’s case is why we built forbidden-topics mode. The same pattern appears in every domain: a farm-advisory bot asked about crops it shouldn’t cover, an HIV helpline asked for a diagnosis, or a youth bot asked about unsafe subjects.

What changed

• Each NGO lists its off-limits topics and writes its own reply.
• The topic list is clearly marked in the prompt, and the model is told never to follow instructions hidden inside it. Without that, the list itself could be used to manipulate the bot.
• When a question is off-limits, the user gets the NGO’s reply in their own language instead of a vague English refusal.

Risk 3: Public numbers attract abuse, in every language

An open WhatsApp number will receive abusive messages. Those messages shouldn’t reach the LLM, and the bot should never repeat them back.

Our toxicity guardrail was a slur list. It caught only words it already knew and missed spelling variants such as bhosdike and kutte. The English profanity filter we paired with it blocked “namaste, kaise ho” because it read “ho” as a slur.

Why toxicity is harder than PII

• PII models label each word, so we can remove just the name: “mera naam Raman hai “→ “mera naam `<PERSON>` hai“. Datasets with that word-by-word labelling exist for PII.
• Toxicity models score the whole sentence instead: “tu bewkoof hai, bhag yahan se” → toxic, 0.97. Most Hindi and Hinglish toxicity datasets are labelled the same way, one label per sentence.
• Removing the slur doesn’t make a message safe. “tu [REDACTED] hai, bhag yahan se“ is still abusive. And “I hope your mother dies slowly“ has no slur to remove at all.
• So we block instead of cleaning. Like most guardrail tools, we reject the whole toxic message and reply with a polite fallback, rather than trying to remove individual words.

How we tested it

• We used a combined benchmark of 1,801 messages: 200 from HASOC (a public hate-speech dataset), 1,000 from ShareChat and 601 from our own slur set. 727 messages are in English and 1,074 in Hindi (both Devanagari and Romanised Hinglish), and 1,139 are toxic.
• We compared our existing validators with four Hugging Face candidates, two TextDetox models and two MuRIL models, all on a CPU and all on the same rows.

Setup	Precision	Recall	F1
Slur list alone (before)	0.90	0.63	0.74
English profanity filter alone	0.78	0.23	0.36
TextDetox BERT alone	0.79	0.83	0.81
Slur list + TextDetox BERT (now)	0.80	0.92	0.85

Why TextDetox

• It had the best F1 score, runs in about 30 ms on a CPU, and is actively maintained. MuRIL did well on HASOC, and its repository hadn’t been updated in 3-4 years.
• A fallback, not a blank. A toxic message gets a polite reply in the user’s language, and the LLM is never called.
• What it costs: false alarms rose from 81 to about 265 out of 662 clean messages, mostly heated political text. For a helpline, asking someone to rephrase is cheaper than letting abuse through.
• Scope: this covers English, Hindi and Hinglish only, because those are the languages we have test data for.

A blocked message still deserves a reply

Before, a blocked user got an empty message or a generic English “please rephrase.” For someone who wrote in Hindi, that ends the conversation.

Now, fallback replies are translated into the user’s language and script: Devanagari for Hindi, Latin script for Hinglish, and English for English. The conversation continues. Each NGO sets its own topics, PII types and replies without writing code.

Responsible AI is a habit, not a feature

Evals tell an NGO whether its bot gives the right answer. Guardrails make sure that, message by message, personal data stays protected, off-limits questions are refused, and abuse is stopped, in the user’s own language. Glific now does both.

We held guardrails to the same standard as evals: no change shipped without being benchmarked on real multilingual data. We don’t ask NGOs to trust their bot without evidence, and we don’t ask them to trust our guardrails without evidence either.

None of these validators were new. They had been running in Kaapi for months. But when we revisited them to bring them to Glific, we didn’t just switch them on. We tested them against the people they would actually protect, found where they fell short, and rebuilt them until the numbers held up.

That is a real step forward, but it is not the finish line. Every number in this post is a starting point. The slur list still misses spelling variants, the toxicity model still flags heated but harmless messages, and many Indian languages still have no test data at all. So we will keep doing what we ask of NGOs with evals: measure, fix one thing, and measure again.

Don’t settle for what’s already built.

That is why we have started working with a first group of NGOs as beta partners. Together we are setting up guardrails on their live chatbots, learning which checks matter in their domain, and using what we see in real conversations to refine the validators.

An AI assistant that sounds confident isn’t necessarily right. Guardrails add the next point: a caring bot isn’t automatically a safe one.

Next time a chatbot answers you in under a second, think about what you just told it and who is checking what happens to that message?