We work closely with NGOs across our platforms and Fractional CxO services. A large number of these are NGOs who focus on education, and for the past few years, we’ve been working with many of them quite closely to innovate and build common tools for the sector. We’ve been supported really nicely on this work by Kyle and his team at Cisco US for the past 4 years. Quite a few features in Glific have come from this collaboration and grant including significant chunks of our AI integration, Custom Certificates, User funnels and more.
We’ve been involved with two AI Cohorts, AI4GD (AI for Global Development) with The Agency Fund and a group of 8 NGOs, with 3 of them in Education (Rocket Learning, Youth Impact and Udhyam). In our AI Cohort, we have a group of 7 NGOs, with another 4 in Education (Avanti Fellows, Simple Education Foundation, Inqui-Lab, Quest Alliance). Along with the Glific Education NGOs, we have a pretty good sense of the technology and AI projects that these NGOs are incorporating into their programs.
Over the past year, both Udhyam Learning Foundation (ULF) and The Apprentice Project (TAP) have been experimenting with solutions to solve grading student submissions at scale. In our work with Inqui-Lab (IQL) during the AI Cohort, they wanted to tackle the same problem. All 3 of these NGOs want to solve a similar problem but with significant differences. Udhyam and Inqui-lab want to evaluate entrepreneurship proposals from their students at a middle school (Inqui-Lab) and high school (Udhyam) level, similar model but with different rubrics and assessments. This typically happens once during the school year, at the end of the academic cycle. TAP on the other hand works with students on social-emotional learning and wants to evaluate how students are doing on their weekly assignments and give them quick feedback specific to their submission for the assignment of the week (which could be different for different schools / states).
The big issue is all 3 of them are dealing with systems at scale, and hence multiple lakhs of students. Training teachers or volunteers to do this consistently and follow the rubric for all students was proving to be a challenge especially with over-worked teachers and/or busy volunteers. Keeping the grading consistent across the program is a really hard problem. However, many of us felt that this was a great use case for LLMs thats could help augment and simplify human evaluation. As Ajay puts it so nicely in his blog: Instead of asking “Can AI replace human evaluation?”, ask “Where can AI effectively augment human evaluation?”
At Tech4Dev we want to solve problems for the broader sector, and we figured that this one seemed applicable to a lot of NGOs out there. We wanted to build on the learnings from TAP, ULF and IQL to build a generic pluggable framework that these NGOs and others can use and customize for their own assessments. As Mekin put it: “A possibly ambitious idea could be to create a broad design with pluggable components – and each org can contribute towards either the overall design and/or the pluggable components (evaluation rubrics for eg)”
Thats what our plans are: Lets document and “open source” our current work, lets share in detail what has been done so far, and then meet in-person to talk design, development and allocation of responsibilities and work, Since the Tech4Dev AI Cohort was meeting in person in Bangalore from Dec 3 – 6th, we decided to meet in Bangalore on December 3rd to hash out the details. All 3 organizations spent some time in documenting their work at a broad level and also a fairly deep technical level. You can read their respective blog posts here:
- Introducing Assignment Analysis: AI-Powered Feedback for Every Student, Delivered via WhatsApp from The Apprentice Project
- Inside Udhyam’s AI Mentor: How We Assess Student Submissions and Deliver Feedback at Scale
- Evaluating student innovation at scale – Building a Multi-LLM assessment pipeline – Project Tech4Dev from Inqui-Lab
- Udhyam Pipeline.pdf
- Agenda and Planning document – In person Meeting
If your organization has done work in this area with a working prototype or more, and would like to get involved please do drop us a line. As always, the doors are open and we want the right people in the room contributing their problem statement, design, and code. If you would love to ue what we end up building, stay tuned, we’ll document and publish our work as it happens.
And since I’m quoting people in this blog post, I might as well end with Bilbo’s quote from Lord of the Rings: “It’s a dangerous business, Frodo, going out your door. You step onto the road, and if you don’t keep your feet, there’s no knowing where you might be swept off to.”
