Recently we concluded our AI focused developer sprint at the historical Fort Kochi. The sprint was aimed at improving collaboration amongst developers, designers and product teams; share learnings, show-and-tell demos in AI and spend some time meeting with other team members with the aim of having a good time over morning walks, team dinners and outings.
Chilled sessions at Aikyam Space
We stayed at The Old Courthouse Heritage hotel by Abad at Fort Kochi. It’s a 200 year old building with a dash of colonial charm. The gentle hospitality of the hotel staff made our week-long stay memorable.
The Old Courthouse by Abad at Fort Kochi
For the first few days, we worked out of Aikyam Space—a cozy shared space where people come together to discuss ideas and get things done. We’re grateful to Aikyam Space for letting us use their space. We also got to chat with Thudippu Dance Foundation, which brings people together through dance, and Olimalar Foundation, a training school that helps build kids’ confidence and emotional intelligence. Thanks to both for taking the time to share what they’re working on with us.
At Aikyam Space with Olimalar and Thudippu
Experiments at Kochi sprint
Before the sprint, Kaapi team had discussed to focus on two problems; one large scale AI enabled assessments and the second is an experimental project on how to hook up github with Claude to speed up PR reviews and AI agents for ad-hoc fixes in the Kaapi frontend. For assessments, the focus was to conduct experiments and come up with an implementation plan to integrate AI agents natively into the assessment pipeline. We have outlined our findings below with detailed reports coming up shortly.
AI Assessments
When you score thousands of student ideas using an AI model, consistency matters. Ask the same model to score the same idea twice with the same rubric or evaluation criteria, and you will often get different numbers. Usually the gap is small, yet we would want the rubric to mimic a human evaluator and return consistent scoring.
We spent a sprint building an agent that rewrites its own rubric until the scores stop wobbling, running four experiments across 19 rubric versions and roughly 90,000 individual scores. We ran multiple experimental runs and come up with a roadmap on how to improve the rubric automatically using agentic loops. The goal was to first make the agent consistent and reduce the spread across the five defined metrics i.e novelty, usefulness, sustainability, feasibility, scalability. We will publish the takeaways in a follow-up technical blog.
Improving the Development Workflow
During the Dev Sprint, two things stood out for us because they were less about building a single feature and more about improving how we work across the team: the code review process and a frontend AI agent for Kaapi.
Making Code Reviews More Consistent
For Kaapi, we set up a Claude-based code review workflow that automatically reviews every pull request and adds comments wherever it finds something that needs attention.
Instead of relying only on a generic review, we added our own custom instructions to the workflow so that the agent knows what to look for in our codebase, including the patterns and practices we want to follow.
We also discussed the approach with other teams to understand how they handle code reviews in their own products. That helped us compare different workflows and think about where automated reviews could fit into our existing process without replacing human review[a].
The goal here isn’t to make the review process fully automated. It’s to catch common issues earlier and make the human review process a little more focused.
A Frontend AI Agent for Kaapi
The second thing we worked on was a small frontend AI agent for Kaapi.
The idea is fairly simple: there are some small UI/UX issues that don’t require changes to business logic or authentication, for example, updating an icon, making a small UI adjustment, or fixing a minor frontend issue. These tasks still take time to pick up, implement, and send through the usual development workflow.
So we built an agent specifically for these kinds of tasks.
Automated Bot Opening Small-Fix PRs
Every morning at 8 AM, the agent checks for issues with the ready-to-agent label, picks up the relevant ones, implements the changes, and creates a pull request. We can also trigger it manually when needed.
The important part is that the agent doesn’t bypass the existing review process. The generated PR still goes through human review, and once it’s reviewed, it can be merged into the main branch.
Concluding Remarks
Team dinner last night at Kochi & Cruise ride in Arabian Sea
In addition to getting to spend a lot of time IRL, bouncing off ideas, what stood out for us was the water cooler conversations, morning walks across the pristine streets of Fort Kochi, and the banter across a shared meal. We did some shopping in the nearby streets of Fort Kochi too. We even took a ferry ride across the Arabian Sea, danced a little on the deck, and had a dinner that ended on a high note. Getting to know how other teams are using AI to build products and simplify workflows has been quite a learning experience. Some of our colleagues could not join the sprint as it was the Ganesh Chaturthi week, but there is always next time to do things better.
Authored by: Kaapi team
