In an attempt to evolve Dalgo as a product, we have experimented with various features like chat with your data, summarize your “filtered” dataset in a warehouse, parse pipeline failures into easy-to-understand summaries, etc.
Some of these have made it into production, but are not being used by our end users as much as we expected them to, and some others didn’t go beyond the experimental stage. However, this journey made us realize that our end users are more interested in fundamental capabilities like KPIs, metrics, reporting, dashboards, etc. Ashwin has beautifully articulated Dalgo’s evolution in this read.
Pivoting in this direction of building dashboards, reports, metrics/KPIs natively in Dalgo, we wanted to understand what level of assistance we can get from AI tools out there. Can these tools help us knock things off our roadmap in less time? I was skeptical.
This is just like chatgpt for devs
I have been using Co-Pilot to assist in my day-to-day development work for the past few months now, but it was just like “ChatGpting” in your VSCode; it was nothing revolutionary.
I used it to ask questions about an issue/bug, or to refactor a piece of code to make it readable & maybe use fewer lines, or to suggest alternate approaches for a logic I had implemented, etc. It was not bad, it did a pretty decent job, especially for all infra/devops-related questions around AWS & Kubernetes.
Some of my questions/prompts to co-pilot
- “I want to set up a VPC peer connection between two VPCs in AWS. How do i go about it ?”
- “Can you re-write this code to use list comprehension in python ?”
- “Write test cases for this function”
- “This is the error <…..error lines….> I see in my console. What could be the issue ?”
- “Generate a function that builds a query using sqlalchemy to fetch data based on this config…”
Some of the simpler questions were answered correctly while others needed more context or some back & forth before reaching the expected outcome.
From chatgpting to agentic tools
We then went on to try agentic tools like Cursor and/or co-pilot (agent mode). Our workflow became more aggressive in the sense that we started asking these tools to build full functionality with some context around it.
We built an internal tool called “Dbt Project Manager” to help our consulting team automate their dbt (a tool used to set up SQL transformations) workflows. This was fully built using Cursor. It did a decent job in terms of scaffolding and setting up the initial project structure. The UI was also pretty neat.
We asked Cursor to set up the UI layout first and then iterated multiple times to pump out functionalities one by one, frontend and backend. We did see some improvement in time saved, but maybe not a lot.
However, as the codebase evolved & the complexity increased, it felt like fighting with the Cursor rather than building. If you ask it to do one thing, it would break 5 other things in other places. It got very frustrating.
We also used Cursor to help us build our v0 prototype or a functionality of it (i.e., charts). It did a similar job as the previous project. I started to contemplate whether we were using the tool correctly. I started questioning. How are other people using it? Why are they so happy & excited about these tools? Is there a different approach to this? Could it have performed better with more context?
From agentic tools to context engineering
When Claude Code released its “Opus” model, you all might have heard people going crazy and sharing on social media what they had built with it. We, too, got curious and wanted to see how good this was.
We decided to go all in and give AI one more chance. But this time our approach was to figure out the right development workflow. After watching a lot of YouTube videos on people explaining their workflows with Claude code, we realized that we are missing 3 things: “context”, “context”, and “context”.
After experimenting for a few days, we were able to use Claude Code to push out multiple PRs in much less time. The PRs covered easy-level & medium-level features/enhancements. Our workflow with Claude Code looks like this (Git repo here with various files)
- Spend a decent amount of time planning the feature. This should include high-level architecture, data flow, low-level design, schema design, api design, etc. I like to call this step “Code in English words”
- Use the above spec and ask Claude to generate a detailed implementation plan. This is a step where developers can nudge Claude to think in the direction that they want it to go.
- Once the plan is approved by a human/developer/us, we let Claude do its magic.
- Review the code generated by Claude
Claude did a surprisingly good job here. It got the entire functionality close to being 95% accurate & in working condition. This was the “Aha” moment that I had been waiting for. You can see in the repo linked above the level of detail that we had to specify and the extensive implementation plan Claude came up with.
I think people might be wondering, This is all good and exciting, but how much time did it save? We are in the early stages of answering that question, but some rough estimates & thoughts around this
- Claude did a medium-level feature in about half the time that a good developer with enough context about the code base would have taken.
- Time is being saved because we went from writing specs + coding + debugging + reviewing to specs + debugging/reviewing.
- Since the AI agent is working on different parts of the codebase, like frontend & backend, the time required to coordinate efforts to integrate these parts is saved.
Way forward
- Context is the new oil
Context for an AI agent doesn’t just mean a dump of some Word document(s). It means structured in a way that guides the agent to do specific tasks on your behalf. Imagine how far we have come from machine-level coding to languages like C, C++, to Python, and now to writing contexts.
- We (engineers) are nowhere close to being replaced
Yes, you heard that right. I don’t think we are going anywhere anytime soon. Our working style will & has evolved for sure. The next set of skills for us becomes planning, architecture, product engineering, debugging, and reviewing.
- Embrace rather than envy
Are we saying we are better than a model trained on millions of repos, which is equivalent to a few hundred thousand developer brains? I don’t see a good reason for developers not to embrace AI tools like Claude
- Keep human in the loop
I will never be able to trust code generated by AI 100% any time soon. I believe it is a good practice to review the AI agent’s work at every stage. Nothing should go directly into production.
- Context engineering is replacing vibe coding
I just like the word context engineering better than vibe coding. By giving the agent more context we are just fine tuning it for the task at hand. You cannot ask it to just build an ETL platform with a nice UI. Even if it works, it will not be something you had in mind.
- Being calculatively ambitious
Although, seemingly, the agentic tools are doing a great job with enough context. But since we are in our early stages of using them, we should take time validating/testing/debugging/doing quality checks to make sure the code generated is production-ready. That being said, we should be more aggressive on the features we plan to deliver in our dev cycles.
If we keep developing this way, are we going to just forget the low-level details of the codebase? That might just happen, too. We need to make sure, every now and then, we get our hands dirty deep in the codebase to take an “AI break”.
Anecdotes from our team
“As a software developer, I’ve been using AI (Cursor and CodeRabbit) for coding, refactoring, generating tests, and generating scripts that automate repetitive tasks. During Spike Week, we were able to rapidly spike a chart feature which otherwise would have taken a minimum 4 weeks.” – Himanshu
“It would have taken me 2 weeks just to understand the backend and frontend Dalgo repositories, but with Forgecode and Claude, within 20 minutes I was already developing.” – Pratiksha
“I had no idea we could do so much in so little time and with such ease. Planning is more fun now because now you get to order around a bunch of developers (agents)” – Ishan
