The year that was 2024: Dalgo reflections

As we come to the culmination of the year 2024, we reflect upon the incredible ride Dalgo has made in simplifying data management and empowering our NGO partners to drive meaningful impact. It’s been a year of impactful collaborations and we’re excited to share the highlights with you.

Empowering NGOs: A Growing Community

This year, Dalgo partnered with 11 new NGOs, bringing our total to 15 organisations using our platform to streamline their data processes. We signed SHOFCO, our first NGO in the African continent this year and it has given us visibility into the regional nuances.  

From managing water quality and conservation initiatives to improving educational outcomes, Dalgo has been instrumental in enabling data-driven decision-making for our partner NGOs.

Key Milestones in 2024

Feature Enhancements:

We added several new features to the platform aligning with our Product Roadmap, published earlier this year. Below is a quick summary,

• Advanced Analytics
  • AI Data Analysis – This feature allows you to leverage AI and write prompts for the  data that’s stored in your warehouse. Explore a range of possibilities; generate quick insights on your data, learn more about the data quality of your dataset, or even summarise qualitative data!
  • Data Insights – generates quick statistics on the columns of your tables – minima, maxima, most common values, and null counts to name a few

• Data Quality and Governance:
  • Role-based access controls It allows you to add different users to the platform and assign them a role that grants the added user relevant ‘view’ or ‘update’ permissions
  • Data Quality with Elementary – we have integrated this popular data quality tool to summarize your dbt test coverage 

• Usability
  • UI for Data Transformations: We launched a low-code, user-friendly interface for data transformations, making it even easier for organizations to process their data efficiently.

• Operational Management
  • Settings Panel – Configure the platform settings for User Management, AI etc. We also show details of your Dalgo subscription and the versions of our underlying components
  • Notifications System Communicate important news and updates to the clients
  • Superset Usage Dashboards This dashboard gives you an overview of how well the visualizations have been adopted by the users in your organization. Insights drawn from here can be used to optimize dashboard utilization and decision making throughout your organization
  • LLM log summarization When pipelines fail, we summarize the logs generated by the underlying tools into a human readable summary

Operations

1. We  performed a major upgrade to our underlying ETL tool Airbyte – “Destinations v2”. This came with a very useful feature for automatic normalization from JSON to column data, which otherwise was done via SQL in our clients’ dbt pipelines
2. We upgraded our infrastructure as our clients grew and their sync needs increased, and encountered a few hiccups along the way. These upgrades have sufficed for now but looking forward we are moving Airbyte to Kubernetes as they recommend 
3. We are also running up to four instances of Superset on a host to rationalize costs 
4. Processed over 2.7 billion (not necessarily distinct!) data records across various projects since January 2024. After removing a couple of outlier months, we ingest around 153 million records every month.

Consulting

• Consulting work has kept us busy, serving NGOs like ATECF, Arghyam, SHOFCO, STiR Education, SHRI, Arogya World. Our engagements with our clients has helped us build deeper understanding on managing data pipelines for them, the nuances and challenges resulting into product/platform feedback. There is that excitement of seeing the data transform into dashboards for the NGOs which helps them into correlating their program performances to organizational goals. We have often seen the requirements evolve as we make progress as the data transformation and visualization leads them to think more on their org needs. We have so far this year spent in 1000+ hours of consulting, which does not include the proof of concepts

Business Development

• We introduced free 10 hour proofs-of-concept for Dalgo, a great value proposition to our prospective clients. This has helped us do a quick turnaround of showcasing the platform capability and in helping our clients like SHOFCO, Ummeed in decision making to go with the platform. We did a PoC drive in Bangalore and Mumbai locations. 
• We have been conducting monthly webinars on various topics relating to Dalgo including case studies, new feature releases etc. This has helped us build awareness for the product and increase outreach. We have been seeing increasing interest in the product through these sessions.
• We continue to do the outreach on the social media platforms like linkedin and have seen growth in our followers
• We also presented Dalgo sessions for 4th Wheel, Ashoka ASPiRE

Ecosystem

We strongly believe in contributing to the ecosystem be it the Open Source community or the Social sector initiatives. This year in addition to the culmination of the DCP Cohort 1 early in January, we initiated our second cohort in August along with Dasra and Goalkeep, in a mission to amplify the impact of data-driven decision-making in the nonprofit sector. Altogether we engaged with 22 NGOs through this program in 2024.

We continue to contribute to the Airbyte platforms in terms of the connectors we build for the NGOS.

We did presentations on Dalgo in the FOSS and PyCon Conferences this year. Here is the link to the video of the presentation at FOSS How Dalgo automates data insights and helps nonprofits learn better from their data.

We continue to hire interns through the C4GT program. We developed the Notification Systems through this internship and was a significant value add to the platform.

Team 

The team met several times throughout the year for the Tech4Dev as well as the Dalgo Sprints in Bangalore. These sprint meetups have not only accelerated the platform or consulting work but also helped us build camaraderie within the team.

In 2024, Himanshu Dube and Pratiksha Rao joined us in our engineering and consulting teams respectively. They have ramped up well and have been contributing effectively to our product goals. They have captured their experiences in the blogs below,

• Himanshu’s first sprint in Bangalore
• Pratiksha’s Shillong Sprint stories  

Poorva, too joined us as a volunteer to work with us on our Kubernetes mission

Success Stories

1. Our engagement with ATECF has been a long standing one and an excellent example of co-working with AVNI and Glific teams to deliver enhanced customer experience for the Water body Rejuvenation project
   • ATE Chandra Foundation’s Waterbody Rejuvenation Project
2. The first AI feature that we developed for the platform was for the use case shared by STiR Education and we closely worked with their team in building the prototype for the Qualitative Data Analysis feature. This further evolved as AI Analysis using Dalgo with support for both Qualitative and Quantitative data analysis.
   • https://projecttech4dev.org/qualitative-data-meet-dalgo/
3. Our support team has been doing a fabulous job and has been lauded several times by our clients for quick turnaround times.
4. We have partnered with our ecosystem partners, Goalkeep and Dhwani RIS in successfully onboarding 2 new NGOs each on Dalgo. 

What we learned in 2024

As we work with more and more organizations, our understanding of the ecosystem has gotten better through the data lens. This has led us into evolving our product as well as processes further. The key learnings being,

1. Simplified Data Access: Democratizing access to critical insights across all levels of an organization from field level workers to leadership and how we can do this effectively through our onboarding process.
2. Process Improvisation: As we understand our clients better, we continue to take feedback from the engagements and reflect upon them to strengthen our consulting and engineering processes 
3. Platform Scalability – As we add new clients to the platform, ensure platform scalability and hence user experience. 
4. Longish Sales cycles – Our Sales cycles have been very long and often dependent on the NGO’s priorities. How do we address this key issue in the upcoming year?

Road Ahead: plan for 2025 

As we step into the year 2025, we plan to join hands with our partners to deep dive into their areas of expertise and identify solutions to their use cases that will help equip them better in their data driven journeys. Some of the feature experiments we plan to run with our NGOs are with the below themes,

1. Chat with your Data converse with your data in plain text English
2. Storytelling using Data Leverage the data, dashboards and other collective information to tell your impact story
3. Simplified Data modelling Simplify data modeling and reduce the time taken for onboarding

We will continue to strive for operational excellence and ease of use, as we progress in the coming year. 

And we plan to jointly work with the NGOs during our Jan 2025 Sprint in understanding their needs and co-developing features for them through the quarter. We continue to engage with our ecosystem partners as well to take our mission forward.. We are truly excited!

A Heartfelt Thank You

None of this would be possible without the trust and collaboration of our clients, the dedication of the Dalgo team, support from our leaders and the feedback from our partners. Together, we’re making data more accessible, actionable, and impactful for the organizations driving social change.

Here’s to another year of growth, impact, and transforming data into actionable insights!! 

Stay Connected

Follow our journey as we continue to innovate and support the social impact sector. Reach out to us at Dalgo Support or join our community on Discord.