Data Challenges
The development team of the fantasy sports platform uses MongoDB to store data from multiple departments, including product, marketing, and sales, due to its schema-less nature. This allows the team to effortlessly store unstructured data while the constant schema alterations happen during the product's development.
In order to build reports, the platform data needs to be integrated with Power BI, which is challenging for Darsh’s team because the schema is constantly changing on MongoDB. Additionally, it becomes very difficult to manage if too many queries are firing on MongoDB. For instance, if the engineering team at the fantasy sports platform is loading data onto MongoDB while Darsh's team is simultaneously retrieving account statements for generating insights, the server becomes strained under pressure.
Therefore, Darsh needed to move this vast amount of data to Snowflake, the primary data repository for the client. This would allow him to perform extensive analysis and generate Power BI reports for them.
However, the challenge was not limited to loading this data but also managing the data schema at the destination. The tables were extensive, including approximately 200 fields and around 200 million rows. To manage these tables and their schema modifications, Darsh would need a team of about 15 engineers. This created a significant bottleneck for Darsh's team.
Therefore Darsh had to look for a data pipeline platform to help them load and map data to the Snowflake with limited resource bandwidth.
The Solution
Darsh chose Hevo to effortlessly mirror the complete client data from MongoDB to Snowflake, including product, marketing, sales, and other business data, due to Hevo's scalable architecture and automated schema mapping capabilities.
He then connected Power BI to Snowflake, enabling him to create self-serving reports for the client's leadership teams and departments like sales and marketing.
Everything and anything that is a requirement other than a feature development then has to go to Snowflake or a tertiary database to decide in the future. And that will only be possible through Hevo. It's like that bridge that you need to cross the ocean.
However, this was only the beginning for Darsh. He discovered a new use case to leverage the centralized data at Snowflake, and started reading product dashboard data from Snowflake instead of MongoDB. For example, he fetched the data for the customer wallet balance dashboard, which includes the overall financial statement of customer activity, from Snowflake.
This resulted in lower costs for querying product data and faster processing. They have already shifted 40+ production product API calls from MongoDB to Snowflake, with plans to shift more.
Product dashboards are crucial for an online betting app, as even a single record can significantly impact calculations. Thanks to Hevo's 99.9% data accuracy, zero data loss, and low latency, Darsh confidently made this transition and delivered more value to the client
Key Results
Darsh is extremely impressed with Hevo's ability to automatically maintain consistent schema changes. If not for Hevo, Darsh's team would have needed to assign 15 engineers solely to monitor and implement these schema changes. He is confident that he can concentrate on other important tasks and rely on Hevo to keep their data schemas up-to-date without any need for manual intervention.
Switching from MongoDB to Snowflake to read the product dashboard's data saved a whopping 20% in data ops costs!
With Snowflake, querying for data became more efficient and cost-effective. The first time they ran a query, it took about 7-10 seconds to get results. But with each subsequent query, the fetch time dropped to a mere 10 milliseconds by the third run!
I'm able to work with their team efficiently and will be able to bring more value onto the table, and every such innovation or development would have a direct impact at an organizational level
After witnessing the impressive results Hevo delivered for its fantasy sports platform client, Wohlig plans to start integrating this same approach for all its other clients. With the level of support and assistance they provided, Darsh is confident that they could achieve similar success for their other clients as well.
Excited to see Hevo in action and understand how a modern data stack can help your business grow? Sign up for our 14-day free trial or register for a personalized demo with our product expert.
