Migrating data from different sources into warehouses can be hard.
Hours of engineering time needs to be spent in hand-coding complex scripts to bring data into the warehouse. That is not it. Often, data streaming fails due to unforeseen errors for eg. the destination is down or an error in a piece of code. With the increase in such overheads, opting for a data migration product becomes impertinent for smooth data migration.
In this post on Hevo Vs AWS DMS, we have made an attempt to highlight the differences between Hevo and AWS Database Migration Service on a few critical parameters to help you make the right choice.
Hevo is a Unified Data Integration platform that lets you bring data into your warehouse in real-time. With a beautiful interface and flawless user experience, any user can transform, enrich and clean the data and build a data pipelines in minutes. Additionally, Hevo also enables users to build joins and aggregates to create materialized views on the data warehouse for faster query computations.
AWS DMS is a fully managed database migration service provided by Amazon. Users can connect various JDBC based data sources and move the data from within the AWS console.
1. Variety of Data Source Connectors
With Hevo you can migrate data from not only JDBC sources, but also from various cloud storage (Google Drive, Box, S3) SaaS (Salesforce, Zendesk, Freshdesk, Asana, etc.), Marketing systems (Google Analytics, Clevertap, Hubspot, Mixpanel, etc.) and SDKs (iOS, Android, Rest, etc.). Hevo supports the migration of both structured and unstructured data. Here is a complete list of sources supported by Hevo.
Hevo supports all the sources supported by DMS and more.
DMS, on the other hand, provides support to only JDBC databases like MySQL, PostgreSQL, MariaDB, Oracle, etc. Here is a complete list. However, if you need to move data from other sources like Google Analytics, Salesforce, Webhooks, etc. you would have to build and maintain complex scripts for migration to bring it into S3. From S3, DMS can be used to migrate the data to the destination DB. This would make migration a tedious two-step process.
DMS does not provide support to move unstructured NoSQL data.
Other noteworthy differences on the source side:
- Hevo promises a secure SSH connection when moving data whereas DMS does not.
- Hevo also allows users to write custom SQL to move partial data or perform table joins and aggregates on the fly while DMS does not.
- With Hevo users can enjoy granular control on Table jobs. Hevo lets you control data migration at table level allowing you to pause the data migration for certain tables in your database at will. DMS does not support such a setup.
- Hevo allows you to move data incrementally through SQL queries and BinLog. With DMS, incremental loading of data is possible only through BinLog.
2. Data Transformations
With Hevo, users can clean, filter, transform and enrich both structured and unstructured data on the fly through a simple Python interface. You can even split an incoming event into multiple arbitrary events making it easy for you to normalize nested NoSQL data. All the standard Python Libraries are made available to ensure users have a hassle-free data transformation experience.
Transformations on Hevo
DMS allows users to create basic data transformations such as adding a prefix, changing letters to uppercase, skip a column, etc. However, advanced transformations like mapping IP to location, skipping rows based on conditions and many others that can be easily done on Hevo are not supported by DMS.
Transformations on DMS
To be sure that the transformation is error free, DMS users will have to hand code sample event pulls and experiment on them or worse, wait for data to reach the destination to check. Hevo lets users test the transformation on a sample data set and preview the result before deployment.
3. Schema handling
Hevo allows you to map the source schema to the destination schema on a beautiful visual interface. DMS does not have an interface for schema mapping. The data starts moving as soon as the job is configured. If the mapping is incorrect the task fails and someone from engineering will have to manually fix the errors.
Additionally, Hevo automatically detects the changing schema and notifies the user of the change so that he can take necessary action.
4. Moving Data into Redshift
Moving Data into Redshift is a cakewalk on Hevo. Users would just need to connect the sources to Redshift, write relevant transformations and voila, data starts streaming.
Moving data into Redshift through DMS comes with a lot of overheads. Users are expected to manage the S3 bucket (creating directories, managing permissions, etc.) themselves. Moreover, DMS compulsorily requires the user’s Redshift cluster region, the DMS region to be the same. While this is not a major drawback, this becomes a problem when users want to change the region of the Redshift cluster but not for S3.
Hevo notifies all exceptions to users on both Slack and Email. The details of the exceptions are also included in the notification to enable users to take quick action.
DMS notifies all the anomalies over AWS Cloudwatch only. The user will have to configure Cloudwatch to receive notification on email.
6. Statistics and Audit log
Hevo provides a detailed audit log to the user to get visibility into activities that happened in the past at the user level. DMS provides logs at the task level.
Hevo provides a simple dashboard that provides a one-stop view of all the tasks you have created. DMS provides data migration statistics on Cloudwatch.
7. Data Modelling
Hevo’s Modelling and Workflows features allow you join and aggregate the data to store results as materialized views on your destination. With these views, users experience faster query response times making any report pulls possible in a few seconds.
DMS restricts its functions to data migration services only. Data Models on Hevo
We, at Hevo, understand the complex processes involved in migrating your data from a source to a destination and have built Hevo to simplify just this for you. With a superior array of features as opposed to DMS, Hevo ensures a hassle-free data migration experience with zero data loss. Sign up for a free trial here.