How AND Digital Uses AWS Database Migration Service
Database migration is a nerve-wracking process. Moving production and sensitive workloads requires careful planning and coordination between many moving parts. This blog will explain what you need to think about before starting a migration to make the process less daunting.
When customers come to us to talk about their migration, they often fit into one of three categories:
The first want to move their on-premises database to the Cloud. But they’re worried about availability, data integrity and making sure everything is up and running on time.
The second is looking to take the next step and move to a truly Cloud-native platform, like Amazon Aurora. This can create challenges around schema transitions that require expert help.
The third come to us for help because they do not have the internal skills to cope with a complex database migration.
So if you’re planning a database migration, the first challenge is to think about which category you fit into and what you want to achieve. Is now a good time to rethink your database strategy? Or do you just need a lift and shift to the Cloud?
That means thinking about future needs, as well as what the database is doing right now. This is a great opportunity to take a broader look at your technology strategy and it could also be a good time to take a more data-centred approach, for instance – making better use of the information you already have or thinking about what new data streams could add to your decision-making capabilities. Will what you’re building still be fit for purpose in a year or should you take this chance to change things up? Are there any benefits in moving to a serverless approach, or using something like a data lake format and querying objects directly in S3 with Athena?
There are a number of migration services you can use, but this blog’s focus is the Amazon Web Services Database Migration Service (AWS DMS). It offers support for most open-source and commercial databases, including both relational and non-relational. Your source database remains online to minimise the downtime to applications and is persisted until you’re ready to make the switch. It’s important to consider the type of migration you will need. There are two basic types of migration available using AWS DMS:
Homogeneous – These are the simplest migrations, when your source and target database engines are compatible, such as an Amazon Elastic Compute Cloud (EC2) MySQL database to Amazon RDS for MySQL. With similar codebases and schemas, these migrations require only defining a source and destination and then allowing AWS DMS to handle the rest. The source can be located on-premises or in the Cloud, as long as the destination is on AWS.
Heterogeneous – These are more complicated and involve a two-step process. If the two databases are incompatible, it will require schema and code changes before the migration can start. This process involves first using the AWS Schema Conversion Tool (SCT) to carry out the required transformations. AWS DMS then converts the data types on the fly and replicates them at the destination.
To help you make your mind up and to fully understand what service is right for you, AWS offers a range of discounts and subsidies for database migrations. For instance, you can get six months use of the AWS DMS Free Tier if you move to Amazon Aurora, Amazon Redshift, Amazon DynamoDB and Amazon DocumentDB.
DMS is a great tool for database migration that removes a considerable amount of complexity, but it has its limits:
Complex heterogeneous migrations – While you might want to switch to a new database to improve performance or cope with more data, DMS and SCT do not support all formats. Some complex migrations can require manual coding, which brings its own challenges.
Continuous replication limitations – DMS is unable to replicate some database elements – such as users and privileges – that aren’t directly derived from the table data.
Performance – DMS struggles with very large databases and can slow the source databases during scanning, forcing you to overprovision capacity. Sources that generate very high amounts of throughput data can lag in replicating that data to the destination. This makes long-term continuous replication difficult for some environments.
AWS service limits – DMS has regional quotas. These are generous, but you cannot store more than 30,000 GB in your replication instances. And if you exceed 100 API requests per second, the excess will be throttled.
There is no doubt that moving any key business database is a daunting task. Even the most experienced professional will admit to a touch of nerves on the day such a move is made. But with proper planning and preparation, even complex migrations can be achieved without negative impacts on the business. And hopefully, this blog has helped to explain what preparation is required.
But beyond just ‘business as usual’, shifting a database can be a great opportunity to up your wider technology strategy, adding capabilities to make better use of data to drive better decision making and better service for your customers.
AND Digital can help you in your Cloud journey. We have a wealth of expertise in Cloud migration initiatives and can advise you on whether DMS is a good choice for your workload.
If you want to know more about AWS Database Migration Service, including how it can work for your complex Cloud deployments, get in touch with us at [email protected].