Once Magento 2 is launched it will become important to provide support for migrating Merchants from Magento 1 to Magento 2. There will be lots of aspects to migration – extensions, themes, custom development, and data. This post focuses on data, specifically the contents of the MySQL database.

To be up front, it is currently not planned to provide much if any tool support for migrating extensions and themes from M1 to M2 as it is recommended to adapt them to take advantage of the new M2 features. This requires human design effort – it cannot be automated. This is being reviewed however on an ongoing basis.

Requirements

So what matters for the migration tool? One of the key requirements identified was migration elapse time. While less important for small sites, this is critical for large sites. A large site could take many hours to days for data migration, subject to the performance of the export/import processes.

It should not be necessary for a Merchant to take their site offline for too long. Performing a full bulk copy of the database for large Merchants is deemed “too long”. This means some form of “catch up” phase after the main bulk copy is required.

During the migration it has been deemed acceptable to place restrictions on operations that must not occur while data is being migrated. For example, a site should not stop customers from placing orders, but while the data migration process is running Administrators can be required to not make changes such as creating new products.

It is also expected that some extension developers will wish to provide migration support of their extension specific data. The level of integration support here is yet to be determined. For example, a simple approach is for extension developers to document how to copy and change the default migration tool configuration files to include their extra data to copy. Another approach is to provide a more flexible, pluggable framework that extensions can automatically plug in to. The latter sounds cool, but is a more work and could be unnecessarily restrictive.

Design

For performance reasons, performing an export of database contents to files on disk, followed by an import into Magento 2 was deemed too slow. The current design performs a direct database-to-database copy of data.

A configuration file is used to specify which tables and columns to be copied from the source Magento 1 database to the destination Magento 2 database. Only recent Magento 1 releases will be officially supported, but earlier releases can be migrated by adjusting the configuration file appropriately. (The community may choose to share such configuration files as a resource to help those on older releases.)

In addition to simple table copies (with simple table and column renaming), PHP code will be able to be plugged in to perform more complicated data transformations. For example, the URL rewrite framework is done differently and so cannot be done with simple record copying. Rather than define complex configuration file rules, PHP code will be used to implement more complex transformations.

Database triggers will be used to capture changes to selected tables (such as tables holding customer order details) while the bulk copy is running. Captured changes will be saved in a separate table to be applied later. Limiting Administrator actions reduces the number of triggers that need to be created. (Note that use of triggers to capture updates is another feature not easily supported by the export/import strategy.)

The migration tool will also migrate product images as well as these are a critical part of a store. Advanced sites that do not use the default image storage support may be required to deal with copying images themselves. The goal is to support basic installations for lower-end Merchants to reduce the friction of migration.

Process

It will be recommended for Merchants to do a full bulk copy and then make a second “catch up” pass to apply changes made since the start of the bulk copy. This catch up pass will be quicker to execute and would typically require taking the site offline. That is, the normal migration sequence would be

1. Create database triggers to capture increment changes
2. Perform bulk copy of data (database triggers will capture updates while this runs)
3. Take old site offline (stops all changes)
4. Apply catch up changes
5. Delete triggers in old database
6. Bringing the new M2 site online

Clearly it is desirable to allow dry runs of the migration process, so another requirement is to allow the process to be conducted multiple times. The old site would only be taken off line for the final real switch over.

Conclusions

The above provides a high level overview of how the M1 to M2 data migration tool is currently being planned. Using export and import tools were considered, but in the end it was decided to do a direct database copy for speed. It also makes it easier to use database triggers to capture changes while the bulk copy is running.

And now the question to you – do you think this work is heading in the right direction?

Oh, and also check out Cyrill Schumacher’s independent work at https://github.com/SchumacherFM/Magento2-Data-Migration.