So, you have details about your server, you know what sites you want to move, what versions of software you need, and have informed everyone of your plan, and you are ready to start actually getting things set up and copied.
Where do you start? What does the execution look like?
In my mind, the migration is broken up into four major phases:
- Planning
- Initial sync
- Testing
- Final sync
These are interspersed with configuration, troubleshooting, testing, troubleshooting, scheduling, and more troubleshooting. Let’s break these phases down so you have a good idea of what actual steps are involved.
Planning
This phase can take the longest amount of time in some cases, but it is usually eclipsed by the testing phase. This is the point in time when you figure out what you need to move, when, and where. It revolves around a few key questions:
- What do you want to get out of your migration? Are you upgrading software, refreshing hardware, changing hosts, or something else?
- What type of sites are you running? Do they use HTML, PHP, Java, or Ruby? Where are their files located on the server, and where are the configuration files?
- What are the external dependencies for those sites? Do you need ImageMagick? How about FFMPEG? Any special PHP modules? What about third party hosting tools, like a CDN?
- How is your server currently controlled, and what kind of access do you have? Do you use cPanel, or do you do everything from the command line? Maybe you only have FTP access; how do you get at your databases?
- What versions of primary software do you have on your source machine? What type of web server (Tomcat/Apache/Nginx) are you running? How about databases (MySQL/PostgreSQL/MariaDB)? Any helper software (Varnish/Memcache/Solr)?
- What is the topology of your current and planned web server and database server? Are they on the same machine, or do you have a load balanced setup with multiple back-end servers? Has replication already been enabled?
From these questions (and the countless more I’m sure you thought of while reading them), you should have a good idea of why a plan is so important, especially if multiple people will be working on the migration at the same time. When creating your plan, you will spend a bit of time in the server, so having full root-level access is ideal, so that you can read all of the appropriate configuration files.
Your plan should have details on it that encompass these points, and maybe more, depending on your migration:
- Server login details
- Domains to transfer
- DocumentRoot list and approximate disk size
- Database names and users (and passwords), and approximate disk size
- Email accounts to configure (with passwords), and on-disk size
- Non-hosting FTP or SSH users to add
- Server inter-connections (if multi-server)
- Non-docroot folders to transfer
- 3rd party software to install, and required software minimum/maximum versions
Now that you have all the information you need in front of you, you can determine a timeframe for configuring the server, copying over the files, testing, and finishing up.
Initial Sync
This is the real meat of the migration, where the environment is set up and data is seeded over. This portion of your migration should involve no downtime for your domains. It’s important to remember that you are copying your sites, not moving them.
What steps do you have to follow for your initial sync? It all comes down to what’s in your plan, but generally, you should ensure that the major software you need is installed and at a compatible version (Apache, MySQL, PHP, and/or whatever other webserver and database applications you use), as well as any tertiary programs that your site uses, like cache or search utilities (Memcache/Varnish/Solr) or processing programs (FFMPEG/ImageMagick). If any recompiling needs to happen, it should happen before you copy any site files. This sets your environment up to receive the data in a compatible format.
If necessary for your migration, create any domains or accounts that need to be in place with document roots, or create any databases and grants that need to accept data. Make a note of these folders and database names so you can import data correctly.
Then, the hands-off portion. Start copying your files into place. There are a wealth of utilities available for this purpose, from rsync to scp , or even lftp in low-access cases. As with the planning stage, it is ideal if you have full access to all machines involved in the migration, and even better if the two machines which are sending and receiving data connect directly to each other. Put the files in the folders and the tables in the databases.
Next, once all of the data has arrived, double-check the configuration of your server and make sure the data will work with the configuration. If it doesn’t, modify either the configuration or the data as needed.
- Is your site coded for PHP 5.3, but you installed PHP7? Downgrade, or update references to deprecated or missing functions in your code.
- Did the files arrive with the appropriate permissions for your PHP handler? Change the ownerships and permissions, or select a more lenient handler.
- Do the configurations for the CMS you use connect to the appropriate server and database using the proper database user? Rename the database on the server or in the configuration file, or add the appropriate grants.
- Are there any connections hard-coded to the IP of the old server? Update these to localhost.
If you have a good plan in place, you should run into very few errors, as you will have already accounted for each of these problems.
Testing
The fun part! Get everyone who uses your site together and have them check over the operation of the new server. There are several ways to do this, and each has its shortcomings, but the most accurate way is to modify the hosts file on your computer to ‘trick’ it into seeing updated DNS. This sends real requests to the server in the format that it will see when it is live (that is, a request for your real domain name). This requires a bit of work on your part, and on the part of those testing, but the accuracy is worthwhile, so there are no surprises when you go live.
Other options include using mod_userdir, if you have it enabled in apache (which uses a format like http://123.45.67.89/~username), or using a proxy like hosts.cx to make a temporary URL to your server. The translation of this service, however, misses some key factors, and mod_userdir, too, sends an inaccurate request to your server (because people don’t know you by your IP). For instance, if you use WordPress, and you send a request in mod_userdir format, as soon as you click a link, you will be redirected to a link prefixed with the siteurl setting, which is probably your domain name. Similarly, if you have a rewrite up to https, you will get the real domain from a redirect, or an invalid SSL warning.
Modifying your site to work with one of these methods is also inaccurate, because that is also not what your site will be doing when it goes live, and you will have more work to do to change the settings back when you are done testing. Again, its worthwhile to modify your hosts file.
As for the testing itself, the format is really up to you with how thorough you wish to be. Check some key components of your site and make sure everything appears as it should. Are all of the CSS files loaded? Do all your links work? Are there any visible PHP or formatting errors? Try making an order or creating an account, or posting a message, or uploading a file. Can you do everything you did on the current server? Investigate the configuration of your applications and tweak any settings you need to to make the server work for you.
Final Sync
After you finish testing, all that is left to do is make the new server live. Assuming you did not tell the rest of the internet to stop using your website for a few days, you will likely have had traffic or new developments on the source server, including posts, comments, orders, and so on. An update sync of data and databases (and mail in certain cases) will make sure that the files on your new server are very up to date when they go live.
Because you want the new and old server to match, it is best to interrupt public traffic to your sites during the final sync, so that no new orders or posts can be created which could be lost when DNS switches (as you will assume you have grabbed everything).
Depending on your access methods to the source server, the ideal method on Unix/Linux with SSH is to use rsync --update to compare document roots and copy any newer files, along with bringing over a brand new copy of the related database. These two steps together, for most sites, take just a few minutes of syncing.
The --update flag on rsync will compare timestamps between files and only bring newer files over; those files that are the same or older on the source machine won’t be copied again, which means that much less data needs to be pushed over the network.
If you happen to have a larger database for your site, you may want an extended window for your final sync, or you may want to scope your sync to just a few active tables. In extreme cases, you may wish to investigate binlogging for MySQL or a similar function for other database programs, which will allow you replay transactions on another machine, rather than performing a full dump of data.
When your sync of data is finished, you should replay through the tasks you did to ensure compatibility during the testing phase which could have been affected by the sync (for instance, if you had to reset permissions for a certain folder, do so again on the freshly copied data). Then, DNS can be updated at your current nameservers to make the new server live.
Congratulations!
That’s the 100-foot view on migrations; I hope it gave you some things to think about when making your migration plan! If that seems a bit vague, don’t worry; the other posts on this site will zoom in and help you break the process down into pieces that you can replicate and try out during your own migration.
