Before starting a large-scale data migration project, you should probably ask the question, “Why do companies undergo data migration in the first place?” Let’s take a look at some of the possible reasons for migrating data to a new system or solution.
Move to New Hardware
Hardware technology is constantly changing and improving by leaps and bounds in processing speed and bandwidth. To take advantage of these gains, you have to keep your hardware as current as your budget allows without reaching the point of diminishing returns.
Technology Changes
Software changes faster than hardware, and new algorithms increase the speed of data analysis and throughput. If your data is so large that you have trouble processing it into meaningful information then you have to try to take advantage of any new gains that may emerge.
Compliance
The laws and regulations are constantly changing at federal and state levels without much notice. If you aren’t keeping up with them, you could incur stiff fines and penalties.
Limited Staff
If you don’t have the amount of well-trained staff needed to manage your current system, then you will need to employ solutions that maximize the effectiveness of the staff that you do have.
Management Goals
If management’s direction and initiatives are demanding things that your current solution can’t provide then you will need to find a new solution that can. Most likely the new solution will also be subject to a budget dictated by management and could prove to be a challenging task.
Pre-Migration Strategy
Adequately preparing for a large-scale data migration is imperative if you want to fully complete the project in a reasonable amount of time and with a minimum of unexpected side effects. Let’s take a look at some of the criteria to consider when preparing for a large scale data migration.
Amount of Data to Be Migrated
The volume of data could possibly be so large that it will break your project timeline just to transfer it. You might have to invest in compression software to minimize the size of the data. You might also consider investing in an infrastructure upgrade to increase bandwidth and throughput.
Target Capacity Planning
The storage capacity and retrieval mechanisms of your new data management system will need to be thoroughly investigated to not only make sure that it can hold your data but that it will be able to work with it after it has been imported.
Performance Expectations
Will the new system be able to quickly handle data transactions and queries to your satisfaction? If your reason for migrating from your current system is because of performance, this becomes the most important feature of your new system.
Existence of Backups
Do you already have your data backed up or archived so that it can be quickly uploaded to the new system without impacting current daily operations? Will you use the backups and then form a reconciliation process to transfer the new data?
How Servers Are Mapped
Make sure that someone in your organization has a firm understanding of the current relationship of all servers and hardware so that you can recreate or augment your server mappings in the new system.
Communication
Management should be kept in the loop throughout the process to appropriately manage their expectations. If there is any way to foresee complications or problem areas that may occur during the data migration, it’s better to state them ahead of time. This will prevent management from being surprised and frustrated when problems do occur.
Common Problems Associated with Large-Scale Data Migration
Awareness of common pitfalls and where they might occur is paramount to preparing everyone involved to expect the unexpected. Some problems that may arise include:
- Unsupported operating systems and software that don’t play well together
- Custom applications that are difficult to migrate or won’t scale to fit the new strategy
- Various physical requirements such as licensing and data backup media
- Data or servers discovered after the data migration process has begun that get added through change requests.
Don’t Get Too Comfortable!
Every large-scale migration project comes with a unique set of challenges. In many cases, major challenges don’t show up until closer to the end of the project. This can be frustrating for your team as well as the team leading the data migration. Knowing what challenges may arise can help make the process smoother.
The last 20% of a project is often the most challenging. At this point, all low hanging fruit has been harvested and it’s time to solve problems that don’t necessarily have a simple solution. Let’s take a look at some examples of potential pitfalls that may occur.
Random Things You May Have in Your Environment
You never know what you might find on a project until you dig in. There may be any number of strange routers, network security appliances, servers, and database configurations put together by staff of varying skill levels. Try to go through everything with your existing staff, especially the ones that have been there the longest. They may be your only clue as to how things have been configured.
Unsupported Operating Systems
Old servers such as XP or Windows Server 2003 that have expired software licenses should be addressed at the beginning of the project, if possible. Those servers should be upgraded or a strategy should be formulated to transfer any useful data off of them before the migration begins. The old servers should then be left running in their current, untouched state until well after the migration project has been completed. This is just to make sure some forgotten critical item doesn’t still reside on them.
Databases
The data from databases whether they’re physical, virtual, or exist on old or unsupported software should be transferred onto a new database management system that can be easily integrated into the new data management solution.
Other Random Failures
Hardware, software, and network failures are hard to predict. Build time into the project for unexpected failures, or things not working together as they should so you have ample time to troubleshoot them.
“Gremlins” from Older Projects
Staff turnover is a part of any business environment, which means the main players in a project carried out years ago may no longer be available to answer questions. Take a look at any existing documentation from older projects. Also, consult with anyone who is still remaining that knows about the purpose of the project. You might also ask if anyone still has contact with any of the previous employees who had experience with the system. Try to gain some understanding of how the old project was structured in an effort to figure out everything it touches and how valuable it actually still is.
Project Considerations
A successful large-scale data migration project should have minimal impact on normal business functions. Preparedness is key to making this happen. Beyond understanding where challenges and pitfalls commonly occur, we also recommend other forms of preparedness.
Plan According to Requirements
As with any project, prior planning is the key to having a successful endeavor. Unexpected problems will almost always happen, but you should always try to minimize the possibility of potential problems and build time into your project plan to address them when they do occur.
Have a Contingency Plan
Think about potential points of contention in your plan and always try to formulate a backup plan for things not working together as expected.
Implement Checkpointing Where Possible
Try to build milestones or checkpoints into your plan, so that you know when stages are complete and how far along you are in the process. Also, be aware of tasks that depend on other tasks being finished on time so the schedule is not impacted. Communicate frequently with management and internal stakeholders to adequately manage their expectations.
Buy Enterprise-Class Hardware
Don’t skimp on hardware. Buy more than enough to adequately work for your situation and that will last well into the future. If you don’t, you’ll be right back where you started before you know it. Also, choose hardware that has a good reputation and that is popular in the industry so that you can easily find technicians that can perform maintenance and upgrades. Make sure the infrastructure will support the new hardware as well or it will all be for nothing.
Use SanityRx
Sanity Solutions employs a team of certified professional services experts dedicated to providing a wide range of infrastructure services to our clients. These services include project assessment, solution deployment, data migration, and system management to help streamline projects helping you execute effectively and efficiently. Sanity Solutions has a strong reputation as a trusted partner that assists clients through the assessment process and architects custom solutions to safeguard their data and infrastructure. Why not consider contacting Sanity Solutions today to see how we can help you manage your data migration project.
