For the past 9 months I have been fortunate to be working with an extremely experienced team on a large scale project for a major retail bank. The organisation is in the midst of separating itself from another retail bank with the ultimate objective of migrating all account, customer and product data onto a new, independent banking platform. This project has offered me the opportunity to develop valuable experience with regard to data migration strategy which I thought I would share.

Strategic Decisions

Obviously in a project of this type and scale, there is a huge focus on the migration of data from the source systems to the target systems. Despite common misconceptions, data migration is not data that moves from one place to another (unless you think of places as being virtual) but rather is the translation of data from one format to another format or from one environment to another environment. As with most Data Migration projects, the differences in architecture and data formats between source and target require the use of complex transformation routines during the ETL process, requiring significant expertise and understanding of both systems. Data doesn't just get up and walk to another format all by itself!

With this in mind, it is therefore necessary to define the data migration strategy as early as possible. One of the key challenges this project encountered was to decide which migration approach to adopt. Two of the most common strategies are; Big Bang and Phased. These approaches differ primarily in the way that the migration is implemented. Below I have given a snapshot of the advantages and disadvantages of each approach.

Big Bang Migration

A Big-Bang data migration is where an entire dataset is moved from source to target systems in one operation. This is typically carried out over a weekend or planned down-time period. The business users would be using the source system on the Friday but come Monday morning be seamlessly switched to the target system.

Advantages Disadvantages
Shorter implementation time High risk
Lower costs Small details or issues can be overlooked in the rush
Pain and frustrations are condensed into one time period, not drawn out Users have to learn the new system immediately – this could result in a dip in performance
No one has to operate their business in two different operating systems Testing of migration and post migration processes is even more critical
Everyone in the company moves forward on the same day Failures in one part of the system can cause problems and failures in others
Training needed only on the new system, not a changeover  

Phased Migration

A Phased data migration has a number of names - Iterative data migration, trickle-feed data migration, synchronised data migration and so on. In essence, they all mean the same thing - the data will be migrated in smaller increments until there is nothing left to move.

Advantages Disadvantages
less risk Longer implementation time to be fully converted
Employees learn as they go – there is no dip in performance caused by the need to learn the new system Not as focused as the Big Bang approach
More time for users to adapt to the new system A state of continuous change can sometimes be disruptive
Small details or issues can be fixed as you go Can have missing information because each module relies on info from others, so in a transitional period there may be some gaps
Skills and experience are gained with each phase which can help smooth the process as you get further along Temporary bridges need to be made from old to new

So what now? How do you choose?

Many companies don’t choose one over the other – they use a combination of migration strategies designed specifically to fit their own requirements and minimise the risk to their organisation. Let’s look at a couple of examples that Xceed has been involved with:

  • Large scale banking and insurance integration involving a major retail bank – this programme used a combination approach, primarily focussed on using a Big Bang migration where the migration of all of the main banking data was completed over a whole weekend. However to try and de-risk specific elements of the Big Bang migration, certain elements were separated out of the main migration event and migrated at a later date.
  • Major separation between leading payments processing company and a retail bank – this programme adopted a phased migration over various weekends. Each phase was a release based upon a particular system rather than a product. The primary reason the programme chose to take on a phased implementation was because the systems were independent within the source platform. Therefore it was more viable to migrate each system one at a time.

It is essential to choose the right strategy for the company. Remember data migration projects are undertaken because they will support business objectives. It is therefore important to remember that each company is different and thus a migration strategy which worked for one company may not be the best option for another.

To receive expert advice on what migration strategy is best for your programme, please Contact us and we would be delighted to help.