Why do we migrate data?

We typically migrate data for several reasons -
A storage device may be sending out alerts that it is not well, and you want to get the data off it as soon as possible to prevent a service outage.
As data storage gets older it becomes more expensive to maintain and more error prone. As a very general rule, a disk storage system lasts for about 4 years, after which it is cheaper to replace it than it is to keep maintaining and upgrading it.
File sizes and disk sizes get ever larger so occasionally you need to consolidate several smaller disks onto one larger disk to simplify maintenance and reduce the disk headcount. This is especially an issue in the mainframe world, where logical disk capacities of 3 or 9 GB were standard until quite recently, but file sizes are typically hundreds of Gigabytes and overall capacity stretches to Terabytes. Every logical disk that is attached to a mainframe requires an address called a UCB (Unit Control Block) and the number of available UCBs is limited, so 27, 54 and even bigger logical disk sizes are now becoming the norm. Open System disks, of course, have been much bigger that that for several years.

Disruptive Maintenance

What you could call the traditional way to migrate data is to shut all your applications down so that nothing is accessing the data, then copy the data to the new disks, point your applications to the new disks, restart your applications again and test. We used to do this with quite critical business systems maybe 20 years ago when we could get the applications down for a weekend, but generally speaking this is not practical today. Businesses require their applications to be up 7*24 or as close as possible and data sizes are so big that migration takes a considerable time. However the requirement to move data still remains, so various tools have been developed to make this as non-disruptive as possible.

Hardware Data Migration tools

Two different types of tool exist, Hardware Virtualisation and Disk Mirroring.

Many devices now virtualise data so that the 'disk' you see in Explorer is actually carved up and spread over several physical disks. Redundant copies pf the data may also be held for resilience. The devices have the ability to move data around internally without affecting applications, and so free up older disks. Some examples of this are the HDS USP device which can virtualise external disk subsystems behind a USP controller. Once an old subsystem is virtualised, the data can then be moved transparently to the USP owned disks, or to another virtualised external disk subsystem. While best done at a quiet time to avoid possible performance issues, this migration can happen with no impact at all on running applications. IBM's SVC virtualisation controller works in pretty much the same way, both systems will virtualise and migrate Open Systems data 'on the fly'. The difference is that the SVC does not support mainframe data at all, and while the USP does support mainframe data, the initial virtualisation process is disruptive.

Just about every major disk subsystem has a facility to remotely mirror data to another subsystem, though usually both subsystems need to be from the same manufacturer. You can use this to shortcut the disruptive migration option. You can set up mirrors between two multi-terabyte subsystems and leave it for some time to complete the mirror process. Once you get there, then you need to stop your application so that no more updates happen to the old system, stop mirroring and convert the target mirror so it can be used for primary data, (the Remote Mirroring section explains this) then point your applications to the target disks and restart them. The non-disruptive mirroring part can take several days, the disruptive swap-over just minutes.

Software Data Migration tools

Software tools exist to transparently migrate disks between subsystems. Two examples are FDRPAS and TDMF, both of which have their own pages on this site.

However, all of the options above move disks, not data. That is, at the end of the move the disk will be exactly the same size as it was when you started, so none of these will assist if you want to make disks bigger. Two mainframe migration tools exist to help here, FDRMOVE and LDMF. These products move datasets, and if they can get exclusive use for a file then they will just move the data. If they cannot get exclusive use they will copy the data and maintain it up to date while waiting for the ENQ to be released so they can swap the file to the new location. It may be necessary to stop applications briefly for this to happen.

Deleting the data

OK, so you have moved all your data off your old disks successfully and you are ready to power them down and ship them off to the scrap yard. Before you can do that, you need to wipe all the data off them, so it does not fall into criminal hands. FDR Erase is a product that can so this for you.

Data Migration

Enterprise Disk

Lascon latest major updates

Welcome to Lascon Storage. This site provides hints and tips on how to manage your data, strategic advice and news items.

back to top