Tape Futures

Just like Mark Twain, the announcements about the death of Tape are exaggerated. As explosive data growth continues year on year, it seems that tape storage is the only thing that can keep up with it, especially for backups and long term archives of stale data. At present, there are three main types of tape drive; the LTO7, the LTO8 and the 'enterprise' TS3592JE. You could also divide the customer base into three types to match these drives; the SME, large customers with lot of data and media customers who have lots of data and need to be able to get to it quickly.

SMEs want automation but usually can't afford fancy tape libraries, so they are moving backup and archive to the Cloud. However, for those who still have their own tapes, the LTO7, with a 6TB raw capacity and 300Mb/s transfer speed is cheap and acceptable
Companies with a large storage requirement, like financials, medical companies, and the Cloud providers need room to store a lot of data at a reasonable access speed. The LTO8 drive can write out 12TB native and 30TB with compression, at 360MB/s (that's just over 1.2TB per hour). They usually front those drives with a virtual tape disk cache to boost the write throughput and improve tape usage.
The TS3592JE can hold 20TB native, with a burst transfer speed of 500MB/s or about 1.7TB/hour. This is more suitable for people who need fast access to large amounts of data held on tape. Examples could be the film or TV industry.

LTO7 uses BaFe particles, as originally used IBM 3592 cartridges. To put it in simple terms, BaFe are better magnets than MP, and BaFe is an oxide rather than a metal alloy so it does not demagnetize over time. This means that the BaFe particles can be made smaller than MP and so even more data can be packed onto a tape.
Some of the reasons for why tape's demise was imminent were:

Tape was always traditionally used for backup, and while disk based backups are available, explosive data growth makes them relatively expensive, even with de-duplication and compression. A suitable backup storage strategy seems to be disk-disk-tape, or DDT. The reason for this is that 95% of restore requests are made within 14 days of a backup, so it might be worth keeping backups on disk for the first two weeks, but after that, tape is suitable and cheap.

Tape is also useful for long term archives, and Active Archives. Tape has a number of advantages over disk when used for long term archive. Cost is the obvious one, but less obvious is that since tape reliability has improved by 700% in the last decade, it is now more reliable that disk for long term retention. The data stored on tape can also be block, file or object.

Active Archive is a relatively new idea where a number of companies are trying to establish standards for accessing data on different types of storage. The difference between Active Archive and HSM is that whereas with HSM, when a migrated file is required it is recalled back to primary storage, with Active Archive, files are accessed from wherever they reside in the storage hierarchy.

IBM is driving a new initiative to introduce a common file system for tape, called LTFS. This has obvious advantages for Active Archive, as it will make it easier to access individual files on tape. It will also have benefits for long term archiving, as currently most applications use their own data storage formats. If a universal format is agreed, then it should be much easier to read 30 year old data. There is no guarantee that any current backup application would be available 30 years from now to read a proprietary tape format, but if it supports an open source piece of software, like LTFS, then the chance is much higher that the LTFS drivers will still be available.

The improvement that comes with LTFS is that the media is mounted and read by the operating system instead of the application. There have been other attempts at introducing open tape, but because IBM has released LTFS as open source and because it is used at a file system level, it has a better chance of adoption as previous solutions were vendor proprietary.

LTFS combines well with LTO5 and above. LTO5 introduced an ability to carve a tape up into two media partitions. LTFS has a directory that details the contents of a tape. This can be placed in Partition 0, which is small and can be quickly read to allow anyone to see what is on the tape. Partition 1 is larger and is used to hold the real data.

Not only would an LTFS export option be ideal for situations where large volumes of data need to be shared with others, but it could be seen as a best practice for archive data.

Another potential use for LTFS/LTO5+ is for large data transfers to the cloud. One of the issues with the cloud is that the internet is not geared up for transferring terabytes of data. Customers who require large data uploads frequently do this by shipping magnetic disks to a cloud data centre for upload. An LTO5 tape that could be read anywhere with a LTFS driver would be an easier and cheaper option.

Where does Virtual tape fit in? Virtual tape consists of software emulated virtual tape drives which write to a disk buffer. The disk buffer is then flushed out to tape when enough data has been stored. The Virtual Tape section explains this in detail. Virtual tape can be used to fill tapes, and can even allow concurrent access to 'tape' data when it is in the disk cache. However the down side is that the cost ratio of disk to tape is not as good with virtual tape.

So the conclusion is that magnetic tape storage is still being developed and improved, and there is no sign of it disappearing for some time yet.

back to top

Lascon updTES

I retired 2 years ago, and so I'm out of touch with the latest in the data storage world. The Lascon site has not been updated since July 2021, and probably will not get updated very much again. The site hosting is paid up until early 2023 when it will almost certainly disappear.
Lascon Storage was conceived in 2000, and technology has changed massively over those 22 years. It's been fun, but I guess it's time to call it a day. Thanks to all my readers in that time. I hope you managed to find something useful in there.
All the best