Virtual Tape Principles
Virtual Tape Principles
This page specifically refers to virtual tape that stages data on disk then writes it to physical tape, rather than virtual tape systems with a pure disk back end. It also has a strong IBM mainframe bias. The basic principles of virtual tape are illustrated in the gif below.
Virtual tape vendors use different names for their virtual tape components, but the components are all basically the same, though they are implemented in a different way.
The diagram shows 16 virtual drives which are software emulated. The operating system sees these as real drives, mounts them and writes data out as virtual tapes. The data is not written directly to tape, but is staged, usually in compressed format, to a disk cache. When there is enough data in the cache, the vtape system copies the data from cache to a real tape drive, onto high capacity real tapes. It either frees up the space in the cache, or marks it as available for use when required, depending on the implementation and your settings. Its usually best practice to keep the disk cache full, and reuse space as required. That means that if you require to read a tape which was written previously, it may be already in the cache
The individual components, and some tips on how to use them, are described below.
Virtual tape drives
Virtual tape drives are defined to the operating system just like ordinary drives. From the viewpoint of the host operating system, they look just like real drives, and will accept all the commands, and return all the conditions that a normal drive would. Virtual drives should not be defined with preferred paths, and they must be defined as library resident. The recommendation is to set the Missing Interrupt Handler to 45 minutes. You should review this to see if it is appropriate for your site. To display the current MIH setting, use the MVS command
The MIH is set in a SYS1.PARMLIB(IECIOSxx) member, it can also be set temporarily using the MVS command
Where the hex numbers is brackets are the address ranges of the tape drives
Virtual tapes need to be initialised just like real tapes. The tape labels are held in a VTS catalog, so a virtual tape does not need to be recalled from physical tape, to scratch it. All the data required for label checking exists in the Vtape catalog.
Make sure you do not define too many scratch tapes, as tape data will not be released until the tape is overwritten. If you have 100,000 scratch tapes, this could mean old, unwanted data hanging around in the system for years past its date. There is a new feature in the IBM VTS which manages this a bit better. You can define your virtual tapes as 'optional' or 'deferred' scratch. Deferred scratch means that virtual tapes will not be copied over in the reclamation process, and at that point, data is permanently lost.
Data is directed to an IBM VTS by DFSMS constructs and this can cause problems with foreign tapes. Suppose you define a virtual tape range from V00000 to V49999. Your favourite supplier then sends you a fix tape with a volser of V23456 and you submit a job to read it. DFSMS will intercept your request, divert the allocation to a virtual tape drive and mount its own virtual tape, not your real physical tape. This will happen even if you code in a specific UNIT=xxxx override to try to force the allocation to a specific non-virtual drive. However, nil desperandum, you can force your allocation, but you need to use a special DFSMS storage class, STORCLAS=DUPT@SMS
//INPUT DD DISP=SHR,DSN=CAI.SAMPJCL,VOL=SER=V23456, // UNIT=CART,LABEL=(9,SL),STORCLAS=DUPT@SMS
The disk cache is a buffer which holds the virtual tapes, at least until they are written out to physical tapes. A virtual tape is usually held in the cache even after it has been written to a physical tape, then if it is required for read again, there is no need for a physical tape mount. On an IBM VTS, the disk cache is best kept full, with older, copied virtual tapes overwritten by new ones, as space is required. An STK VSM will issue warning messages once it hits a capacity warning threshold. At this point, automated space reclamation should kick in, but don't ignore the warnings. NEVER fill up the cache on an STK VSM. If you do that, the VSM will grind to halt, and will require a service call to STK to fix.
Physical tape drives
Physical tape drives are usually high capacity IBM TS1140 a SUN T10000C or an LTO-5. The Virtual Tape drives are not defined to the operating system, they are only known to the Vtape system.
Physical tapes cannot be ejected from the tape library until all the data has been removed from them (IBM)
The controlling software looks after the disk cache, and also cleans up deleted data from real tapes.
Reconciliation: checks for invalidated volumes, i.e. virtual volumes which have been re-written. Reconciliation is best run before Reclamation
Reclamation: consolidates stacked volumes which contain expired virtual volumes. Reclamation requires scratch volumes, and requires free drives, so it must be run at a time when recalls are minimal. Keep the reclamation percentage low, between 10 and 30%, otherwise reclamation will run for too long, and not achieve much
The consensus now is that you can put any kind of existing tape data onto virtual tape. In practice, some kinds of data are better than others. Also, virtual tape will provide a lot more drives, and that can benefit all applications. As always, the choice is up to you. Some things to consider are
- DFDSSHSM data.
DFHSM is already good at filling up large tapes, and has an internal mechanism, recycle, to free up deleted data. The HSM recycle function can conflict with the VTS reclaim function, so it is best to set the HSM percent valid parameter low, maybe 10%, and let the VTS do the work.
HSM Backup and HSM Migration data have totally different expiration patterns, so keep them on seperate logical tapes using SMS management classes. Ideally, backup data should be flushed straight through the VTS cache as it is unlikely to be needed again. If you keep migrated data resident in cache then you could reduce the size of your ML1 pool.
If you want to recall a single file from an HSM tape, you first have to copy the entire virtual volume back into the VTS cache, which could mean that recalls from ML2 will take longer, depending on the size of the virtual tape. Also, HSM can run several recalls in parallel, and you may not have enough real drives to satisfy them all.
On the plus side, it takes several hours to run an HSM audit against a 9840 or 3590 ML2 tape. Virtual tape usually emulates 3480 or 3490 tapes and so audits will not take so long.
HSM duplexing is prone to error, especially when you have spanned datasets. The solution is to use VTS duplexing instead. This also removes the need to swap in alternate cartridges, as the VTS does that for you automatically.
You really want to avoid having any spanned datasets. As the HSM tapes are now virtual and do not need to be filled up, set the tape percentage to 80% to avoid spanning.
ABARS by definition is for disaster recovery, so you should not write ABARS data to a local VTS. However, you could write ABARS to a remote VTS. The IBM software team recommend that you run ABARS with the STACK option to consolidate data onto one tape, even with a VTS. The IBM VTS team recommend that you specify the NOSTACK option to avoid unnecessary multi-volume files, and let virtual tape do the stacking. The answer is that it depends on the size of the aggregates. If your aggregates are large, then it probably does not matter whether you use STACK or NOSTACK, as the VTS emulates small 3490 volumes so the aggregate will go to multi-volume files anyway. However, for small aggregates, NOSTACK means that DFHSM will write to at least two virtual tapes, both of which need to be retrieved back to cache before recovery can start. So on balance, it is probably best to specify ABARS SETSYS ABARSTAPES(STACK)
- DB2 database backups are often small files, and so JCL stacking is used to put several backups onto one tape. Virtual tape eliminates the need for JCL stacking
- Most people write their TSM backups to a disk storage pool initially, then copy the data off to tape, which is just the virtual tape principle. To then send this data to a VTS seems to be intuitively a waste of resource. TSM also has its own recycle mechanism to recover unused tape data, and again, recoveries can take longer, as the entire virtual tape has to be written back to cache before the restore can start. In general, TSM is not a good virtual tape candidate, but a lot of people use it.
- DFDSS physical dumps, and FDRABR volume dumps move large amounts of data and could fill up the virtual tape DASD cache. They may not be good candidates if your cache is limited. DFDSS logical dumps, and FDR dataset dumps are good candidates.
- Transient batch data, which is written out by one job, then read back again soon after, is a very good candidate, and should speed up a batch run considerably.
Virtual tape requires a different type of performance monitoring, than real tape. There are three areas to watch
- The 'front end' channel usage. Because you will be typically sharing up to 256 virtual drives between 4 to 8 FICON channels, these channels can become a bottleneck
- The internal throughput, basically the speed at which the VTS or VSM can transfer data internally, and the residency time of virtual volumes in the disk cache.
- The 'back end' performance, which is, how busy the real drives are, and how often the system has to wait for a free drive to service a recall.
There are tools available to help with this. An example is Perfman for Tape Libraries, which will collect virtual tape performance data and report on it.