ICF Catalog Maintenance Products
Catalog Management Products
There are several products which have been developed to simplify catalog management. The following page lists the facilities offered by some of them. The first product, Integrated Catalog Forward Recovery Utility (ICFRU) from IBM, is a simple catalog recovery utility. The other products are Catalog Solution from EMC, and Catalog Recovery Plus from Mainstar Software Corporation, Catalog Information Manager from Software-Europe and T-REX from Dino Software Corporation. These produces are more sophisticated and have an ISPF interface to assist in generating JCL. Each product is summarised below.
The Integrated Catalog Forward Recovery Utility ICFRU is an IBM tool for basic catalog recovery that comes with z/OS 1.7 and above. It is not a full catalog management tool like the rest of the utilities on this page, but is useful to consider as a baseline for comparison purposes.
ICFRU does not have any diagnostics capability. For this you need to use the IDCAMS utility explained on page 012002
ICFRU does not have a native backup facility; it relies on an IDCAMS EXPORT backup as explained on the 012003 page. If your catalog recovery plan will rely on ICFRU then you must schedule frequent EXPORTS of all your catalogs.
To recover a catalog with ICFRU you must have a valid IDCAMS EXPORT of the catalog from before it was damaged, and a full set of type 61, 65, and 66 SMF records (d017004) from every LPAR that is connected to your faulty catalog.
The recovery process in outline then goes like this
- LOCK the catalog to keep users out with an IDCAMS ALTER command
- Run the ICFRRSV program against the SMF records, selecting both a start and end data/time, and the name of the catalog to be recovered. ICFRRSV then processes the SMF records and builds a list of all catalog updates
- Sort the records to get them in the correct order
- Run the ICFRRAP program against that output file, and also the good catalog export. This produces a file that can be used as an IDCAMS IMPORT
- Run an IMPORT to create a good copy of the catalog
- Test to make sure all is well, then UNLOCK the catalog
ICFRU does not have any catalog reorganisation capability. You would need to use IDCAMS.
ICFRU does not have any extra features.
Catalog Solution was originally developed by Softworks, Inc. and was called VSAM Mechanic in its early days. At that time it was the only VSAM and Catalog management tool on the market. EMC bought out Softworks in January 2000, but the Softworks mainframe products were not seen to be strategic, so Catalog Solution has lagged its rivals a bit.
Catalog Solution will check the VVDS and VTOCs against the BCS, and the BCS
against the VVDS files. It will also automatically generate fixes if
problems are found for both BCS and internal VVDS errors.
This can be done at individually catalog and VVDS level, or for every catalog and VVDS on the system using a single command. Catalog Solution can also use SMS constructs like Storage class, Data class and Management class to select catalog entries by dataset.
Catalog Solution can also List specific dataset information from and BCS EXPORT file or from SMF data.
Catalog Solution can backup every connected catalog and VVDS with two commands, which means that you do not have to manually change the backup jobs when catalogs are added and deleted.
Catalog Solution will not automatically lock a catalog before a restore. It
requires you to issue a query command to see if catalog is under VLF
control, then manual issue MODIFY CAS and IDCAMS LOCK commands.
It reads the data portion during a backup then rebuilds the index on restore, so index errors are automatically fixed.
The BCSRECOVER command will automatically restore aliases to the Master Catalog.
The REBUILD command will automatically forward recover the catalog by reading data from SMF records to synchronise a BCS with VVDS and VTOCs A VVDS can also be recovered from backup, and it too will be forward recovered with SMF records
Catalog Solution does have a reorganisation utility, but it appears that it needs the catalog to be in an unavailable locked status to do this.
Catalog Solution can resize a VVDS as part of the DUMP and rebuild process. It has a VOLSER rename function that will alter VVDS and BCS entries to match the new volume name. It can also repair a broken KSDS, and merge two KSDS files with identical attributes.
Catalog Recovery Plus (CR+)
CR+ is developed and supplied by Mainstar Software Corporation. It is also an IBM recommended product and is resold by them.
CR+ has 4 sub commands within its DIAGNOSE command;
DIAGNOSE ALIAS is used to compare two master catalogs and check the connections and alias relationships with their user catalogs
DIAGNOSE BCS-VVDS can be used both ways, to check for 'orphaned' BCS records or for 'orphaned' VVDS records.
DIAGNOSE VOL-BCS will check for uncatalogued datasets
DIAGNOSE VVDS-VTOC will check that VVDS entries are synchronised with the VTOCs
All functions will automatically generation commands to fix any problems that they find.
The backup command has two options; BACKUP BCS and BACKUP VVDS. You have the
option to select catalogs for backup individually, by name masks, or
every connected catalog with BACKUP BCS(**). A VVDS is selected by VOLSER,
not the full VVDS name. BACKUP VVDS(**) will backup the VVDS on every
A PARALLEL keyword can be used to multi-task the backup and make it run faster.
Each Catalog is opened as a dataset rather than through Catalog Services. This means the processing bypasses the index so you can be sure to read all the data portion. Each BCS backup will also copy the aliases definitions in the master catalog, though you can optionally turn that off. You also have the option to check the consistency of the catalogs at backup time by using keywords like DIAGNOSEBCS and DIAGNOSEVVDS.
The RECOVER command will only process one catalog at a time. The selected catalog
will be locked by default. As the backup just reads the data portion
of the catalog it must rebuild the index on restore, so guaranteeing
index integrity with the data. If you kept the ALIAS keyword active
on the backup, then the default action on restore is to rebuild all
aliases in the master catalog.
The RECOVER BCS FORWARD option will read SMF dump files to recover a catalog to a point in time. It is also possible to forward recover a VVDS file from SMF records. The RECOVER command also has a simulate option for testing.
A couple of useful options for DR is EMPTY-GDG, which will just recover GDG bases and the CATSCRUB command which will synchronise a catalog with the data on a specified set of volumes. This is useful if you do not recover all your data in a DR test. CATSCRUB can take numerous parameters, for example it can be set to delete all MIGRAT datasets.
CR+ can do a catalog reorg while the catalog is open. This process also lets
you change catalog attributes on the fly.
A single step MERGECAT function will move records between catalogs for alias balancing.
CR+ can be used to resize a VVDS resizing, rename a VOLSER with a SUPERCLIP command and rename a catalog with the RECOVER - NEWNAME option.
Catalog Information Manager (CIM)
Catalog Information Manager from Software Europe is an ISPF driven system.
My own preference is to generate the first job using CIM, save the JCL
then run it in batch in future.
A quote from a CIM user; 'My last day before I was taking time off to get married and go on a week-long honeymoon, I inadvertently deleted a catalog index for our TSO catalog, no less! With the help of CIM, I was able to restore the most recent backup of the catalog and then forward recover, using SMF records. The whole process took less than 30 minutes, and I was still able to get to the church on time.' Some male cynics might say that was a good reason to stick with IDCAMS.
CIM has five principle diagnostics options from its main ISPF panel
VVDS/VTOC to check the physical, logical and SMS relationships between datasets on disk and in the VVDS
Usercatalog performs various checks on the user catalogs, including consistency with tape management catalogs
Mastercatalogs checks user catalog connections, aliases and VVDS relationships in master catalogs
Sams/Tms/Hsm/Rmm checks various tape management catalogs and the DFHSM MCDS and BCDS.
IDCAMS will generate standard IDCAMS diagnostic commands
All these commands can automatically generate fixes if problems are found.
The Catalog Monitor can continuously check the status of your catalogs and report problems by e-mail or TSO Notify.
CIM uses separate commands to backup catalogs and VVDS files. CAT=** will backup all connected catalogs. VVDS can be selected by SMS Storage Group. You can optionally request that CIM does an index and data test at backup time
CIM will recover from a broken index if the backup used dump without index.
A recovery will automatically rebuild aliases in the master catalog.
CIM will lock a catalog before recovery starts and it also has commands
to deallocate catalogs from VLF if VLF is used.
Forward recovery is automatic with the RECOVER command which reads SMF dump files. CIM also has comprehensive VVDS recovery utilities
CIM has the ability to reorganise a catalog while it is open, without affecting other running processes.
CIM will analyse HFS files, navigate the HFS tree and simplify mount and dismount commands. It has an advanced REPRO MERGECAT facility to split or combine catalogs and a VOLSER rename facility.
When EMC bought out Softworks, the original Catalog Solution developers left the company and formed DINO-Software. One of DINO-Software's products is T-Rex, a catalog management utility.
T-REX can check that multiple master catalogs are synchronised, that the VVDS/BCS and BCS/VVDS relationships are correct, and that HSM CDS files match the BCS. It will also verify various tape management catalogs against the ICF system. T-REX can automatically fix problems, or generate control statements to run at a later time.
The T-REX EXPORT command can backup all BCS and VVDS files with a single command
to multiple output files.
The DUMP command can also be used for VSAM clusters. T-REX is efficient at multitasking. It does its own performance analysis to determine the optimum number of tasks to run.
T-REX can also generate a point in time backup by combining a previous backup with SMF extracts
T-REX will always recover from broken index as dump without index is the default.
By default it will lock a catalog before recovery and will automatically
rebuild the aliases in a master catalog.
The ICFRU command provides automatic forward recovery from SMS dumps. There is also a VVDS recovery feature.
The IMPORT command will restore VSAM files, as well as Catalogs and VVDS files.
T-REX can reorganise a catalog while it is open, and can remove imbed/replicate parameters while the catalog is open.
T-REX will maintain VVDS integrity if the VVDS is held in the coupling facility.
It can recovery GDG base catalog entries only, leaving GDG data files to be recovered and catalogued by a file restore utility like FDRABR or DFDSS. It has VOLSER rename and VVDS resize functions
T-REX supports all 4 types of VSAM, as well as BCS and VVDS.