- VSAM structures
- VSAM commands
- Performance tuning
- JCL Buffers
- LSR Buffers
- System Buffers
- VSAM parameters
- IAM, a VSAM alternative
- VSAM Recovery
- VSAM RLS, DFSMStvs
An old process, but still and important one as a properly tuned VSAM file will save you both CPU and disk costs, and those savings can be considerable. If you want to do the tuning yourself then you could look at the following options:
Otherwise your could look at purchasing a VSAM tuning product like TurboTune from Critical Path Software Inc. (CPSI)
CPSI makes the bold statement about TurboTune. "If your returns don't outweigh your investment, you don't pay for our services" and claim that they have never failed to be paid for their work in 20 years. A typical site can expect to see savings and a return on their investment in between 6 and 12 months. Those savings would come from reduced CPU usage and DASD usage.
The CPSI methodology includes providing access to the TurboTune database, TurboTune performance modules and experienced consultants.
The Database contains hundreds of millions of statistics that have been gathered from tuning exercises in hundreds of client sites over the past 20 years. It combines CPSI knowledge with data gained from software vendors, performance experts, IBM and consultants from around the world. The database is constantly updated with new products and tuning parameters as they are found in clients environments.
This data is sent to an expert system, a suite of mathematical relationships which analyzes each data set in order to identify 'problem' data sets. The system performs an average of 1,500 calculations per file to generate the appropriate procedures to fix the problem.
One of the problems companies face with this type of analysis is finding the time and resources to run it. All TurboTune needs is access to VSAM/CICS statistics, LISTCATs, File usage statistics and JES Logs to begin an in-depth analysis and CPSI claims that their expert consultants do the bulk of the work. This data is fed into a spreadsheet and compared with an existing comparable 'best in class' client.
Finally, CPSI will produce a series of reports with detailed instructions on which files require improvement and what must be done to improve them.
Critical Path Software also have two TURBO products, DB2 Contention Analyser and CICS Contention Analyser. These products use the expert performance tuning database combined with statistics from your DB2, CICS and User Catalogs to identify performance problems in DB2/CICS regions. Specifically, these products will reduce contention and CPU usage, while improving throughput and response times.
Critical Path Software has a new addition to its TurboTune range, called Turbo-Pro. TURBO-PRO utilizes standard INM utilities and does not change or look at any confidential client data, but simply monitors, gathers and reports detailed information concerning Structures, Performance, Usage, Activity, Efficiency and current file definitions. The data is sent to CPSI and is processed through Turbo Tune, which analyzes the data, compares every file against an expert data base and returns fully updated and improved file definitions that will reduce the resource consumption on that mainframe.
The following VSAM Tuning Tips were presented by Ralph E. Bertrum of Critical Path Software Inc. (www.cpsigroup.com). As always, make sure you test these tips in your environment before you apply them to production data.
IMBED takes 7% of the file space to write multiple copies of the data sequence set. It performs I/O, EXCPS, and CPU to do this. It places multiple copies of the sequence set in the cache controllers, thereby, blocking the space, which could be used for data. It will no longer be supported by IBM and therefore should be removed from all VSAM DATA SETS.
Comment: No supported release of z/OD accepts IMBED now
REPLICATE writes multiple copies of the index each time the index is changed. It performs I/O, EXCPS, and CPU to do this. Like IMBED it places multiple copies of the index in the cache controllers, thereby, blocking the space, which could be used for data. Like IMBED will no longer be supported by IBM and should be removed from all VSAM DATA SETS.
Comment: No supported release of z/OD accepts REPLICATE now
ERASE writes Binary zeros to the entire allocated file space each time the file is deleted using Idcams. It causes a tremendous amount of I/O, EXCPS, CPU, and run time.
WRITECHECK writes Binary zeros to the entire allocated file space and then reads it back and compares it in memory to Binary zeros to make sure it wrote them. At file deletion time it writes Binary zeros. Very much like ERASE only worse. It reads back the zeros and compares them in memory to see if it really wrote them.
When the SPEED Parameter is not in the Cluster Define, the default is Recovery. Recovery creates a CheckPoint restart file. This Option is only valid when running Reorgs of over 12 hours long. When the Speed Parameter is added, the Recovery File is not created and all of its CPU, I/O, and EXCPS are saved. Idcams Reorgs will run almost twice as fast when Speed is in effect. Speed and Recovery are only used with Idcams.
Buffering for CICS and Batch is better handled outside of the Cluster Define. Take the default and let CICS LSR buffer pools handle online buffers and batch buffer software, i.e.: vio plus, Batch BLSR or AMP parameters handle the batch buffering.
When the CISIZE is not coded, AMS will calculate a CISIZE for you. The calculated CISIZE will be the smallest size, which will allow the file to run; NOT the best for file Performance.
Changes to parameters within the define impact the size requirement of the KSDS INDEX. Changes from Tracks to Cylinders, removing IMBED, REPLICATE, and changes to FREESPACE all affect the INDEX Control Interval SIZE. These parameters should always be altered before coding the INDEX CISIZE.
The INDEX Control Interval Size (CISIZE) should always be coded under the Index component of the Define. When it is not coded, AMS will calculate a CISIZE for you. The calculated CISIZE will be the smallest size, which will allow the file to run; NOT the best for file Performance. Changes to other parameters within the define impact the size requirement of the KSDS INDEX. Changes from Tracks to Cylinders, removing IMBED, REPLICATE, and changes to FREESPACE all affect the INDEX Control Interval SIZE. These other parameters should always be made before making changes to the INDEX CISIZE.
When the KSDS Index is created it is compressed to fit into 1 Index record. As the data portion of the file grows, so does the index. When the index can no longer fit into 1 record a second level of compression is created and a second index record is created. By increasing the size of the index record, the index can remain at 1 level. Level 4 becomes Level 3; Level 3 becomes Level 2, etc. Fewer Index Levels equal less I/O, EXCPS, and CPU. When it becomes necessary to increase the Index CISIZE, the buffers in both CICS LSR and Batch BLSR will also have to be adjusted. We recommend that INDEX CISIZE never be greater than 8192 in size.
When Files grow, they fill their primary space and create a secondary space called an EXTENT. This happens to both the DATA and the INDEX portion of a Vsam File. When the additional Extents are read, it takes additional I/O, CPU, and EXCPS to find and move to the next Extent. This is because the program reading the file has to go to the CATALOG to find the address of the next EXTENT, even if it is contiguous. This becomes much worse when the program is reading the file RANDOMLY.
To expand on Ralph's recommendation, don't use a small secondary allocation, as this will mean that the file will frequently need to extend, and the act of extending a file is a performance overhead. For example, and allocation of CYL(200,1) is a bad idea.
Multiple Extents also Fragment DASD and should always be kept to a minimum. Space should always be assigned to both the Data and the Index component of a file. Multiple DASD Volumes should always be coded, using Volume names or a string of asterisks (************). This assigns additional candidate packs so file growth will not cause space ABENDS. At Wachovia this is achieved thru the use of SMS Dataclass (VOLUMEn) parameter.
Files greater than 7 tracks in size should always be defined in Cylinders. Files less than 7 Tracks in size, which receive INSERTS, should also always be defined in Cylinders. Files defined in Tracks have fewer Tracks per CA and will have more CI and CA SPLITS because of this. Files defined in Cylinders which have small Control Interval Sizes will store more data per Track when defined in Cylinders than they will if defined in Tracks. Because of the above, File growth should be continuously monitored and adjusted.
Whenever possible always change REUSE to NOREUSE. A File Defined as REUSE does not have to be deleted and defined when reorganized. It's High Used Record Block Address and Record Counter are set to zero and data is reloaded. This sounds simple enough, but all of the CI SPLITS, CA SPLITS and EXTENTS remain in place. These files become worse and worse over time and the reorg does not accomplish one of its most important functions.
REUSE is one of the most important NO, NO'S in a Data Center. In some cases, application programs require the use of REUSE. When the use of REUSE cannot be avoided, always run a delete/define preceding the data load.
When a file is open for processing and an ABEND occurs, the record counter goes negative and this results in a record count of 4 billion plus records when looking at a LISTCAT of the file. These kinds of files can only be partially tuned. To fix the problem, the file should be backed up, deleted and defined and then restored.
When the Listcat was taken, a file had a CLUSTER name, but there was a missing Data Portion or Index Portion. These files should be carefully checked to determine why this happened and the CLUSTER name should be deleted from the CATALOG or the file should be recreated in whole.
Completely empty files should be checked for why they are EMPTY. They may have been created years ago and never used or deleted. They may be holding space for year-end processing and are causing costs all year, which are only needed at year-end. Define these files with a small Primary and larger Secondary space allocation.
When a file has an INDEX CISIZE greater than 8192, it is almost always a typo, with the DATA and INDEX CISIZES reversed. Sometimes it is simply an invalid INDEX CISIZE and the DATA CISIZE is correct. All INDEX CISIZES over 8192 should be checked and corrected, especially if they are CICS online files.
ESDS files, which have VARIABLE LENGTH RECORDS, should always be defined as SPANNED files. When they are not, they waste DASD space and require additional EXCP's and I/O and CPU to process.
This may be the single most IMPORTANT setting in any Vsam file. It is O.K. to have some CI and CA splits in Vsam files. Many files are created by copying an existing file definition and all of its parameters. This results in incorrect FREESPACE settings. The wrong FREESPACE setting can waste DASD, cause CI and CA splits, increase EXCP's and I/O and waste application overhead CPU. FreeSpace is coded in an attempt to reduce or prevent CI and CA splits.
Running reorgs every night does not fix the cause of the splitting. It only covers up the ability to find and fix the problem. Splitting can be almost eliminated by understanding your DATA, proper calculation of FREESPACE, usage of the ALTER command and the use of the KEY-RANGE parameter.
The use of EXTENDED ADDRESSING should be restricted to files of 3.5 GIGA BYTES and ABOVE in size. When EXTENDED ADDRESSING is used, it changes the way the Physical Records are stored within the Control Interval. With certain Control Intervals this can double or triple the physical I/O when processing these files.
KSDS files are INDEXED files. When records are inserted into these kind of files a CI and/or CA SPLIT can occur. If the file contains the SPANNED parameter, a great many more CI and CA splits will occur.
ON 3380 DASD, 22528 DATA CI-SIZE was the very best DATA CI-SIZE for batch processing. On 3390 DASD, 22528 DATA CI-SIZE is the very worst DATA CI-SIZE to use and should always be avoided. If 22528 is used on 3390 DASD it will cause 4 times the number of PHYSICAL I/O than it does on a 3380 DASD device.
Also, the opposite will occur when using a 18432 DATA CI-SIZE on a 3380 DASD device.
One of the most wasteful practices in any Data Center is that of Reorging files, which do not have to be REORGED. RRDS Vsam files should be copied to BACKUP, but never have to be deleted, defined, and restored. Running REORGS on RRDS files changes nothing. The file will be exactly the way it was before the REORG. On-Line viewing files also do not have to be REORGED. The reorg will not change them in any way what so ever.