IAM, an alternative to VSAM

Advantages of IAM over VSAM

IAM (Innovation Access Method), is non-VSAM, but looks like VSAM to application programs. So why buy IAM, when VSAM comes with z/OS? The basic advantages of IAM are :-

back to top

IAM installation and file structure

IAM interacts with programs that would normally use VSAM, like Cobol for instance, using the VSAM Interface (VIF). The VIF is dynamically allocated in the SVC table at IPL time, and then it intercepts SVC calls like VSAM OPEN and VSAM CLOSE. If it finds that the request is for a VSAM file it just passes control to the normal IBM routines and they process the VSAM calls as normal.
If the request is for an IAM file, it opens the IAM ACB in a way that makes it look like a VSAM file, then, when the application program issues a VSAM record request (PUT, GET etc.), this request is converted by the VIF into commands to process the IAM file. This is completely transparent to the application, as far as it is concerned it is using standard VSAM commands to a file that looks just like VSAM.

An IAM file consists of the following components:

A VSAM KSDS is split into Control Areas (CA) and Control Intervals (CI) and free space is maintained in both of these to avoid CA and CI splits. This is explained in detail in the VSAM structures page

An IAM file is allocated in blocks that are determined by the CIsize parameter in the IDCAMS define command, but these blocksizes are usually much bigger than the ones used by VSAM. While IAM does not have control interval and control area concepts, it uses the FREESPACE(CI% CA%) as follows.
CI% is used to calculate the amount of space to keep free in each block for an Integrated Overflow area when the file is initially loaded. As new records are inserted into a file they are placed into the appropriate Integrated Overflow area, and when that area fills up, they are placed into an Extended Overflow block at the end of the file
CA% is used to decide how much DASD space to release at the end of a file load. Inserts that go to the end of an IAM file because their keys are higher than existing records are allocated into Extended PE blocks.

Prime Related Overflow was introduced with IAM 8.1. and is an option for IAM files that are exposed to a very high volume of records being inserted, especially if those inserts are clustered around a certain key range. This matches overflow blocks with primary data blocks and so reduces the requirement for overflow indexing, which in turn improves performance.

back to top

IAM performance

The in-Core Index is possibly the biggest IAM performance enhancement. When an IAM file is opened, IAM places the entire INDEX of the file into Virtual Storage, so every subsequent access to that index comes from storage and does not require any IO.

IAM uses dynamic buffering to reduce the real IO requirement for the Data area. To achieve this, IAM monitors the IO access pattern to a file and then adjusts the number of data buffers as required. IAM uses various techniques to achieve this, such as random or sequential processing detection and LRU algorithms.
IAM also has an optional 'TURBO' mode than can acquire buffers much more aggressively if it detects heavy IO activity.
IAM can also schedule multiple-concurrent IOs to multi-volume files. This technique is used to speed up processing by FDRREORG when reorganising multi-volume IAM files.

Another IAM feature is Dynamic Tabling. Here, IAM creates a 'table' in virtual storage that contains any records that are read from a file. If the record is requested again, IAM retrieves it from the table instead of having to go out to disk with a real IO. This will obviously not be suitable for every file but works well for files that have a high read, low update access pattern, especially if the reads are random and the same records are continually read.

back to top

IAM Journalling

IAM provides RLS functionality, and it also has a journalling facility to log batch updates that are made to IAM files. This can be used to back-out failed batch jobs, similar to VSAM TVS. The back-out can be automated, so if a job abends, its updates are automatically removed from the IAM file, either back to the beginning of the job, or back to the last sync point. The journalling does not affect CICS as CICS has its own journalling system.

IAM can write both BEFORE and/or AFTER journal records when an IAM file is updated. Typically, these are used for:

If a batch job fails while updating an IAM file, the data will probably be in an indeterminate state, or corrupt. The BEFORE images can be used to backout the updates in reverse order, so getting the file back to the state it was in before the failing job started.

A 'forward recovery' is used if an older copy of an IAM file has to be restored, perhaps after encountering a media failure, or data corruption, or maybe even in the event of Disaster Recovery. Once the older copy of the file is back on disk, the 'After' images can then be re-applied to the file, working forwards in time, thus recreating the updates that had taken place.

These procedures, which have been available for some time in IAM, have been enhanced to accommodate the changes introduced by the IAM/RLS and IAM/PLEX.
This can be automated with IAM's 'Dynamic Job Backout' (DJB) facility, which provides a controlled, automated and fully dynamic 'backout recovery' of IAM/RLS and IAM/PLEX files. If an updating job step abends, all uncommitted updates (for that job) are automatically removed from all IAM/RLS or IAM/PLEX files being updated by that job. This negates the need to run separate, manual, batch-driven recoveries for each affected file.

back to top

IAM and RLS- Record Level Sharing in a single LPAR

The principle behind Record Level Sharing (RLS) is explained in the VSAM RLS section. IAM implements RLS at two levels: The original IAM/RLS implemented record sharing within a single z/OS image or LPAR as they are usually called. IAM/PLEX was introduced in IAM Version 9.0 as an extra cost-option. This permits applications running in either the same LPAR, or different LPARs in a SYSPLEX to concurrently access an IAM file and is discussed in the next section.

VSAM RLS and TVS require that z/OS be part of a SYSPLEX, and store the sharing and log records in the coupling facility. IAM/RLS does not do this; it uses an IAM RLS address space and journalling datasets. IAM implements RLS by passing all I/O requests for the files it is sharing to an IAM RLS address space. It then uses a lock manager to handle locking requests.
IAM decides which files are eligible for IAM/RLS processing based on a combination of the file share options, an RLSID parameter that is set when the IAM file is defined, and a special IAM/RLS data set name table.

IAM/RLS requires that some exits be installed in CICS but little in the way of JCL changes are required. Concurrent update processing is managed by using locks at record level, utilizing the IAM/RLS record locking facility. For recoverable files, these record locks are held until the job or transaction terminates, or the job or transaction issues a SYNCPOINT. This means that batch jobs that do lots of updates should have Batch Syncpoint and appropriate restart capabilities installed.

An ISPF interface is provided to monitor activity within the IAM/RLS address space

back to top

IAM/PLEX - Record Level sharing in a SYSPLEX

IAM/PLEX is an extra cost option on IAM 9.0, and uses the Cross Coupling Facility (XCF) service to manage record level data sharing in IAM files between applications in a SYSPLEX. The applications can be a mixture of CICS transactions, batch programs, or TSO users.

IAM/PLEX is basically just an extension of IAM/RLS and so it is much easier to implement than native VSAM file sharing. CICS requires 2 IAM exits to be installed, but no changes are required to application programs. TSM and batch programs that have a heavy write activity need syncpoints adding, but otherwise, no changes are required.

An IAM/PLEX 'instance' runs as an address space on all the z/OS LPARS that need to share IAM files in a SYSPLEX. These instances combined to make IAM/PLEX RLS group. Each IAM dataset is owned by one of the IAM/PLEX instances and all I/O to that IAM dataset goes through that owning instance. Record locking is managed by the owning instance and this ensures data consistency, even though the file might be updated by applications running on several LPARs. The I/O requests from the other LPARs are routed to the owning instance through the SYSPLEX XCF.

IAM datasets use Journalling for forward or backward recovery, and Journalling for shared IAM file use the shared z/OS sysplex logger.

IAM is also compatible with GDPS Active/Active, which allows the cross sharing of data between remote sites. This can assist with workload balancing, continuous availability and disaster recovery.

back to top

Alternate Index support

Alternate Index support for IAM can be purchased as an extra option and can be used on KSDS, ESDS and RRDS file types. If you need alternate indexes then you define them and their paths using the standard IDCAMS DEFINE ALTERNATEINDEX, then you build the index with the BLDINDEX command.

IAM has a Dynamic Reorg function, where the updates required to take a reorg are identified then logged. The IAM file is temporarily deallocated from CICS, the updates applied, then the file opened again. The downtime involved is measured in seconds.

back to top

Managing VSAM files

Lascon updTES

I retired 2 years ago, and so I'm out of touch with the latest in the data storage world. The Lascon site has not been updated since July 2021, and probably will not get updated very much again. The site hosting is paid up until early 2023 when it will almost certainly disappear.
Lascon Storage was conceived in 2000, and technology has changed massively over those 22 years. It's been fun, but I guess it's time to call it a day. Thanks to all my readers in that time. I hope you managed to find something useful in there.
All the best