Parallel Access Volumes and Multiple Allegiance

Parallel Access Volumes (PAV)

One of the problems with mechanical storage disks is that they are single threading; Only one process can access the data on a disk at a time. OS/390 managed this by allocating a Unit Control Block (UCB) to each storage device. If several applications wanted to access a device at the same time, then the IO operations were queued up by the IO supervisor. In performance terms, this is called IOSQ delay.
IOSQ is an especial problem for disks with several very active datasets, and storage administrators allocated busy databases, RACF files, HSM control files, Page and Spool datasets etc. on their own dedicated volumes. This required careful monitoring, and manual effort to change.
The principle behind IOSQ is illustrated in the GIF below. 4 applications are trying to access a JBOD disk. (JBOD is the opposite of RAID. The acronym is Just a Bunch Of Disks and it means one simple disk assembly) When an application turns green, it is successfully getting to the disk. The other applications are queued, waiting for their turn.

Modern disks subsystems do not have this physical restriction. They are usually RAID so data is spread over several physical disks. Also, all write IOs go to solid state cache and over 90% of read IOs come from cache. It is possible to schedule concurrent IOs to a cache as it has several access paths, but OS/390 was unaware of the physical implementation behind its virtual disks. The UCB architecture still said that only one IO was allowed to a disk. PAV was introduced to fix this. The concept behind PAV is that every disk has its normal 'base' UCB and also a number of 'alias' UCBs, all of which connect to the logical disk. This means that it is possible to schedule concurrent IOs to a disk, one through the base UCB, the rest through alias UCBs. If there are enough alias UCBs available, IOSQ should not happen.

There are three flavours or PAV; STATIC, DYNAMIC and HyperPAV
STATIC means you specify how many PAV aliases each BASE (the real UCB) alias can have, and then that number is fixed.

DYNAMIC means you define just a bunch of aliases, and workload manager decides how many aliases are needed by each base aliases, depending on how busy the virtual disk is, and how important the application is. However Workload Manager can take a while to work out that a disk is busy, and will not eliminate IOSQ completely.
DYNAMIC PAV needs fewer aliases than Static PAV and performs better, as more aliases are available for busy disks.

Dynamic PAV has two problems; it uses up a lot of UCB addresses and it takes workload manager a while to notice that a disk is busy and needs more aliases. HyperPAV is designed to fix these problems.
HyperPAV keeps all its aliases in a pool and just assigns one to a volume when it is needed to service an IO. It does not use WLM to decide when to allocate an alias. Each HyperPAV host can also use the same alias to service a different base, which means fewer aliases are needed.

HyperPAV then requires fewer aliases per base, I've seen a ratio of one alias to 4 bases work well, but your requirement will depend on your workload.
HyperPAV is especially useful if you are planning to use EAV volumes.

Invoking PAV

If you set MIH times in SYS1.PARMLIB(IECIOSxx), then IBM recommends that you do not set them for PAV alias devices.

The following steps are needed to invoke Static PAV on an ESS disk subsystem

• Define PAV aliases in HCD, associated with each DASD subsystem. Base definitions are added as type 3390B, and aliases of type 3390A
• Through storwatch on an ESS DASD subsystem, define a number of PAV aliases to every disk.
  Select 'S/390 Storage' to get the config up
  Hilight the LCU you want from the list presented. You may have to scroll down to find it. The screen has scrolling windows within scrolling windows, which can be a bit confusing.
  Select the 'Configure LCU' button, then change the 'Parallel access volumes' button to Enabled, and set your 'PAV Starting Address' window to the correct level as discussed in the ESS configuration section section.
  Hilight 'volumes' then select them all by holding the shift key down while scrolling down the list. Then go the ADD window, and select how many aliases you want to add to each base unit.
  The total number must match the HCD. See the ESS configuration section for some hints. Other DASD subsystems will have equivalent definition methods.

and that is about it.

The following steps are needed to invoke Dynamic PAV

• Define PAV aliases in HCD, as above
• Through storwatch on an ESS DASD subsystem, define a number of PAV aliases to every disk. You should need fewer aliases than with static PAV
• In Workload Manager, set WLMPAV=YES, and run Workload Manager in Goal mode, so Workload Manager moves the aliases around as required

To set up HyperPAV, you need to

• Define the aliases in HCD
• Authorise HyperPAV on your disk subsystem
• Add them to your disk subsystem
• Add HYPERPAV=YES in SYS1.PARMLIB(IECIOSxx)

You can enable HyperPAV dynamically, but IBM recommends that you do this at a quiet time, with no other configuration work running on a DS8K. You use the command .

 SETIOS HYPERPAV=YES 

You can check if HyperPAV is active with the command

D IOS,HYPERPAV

RESPONSE=SP00                        
 IOS098I 15.37.01 HYPERPAV DATA 109  
 HYPERPAV MODE IS SET TO YES     

Hyperpav requires z/OS 1.8. or higher, although fixes are available for earlier z/OS releases. It is supported by the latest IBM, EMC and HDS (including SUN and HP) devices, but usually needs a chargeable code upgrade. It will only work with FICON channels, ESCON is not supported.

Be sure to define the same typs of PAV on the same range of volumes for each LPAR.

Querying PAV status

From SDSF, use the devserv command /DS QPAV,uuuu,nn where uuuu is the starting unit address, and nn is the number of units you want to display. If you display a base address, you'll see something like

DS QPAV,F000,4 
IEE459I 10.25.49 DEVSERV QPAVS 459  
     HOST                             SUBSYSTEM
 CONFIGURATION                      CONFIGURATION 
 -------------                   ------------------
 UNIT                                  UNIT    UA 
 NUM. UA  TYPE        STATUS     SSID  ADDR.   TYPE
 ---- --  ----        ------     ----  ----    -----
 F000 00  BASE                   0E10   00     BASE
 F001 01  BASE                   0E10   01     BASE
 F002 02  BASE                   0E10   02     BASE
 F003 03  BASE                   0E10   03     BASE

If you display an alias address, you'll see

DS QPAV,F060,4 
IEE459I 10.26.48 DEVSERV QPAVS 714 
     HOST                             SUBSYSTEM 
 CONFIGURATION                      CONFIGURATION
 -------------                   --------------------
 UNIT                                  UNIT    UA 
 NUM. UA  TYPE        STATUS     SSID  ADDR.   TYPE 
 ---- --  ----        ------     ----  ----    --------
 F060 60  ALIAS-F017             0E10   60     ALIAS-17     
 F061 61  ALIAS-F018             0E10   61     ALIAS-18    
 F062 62  ALIAS-F00B             0E10   62     ALIAS-0B
 F063 63  ALIAS-F005             0E10   63     ALIAS-05

To find out what aliases are active to a volume, use the command DS QPAV,uuuu,VOLUME. An example of dynamic PAV in action is -

DS QPAV,DC0A,VOLUME                                          
IEE459I 10.43.46 DEVSERV QPAVS 363                           
     HOST                             SUBSYSTEM              
 CONFIGURATION                      CONFIGURATION            
 -------------                   --------------------        
 UNIT                                  UNIT    UA            
 NUM. UA  TYPE        STATUS     SSID  ADDR.   TYPE          
 ---- --  ----        ------     ----  ----    ------------  
 DC0A 0A  BASE                   E30C   0A     BASE          
 DC47 47  ALIAS-DC0A             E30C   47     ALIAS-0A      
 DC4B 4B  ALIAS-DC0A             E30C   4B     ALIAS-0A      
 DC50 50  ALIAS-DC0A             E30C   50     ALIAS-0A      

The output from the same commands with hyperpav active looks like

RESPONSE=SP00                                             
 IEE459I 16.00.34 DEVSERV QPAVS 055                       
      HOST                             SUBSYSTEM          
  CONFIGURATION                      CONFIGURATION        
 ---------------                  ---------------------   
  UNIT                                  UNIT    UA        
  NUM. UA  TYPE        STATUS     SSID  ADDR.   TYPE      
 ----- --  ----        ------     ----  ----    ----------
 05BA9 A9  BASE-H                 B134   A9     BASE      
 15BFD FD  ALIAS-H                B134   FD               
 ****      2 DEVICE(S) MET THE SELECTION CRITERIA           


D M=DEV(200F)

RESPONSE=SP00                                                     
IEE174I 15.27.53 DISPLAY M 871                                   
DEVICE 400F   STATUS=ONLINE                                      
CHP                   	16   19   1A   26   27   3B   43   54      
ENTRY LINK ADDRESS    	4C0F 4C8F 4C1F 4C9F 4D4F 4DCF 4D5F 4DDF    
DEST LINK ADDRESS     	7D0B 7D8B 7D1B 7D9B 7C4B 7CCB 7C5B 7CDB    
PATH ONLINE          	N    Y    N    Y    N    Y    N    Y       
CHP PHYSICALLY ONLINE	N    Y    N    Y    N    Y    N    Y       
PATH OPERATIONAL      	N    Y    N    Y    N    Y    N    Y       
MANAGED              	N    N    N    N    N    N    N    N       
CU NUMBER            	2000 2000 2000 2000 2000 2000 2000 2000    
MAXIMUM MANAGED CHPID(S) ALLOWED:  0                             
DESTINATION CU LOGICAL ADDRESS = 36                              
SCP CU ND         		= 002107.900.IBM.75.0000002AX701.0320          
SCP TOKEN NED     		= 002107.900.IBM.75.0000002AX701.3720          
SCP DEVICE NED    		= 002107.900.IBM.75.0000002AX701.372F          
HYPERPAV ALIASES CONFIGURED = 32                                 

An RMF Snapshot report with both Dynamic and HyperPAV active looks like this

                                    
15:51:26 I=43%   DEV                 ACTV   RESP  IOSQ
STG GRP VOLSER   NUM  PAV     LCU    RATE   TIME  TIME
                                              
TEMP3   T5E20A   E20A  3      020A   657.2   1.2  0.0
INFSAS  SXCE60   6DCC  1.0H   029F   617.1   0.7  0.0
TEMP3   S5E231   E231  2      020A   338.8   1.9  0.0
TEMP3   T5FD67   3BBA  1.0H   00E3   162.1   1.8  0.0
OPAC5   SAA55F   5BA8  1.0H   0294   81.74   0.5  0.0
RXZ22   SAC99B   2827  1.0H   02D4   80.38   0.9  0.0
TEMP3   T5FD3B   34C7  1.0H   025C   74.78   2.7  0.0
INFO    SXE21E   E21E  2      020A   68.66   6.1  2.8

There are three volumes on ESS using dynamic PAV and five on DS8K using HyperPAV as indicated by the 1.0H in the PAV column. One ESS volume has 2 aliases and a bit of IOSQ wait, but all the hyperpav volumes have one alias and no wait.

Multiple Allegiance (MA)

PAV addresses queuing issues with IOs coming from the same CPU or LPAR. If a disk was shared between several CPUs or LPARs, the owning LPAR puts a hardware reserve on the disk, to prevent others from interfering. Multiple Allegiance removes that requirement, and lets the storage controller manage cross system IOs.
You do not set up MA, or switch it on. If your disk subsystems are MA capable, it happens.

back to top

Copyright © Lascon Storage Ltd. 2000 to present date. By entering and using this site, you accept the conditions and limitations of use