DS8K Architecture

The DS8K architecture is discussed in the Enterprise Storage Selection page, but in brief the DS8K is controlled by a pair of 2-way or 4-way processor complexes. These processors are combined then split into two processor LPARs called Server0 and Server1. The splits are designed so that if one processor complex fails then both processor LPARS are still active, but with degraded performance. Processor LPARS should not be confused with Storage LPARS or Storage Facility Images (SFI), which can only exist on 4-way processors. If a DS8K is split into two SFIs, then each SFI has a Server0 and a Server1. So if you do not split your box into two storage LPARS, then you have two virtual servers running the system, and if you do split your box into two storage LPARS, then you effectively have two different storage devices, both running two virtual servers.

Data is organised within the DS8K as Extent Pools, and both Server0 and Server1 processors must have at least one extent pool allocated. How you organise your extent pools will depend on how many different types of physical disk you are using, and whether or not you are mixing CKD and FB data on a box. CKD and FB data must be in separate extent pools, and it pretty much makes sense to separate slow 500GB FATA disks and fast 146GB fibre channel disks into different pools.

IBM recommends that you balance the storage between servers, so assuming a mix of CKD and FB data, both with 500GB FATA and faster FC disks, your extent pool layout could look like this

Extent pools are called P0,P1,P2 etc. The even pools belong to Server0 and the odd pools to Server1 as shown. You could of course decide that you want to have a CKD server and an FB server, in which case your extent pool layout could look like this.

This might look simpler and so easier to manage, but it might not perform as well as the first design, as the workloads are not necessarily balanced between servers.

Extent pools contain storage extents of either 1GB for FB or the size of a 3390-1 for CKD. To understand where the extents come from we need to switch to a physical view of the disks.

Enclosures, Arrays and Ranks

Physical disks are installed in groups of 16, called disk groups or enclosures. Each enclosure contains two fibre channel switches, and every disk has two fibre connections, one to each switch. These switches are then connected to two device adaptors over two independent RIO-G loops. Disk enclosures are installed into the front and rear of the DMX, and two enclosures are then paired up to form four Array Sites. An Array site contain 8 disks, 4 from a front enclosure and 4 from a rear enclosure, so an array site will also span two RIO-G loops. Array Sites are numbered S1, S2 etc. n.b. there is no S0! This is illustrated in the GIF below

When you format an Array site up as RAID, it becomes an Array. RAID options are either RAID5, 6 or 10. Typically, a CKD formatted RAID5 array based on 146GB disks will contain 826 GB of data, as one disk is reserved for a spare, and the equivalent of one disk is needed for RAID5 parity. For the benefit of you people reaching for your calculators, the missing space is the CKD formatting overhead. The FB formatting overhead is lower, so an FB RAID 5 array with a dynamic spare will contain 836 GB. The rules for calculating how many spares are needed are complicated, but in general the DS8K will require one spare for every Array, until it has four spares per Device Adaptor pair, but it may reserve more if different disk sizes or speeds are combined in an Array Site. The Arrays are numbered A0,A1, etc.

At present, a Rank consists of one RAID array. Each Rank is split into extents of 1GB each for FB format and 0.98GB (the size of a 3390-1) for CKD format. These are the extents that make up an Extent Pool. Ranks names are allocated by the system and are called R0, R1, R2 etc.

Ranks should ideally be allocated to Extent Pools so that the capacity is equally shared between Server0 and Server1. Ranks, Array Sites and Arrays are independent until the Rank is placed into an Extent Pool. It is then bound to the server associated with that pool.

As mentioned above, you need an absolute minimum of two Extent Pools per DS8K, and in theory you can have as many Extent Pools as you have Ranks. In practice, if you place more than one Rank into an Extent Pool, then you can stripe the data over the Ranks to improve performance. The IBM recommendation is to place between 4 and 8 Ranks in an Extent Pool. If you allocate all the space in an Extent Pool to volumes, then add an extra rank for more capacity, you will not get the benefit of striping as there is only one rank to strip across. Always try to add at least 4 ranks to a full pool, or to a new pool, so striping can be effective.

If you want to use space efficient volumes, then you must define a repository for them within an Extent Pool. Only one repository is allowed per Extent Pool and once defined it cannot be resized, except by deleting all the space efficient volumes, then deleting and reallocating the repository. Space Efficient volumes is a chargeable option.

Volumes, LCUs and Disk Groups

Ok, so now we have a few Extent Pools all containing lots of 1GB extents. How do you present those extents to a server? A logical volume is made up of a set of extents from one Extent Pool. You can define a volume to be almost any size you want, but the space is added internally is fixed increments. This means you should make the total size of the logical volume to be a multiple of 1GB for FB or 3390-1 for CKD to prevent space wastage. Up to 64K logical volumes can be defined

In a mainframe environment, all the extents in a CKD volume must be contained in one Extent Pool, but can be striped over several Ranks. A CKD volume can be any size between a 3390-1 and a 3390-A (1113 to 262,668 cylinders) provided your z/OS release supports this.

The point behind this apparently complicated setup is that it makes the storage environment more flexible, especially for future maintenance. With as ESS, if you wanted to resize volumes, you had to empty out an entire array, delete it then redefine the whole array. The concept of breaking a CKD volume space into fixed size extents makes it possible to delete one volume from within a range, then add it back in again as a different size. You can also convert a 3390-9 to a 3390-27 in place by simply adding more extents to it and running and IDCAMS REFORMAT …. REFVTOC

In an Open Systems environment, a logical subsystem looks like a SCSI controller with up to 256 associated LUNs. Up to 256 Logical Subsystems can be defined with even addresses associated with Server0 and odds with Server1.

The DS8K supports several Open Operating Systems, including Windows, Linux, AIX, Sun Solaris, HP-UX, VMWARE and Open VMS. Each of these operating systems has its own quirks, so the best thing is to look up the latest data for your own OS.

back to top