Navigation Bar

Enterprise Storage Selection

An enterprise subsystem can be defined as one that supports all the major operating systems; z/OS, Unix variants, Linux variants, Windows and Netware. The major enterprise vendors are discussed below.

The first section discusses the enterprise products from the five big enterprise vendors; EMC, HDS, IBM, SUN and HP. The second section is a table that compares their products.

EMC

History

EMC started out producing cache memory and developed solid state disks, memory devices that emulated spinning disks, but with much faster performance. These solid state disks were usually re-badged and sold by StorageTek.

Around 1988, EMC entered the storage market in its own name, selling symmetrix disk subsystems with what at that time was a very large, 256MB cache fronting 24GB of RAID 1 storage. Their mosaic architecture was the first to map IBM CKD mainframe disk format to standard FBA open system backend disks, and as such, could claim to be the first big user of storage virtualisation. In those days, EMC developed a reputation for delivering best performance, but at a price.

EMC introduced their latest addition to the symmetrix range, the V-MAX, in April 2009. This is the successor to the DMX-4, so is it just coincidence that 'V' is the Roman number for 5, as well as standing for Virtual?

Architecture

The DMX series uses the Direct Matrix architecture, now called Enginuity. The principle behind Direct Matrix is that all IO comes into the box the front-end directors. These are connected to global memory cache modules, which are in turn connected to back-end directors that drive the IO down to the physical disks. This connectivity is all done by a directly connected, point-to-point fibre-channel matrix. There is no switching or bus arbitration involved. Each DMX-4 data path runs at 4GB/s, and there are 128 concurrent data paths. In 2008, EMC became the first to use flash storage in an enterprise subsystem, for high performance applications.

The V-MAX architecture builds on the older DMX architecture. The first release of V-MAX consists of between 1 and 8 engines. Each engine is built from commodity processors, cache, host adapters and disk adapters. This makes them relatively cheap to produce and easier to upgrade later. Internally, the engine components communicate locally, so memory access is local. However, the engines must also communicate with each other and also support the Enginuity global memory concept. To achieve this, the memory is virtualised, and each engine communicates with other engines using fiber connect and RAPIDIO technology. When a director gets a memory request it then checks the location, and if it is local it is served at memory bus speeds. If it is remote, then the request is packaged up and sent off to the remote director for processing. Presumably EMC have optimised this setup to ensure that most memory accesses are local. Certainly the EMC diagrams show each engine with 2 directors, 16 host ports and 16 disk ports, but only 4 virtual matrix ports. There are two of these ports per director, and they are connected to other engines with two MIBE (Matrix Interface Boards). The Cache memory is mirrored, and in configurations with 2 or more engines, it is mirrored between engines.

This architecture extends the direct matrix principle, but now the matrix is virtual. One of the difficulties in machine hall design is leaving room for various frames to grow as cabinets are added to increase capacity. The plan is that V-MAX frames do not need to be in the same footprint, they can be 100m apart.

One interesting feature is the storage tiering, based on T0 Flash storage, T1 FC drives and T2 SATA drives. A new FAST (Fully Automated Storage Tiering) product will automate movement between tiers. Again, the devil will be in the detail. Will FAST move data at file level, block level or LUN level? Will it be driven by business policies?

Models

The Symmetrix DMX-4 became generally available in August 2007. It is very scalable, from 240-2,400 drives. It supports 1 TB SATAII drives and 73 or 146GB flash drives. Internally, it uses 4Gb/s communications end to end, with 4 Gb/s support for FICON or Fibre Channel host connections, internal connectivity and Fibre Channel Drives. The backend architecture is FCAL.

The V-MAX starts with the single-cabinet, entry-level Symmetrix V-MAX SE that can hold 120 disks. This can be extended by adding up to 10 more frames, each holding 240 disks. Note that in this first release, all the frames must be adjacent to each other.

Software

DMX software includes EMC Symmetrix Management Console for defining and provisioning volumes and managing replication. The Time Finder products are used for in-subsystem and PIT replication, and SRDF for remote replication. SRDF can run in full PPRC compatibility mode, and can also replicate to three sites in a star configuration.
Enginuity 5784 adds new features including SRDF/EDP (Extended Distance Protection) which is similar to cascaded SRDF except that it uses a DLDEV (DiskLess Device) for the intermediate hop.
EMC was lacking in z/OS support for some years, but they have recently licensed PAV and MA software from IBM, and have provided z/OS Storage Manager to manage mainframe volumes, datasets and replication.

Openness

The DMX in general is not an Open implementation. SRDF, for example, will only work between EMC devices, and even then, not with all of them. EMC Open Replicator has the ability to take PIT copies from selected non-EMC subsystems to DMX, or to copy from DMX to selected non-EMC devices.
The V-MAX is a closed virtual system, as it cannot connect to storage subsystems within the EMC range.

Full FAST capability is planned for 2010, and full intersite connectivity is also planned, permitting a geographically dispersed storage subsystem, but no date is announced yet.

HDS

History

Hitachi Data Systems was always known as the company that manufactured disks that were exactly compatible with IBM, but worked a little faster and cost a little less. HDS broke that mould when they introduced the 'Lightning' range of subsystems in 2000, which was a merging of telephony cross-bar technology and storage subsystem technology. They extended and developed that architecture further with the USP (Universal Storage Platform), released in September 2004.

Architecture

The USP architecture is based on cross-bar switch connectivity, in-subsystem virtualisation, the ability to partition a subsystem into component LPARS and to ability to replicate data to externally attached subsystems.
The USP components are Disk Adaptor modules, Front End Director modules and Cache modules. These are connected by a set of massively parallel cross-bar switches called a Hierarchical Star Network (HSN). The 'massively parallel' bit means that each switch has a dedicated fibre link to every input and output component, so the switch can set up multiple and parallel paths between devices. This makes the architecture non-blocking. The USP also has two HSNs, one for the physical data, and one for the control data.
The USP supports subsystem based virtualisation, with the ability to attach OEM devices behind the subsystem and virtualise them. This means that the USP can replicate and transparently migrate data internally, and also to externally attached subsystems
The USP can also be partitioned into 32 LPARs or Virtual Private Storage Machines which can be used to isolate workloads. These LPARS can consist of a mixture of internal USP storage and externally attached storage. It is not possible to replicate data between LPARS, and all z/OS data must be kept in the base LPAR.

Models

The enterprise USP comes in four models, with an entry level NSC55 device. Hardware wise, the newer USPV looks the same as a USP1100. The USP models are all cabinet mounted and differ in maximum cache size, number of array frames and so number of internal disks supported and number of cross-bar switches. The older USP devices can connect up to 32 Petabytes of external storage, at least in theory, while the USPV can connect up to 247 PB externally. It is possible to upgrade between USP models without requiring an outage.

Model Frames Cache size Internal disk drives Internal raw capacity Number of cross-bar switches
USP100 1+1 64 GB 256 76 TB 2
USP600 1+2 128 GB 512 153 TB 4
USP V 1+4 512 GB 1,152 2.269 TB 8
USP VM 1+1 racks 472 TB 240 72 TB 1

Software

The USP uses HiCommand to manage both disk and virtualisation configuration.

Replication software is provided under an umbrella product called Business Continuity Manager. This is integrated into the HiCommand product which means that z/OS and Open Systems replication can all be managed and monitored from one point. The actual replication software includes ShadowImage (FlashCopy compatible) for in-subsystem replication, TrueCopy for remote synchronous and asynchronous replication and Universal Replicator for remote asynchronous replication.

Openness

The USP is an open architecture, in that it works with disks from many other vendors and virtualises the data. The list of supported vendors includes EMC, HP, IBM and SUN, as well as older HDS devices. In general, the USP will support the hardware, but replaces the OEM replication software with its own.

IBM

History

The original IBM hard drive, the RAMAC 350, was manufactured in 1956, had a 24 inch (609mm) platter, and held 5 MB. The subsystem also weighed about 1 ton. That was a bit before my time, but when I joined IT 23 years ago, the storage market was dominated by IBM, the mainframe was king, and the standard disk type was the IBM 3380 model K which contained 1.89 GB. IBM lost their market leader position to EMC sometime in the 1990s.

IBM introduced their latest subsystem family, the DSxxxx series, in late 2004 in response to competition from EMC and HDS. They updated their internal bus architecture to increase the internal transfer speed by 200% plus over the ESxxx series, and also abandoned their SSA disk architecture for a switched FC-AL standard. The DS8300 is essentially a follow-on from the ESS disk series, and re-uses much of the ESS microcode.

Architecture

The DS8000 architecture effectively consists of two processor complexes called servers that are connected to hosts using host adaptors, and disks using device adaptors.

The processor complex consists of 2-way or 4-way p-570 servers containing two types of cache, volatile and persistent memory. Every write IO is written to volatile memory in one processor, and non-volatile in the other before the write is acknowledged as complete. The subsystem effectively works internally as two separate units, but one server can run the whole subsystem if the other fails.
If the processor complexes hold 4 way power5+ servers then the DS8K subsystem can also be split logically into two completely independent LPARS, either as a 50/50 or a 75/25 split. Each processor complex is split into two server LPARS, and then a Storage Facility Image or SFI is built using one server LPAR from each processor complex. An SFI is sometimes called a storage LPAR, but note that this is not the same as a server LPAR.

Internal connectivity between servers and device adaptors uses RI0-G connectors, the same as is used internally in the p-series servers. These links can run at a 2 GB per second sustained bandwidth and permit the sharing of host adapters between servers. Host adaptors can be either ESCON, or 4Gb/s FICON / FC. Each Host adaptor provides 4 Host connections, but has only two internal protocol engines, so there may be a degree of blocking.

Device adaptors are installed in pairs, and include the RAID controllers. The device adaptors are connected to the disks using Switched FC-AL, but they are not in an FC-AL loop. The disks are allocated FC-AL addresses to allow the switching to work, but once the connection is made, communications are point-to-point Fibre Channel.

For more detail, try http://www.redbooks.ibm.com/redpieces/pdfs/sg246786.pdf

Models

The top range DS8300 is a cabinet mounted subsystem that can support a maximum of 1024 disk drives and will hold a maximum of 460 TB raw capacity using 500GB FATA disks, or 192 TB using 300GB FC disks. The base cabinet holds 128 disk drives, up to two expansion cabinets can be added, each holding 256 disk drives. The raw disks are supplied in blocks of sixteen, but are configured in groups of eight, with each group being called an array group. All the disks in an array group must have identical size and rotation speed. The DS8300 uses two four-way p-570 servers

The DS8100 spec. is similar to the DS8300, except that it will only support one expansion frame, contains a maximum of 384 drives, and has a maximum raw capacity of 192 TB. The internal processor specification is also lower, at two dual processor p-570 servers.

The DS8000 series currently has three Turbo models available: the DS8100 Turbo Model 931, and the DS8300 Turbo Models 932 and 9B2. The 931 and 932 models do not support LPARS, and have 2-way or 4-way processors respectively. The 9B2 model does support LPARS and has 4-way processors.

The DS6800 is rack mounted, and supports up to 16 disk drives in the base unit. With seven expansion units, it can hold up to 128 drives, giving a maximum raw capacity of 38.4 TB with 300GB drives.

Software

The DS software includes Flashcopy for internal subsystem point-in-time data copies, IBM Total Storage DS Manager for configuration and Metro/Global mirror for continuous inter-subsystem data replication.

The older ESS subsystems supported two kinds of z/OS Flashcopy, a basic version that just copied disks, and an advanced version that copied disks and files. DS only supports the advanced Flashcopy.
Flashcopy versions include; multi-relationship, will support up to 12 targets;
Incremental, can refresh an old Flashcopy to bring the data to a new point-in-time without needing to recopy unchanged data;
Remote Mirror Flashcopy, permits dataset flash operations to a primary mirrored disk; Inband Flashcopy commands, permits the transmission of flashcopy commands to a remote site through a Metro Mirror link;
Consistency Groups, flash a group of volumes to a consistent point-in-time. A consistency group can span multiple disk subsystems.

Remote mirroring versions include;
Metro Mirror, synchronous remote mirroring up to 300km, was PPRC; Global Copy, asynchronous remote data copy intended for data migration or backup, was PPRC-XD;
Global Mirror, asynchronous remote mirroring;
Metro/Global Mirror, three site remote replication, two sites being synchronous and the third asynchronous;
z/OS Global Mirror, z/OS host based asynchronous remote mirror, was called XRC;
Z/OS Metro/Global Mirror, three site remote replication, two sites being synchronous and quite close together, the third asynchronous and remote.

Openness

The DS subsystem series is self contained and does not interface with any other vendor's storage subsystem. For Open Systems data, IBM does support mirroring and copying to other vendor's subsystems if they are fronted with SVC virtualisation.

Futures

IBM promised a lot of enhancements to the DS series when they were first announced, and has delivered SATA drive support, space efficient Flashcopy, 4Gb FICON and virtual LUN space support so far.
The DS Subsystem LPARing is currently restricted to two LPARS both of which must be the same size. IBM propose to increase the number of LPARs, allow them to be different sizes and allow them to support different microcode levels in future

HP

History

I've always viewed HP as a major Intel player, but not a supplier with much of a presence in the mainframe market. I'm willing to change that view if someone can share their experiences. HP has been a Hitachi reseller for some time, but while they buy USP hardware from Hitachi, HP supplies its own software.

Architecture

Because the HP XP24000 is a re-badged Hitachi USP, it has the same basic architecture. Older HP devices supported an extra CHIP set to provide more connectivity, but the latest models seems to use standard HDS hardware.

Models

HP now supplies two USP type models, the XP24000 and the XP20000, which are the equivalents of the USP V and the USP VM.

Software

For mainframe solutions, HP seems to supply standard HDS software. For Open Systems solutions, HP supplies its own software. This includes

  • Storageworks Continuous Access which provides synchronous data mirroring between subsystems
  • Storageworks Business Copy which provides full volume copy within the subsystem. This looks similar to EMC Timefinder rather than IBM FlashCopy
  • Storageworks Virtualization system, an internal and external virtualisation manager, can be used for data migration and replication
  • Storageworks LUN configuration and Security manager which is used to configure the XP12000, to define paths, array groups, volumes and LUNs
  • StorageWorks Performance Advisor which monitors performance within the XP subsystem

Openness

The XP24000 has the same open architecture as the HDS USP and supports the same range of OEM devices, plus it supports HP MSA devices.

Oracle / SUN

History

Until recently, I've always considered SUN to be a midrange company that supplied storage for its server devices. While they have been Hitachi resellers for some time, they were never big in the enterprise market. However, SUN bought out StorageTek in June 2005 and StorageTek was an enterprise player. StorageTek was the first major company to put solid state cache in front of disks to make them perform better. In the late 1990s they brought out the Iceberg product. From an architectural perspective this box should have cleaned up the market with its innovative design and data management facilities. However even with IBM backing it was unable to really compete with the EMC 8xxx series and HDS 77xx series.
For the enterprise market, SUN no longer sells the Iceberg based Flexline STK V2X devices but now just acts as an HDS reseller, offering the Hitachi USP as a re-badged SUN device complete with standard HDS software. Oracle completed its takeover of SUN in early 2010 and it remains to be seen how much emphasis they will put on their storage arm.

Architecture

As it is the same device, the SUN 9990 architecture is identical to the HDS USP. SUN is not mentioned in the comparison table below, but will have the same data as HDS.

Models

SUN has opted to resell the USP range as a single product, called the SUN Storedge 9990. They resell the NSC55 as the SUN Storedge 9985

Software

The SUN Storedge 9990 runs the same HDS software as the USP

Openness

As the SUN 9990 is a re-badged HDS, it shares the same open architecture.

Netapps

NetApps is becoming one of the major Open Systems vendors. They specialise in the Intel market (Windows, Netware, Linux) and should be considered 'one to watch', rather than a current enterprise storage contender.

Their hardware range includes the Network Appliance series, of which the FAS6070 is expandable up to 500TB.

Their software range includes various SnapMirror and SnapClone products for inter subsystem and intra subsystem data copying. They have MSSQL, Oracle and Exchange agents.

Storage Subsystem Features table

The various suppliers of mainframe disks are contrasted in the tables below. The first row explains why the factor might be important, the second row just presents the facts, which were correct at time of writing, January 2009. However I'd advise you to check with your salesperson for up to date details.

Manufacturer IBM EMC HDS HP
Device DS8700 V-MAX Platform V
(SUN SE9990)
Platform VM XP24000
Architecture See the previous page for an explanation of the various types of disk architecture
PCI BUS Virtual Matrix Switch Switch
Maximum, and maximum effective capacity How much data can you cram into the box? The maximum configured capacity will be less than the rated capacity, partly due to RAID overhead, and partly due to 3390 emulation overhead. The configured figures are for Mainframe emulation, Open Systems emulation will be higher. The maximum EFFECTIVE capacity for a mainframe workload running IO intensive TP systems can be as little as 33% of the maximum capacity, if you want adequate performance.
450 TB with 450GB FC disks
1 PB with 1TB SATA disks (raw)
Using 2,400*1 TB drives, 2.4 PB raw or 1,967TB usable with RAID7+1 on z/OS
Using 2,400*146 GB drives, 350TB raw or 287TB usable with RAID7+1 on Open Systems.
2,269B raw maximum, RAID6 usable capacities
1,690 TB; Open Systems
796 TB; z/OS
472TB with 2TB SATA drives, 331TB usable with RAID6 2.26 PB raw maximum, 1,980 TB usable
Connectivity What kind of cables you can plug into the box. A good box will support both FICON and Fibre at 4Gb/s and maybe 8Gb/s Ethernet. All boxes should support ESCON and iSCSI. The number of channels available is also important as that determines the overall throughput.
218 4Gb FICON or FC,
64 ESCON
Up to 128*4Gb FICON
128*4Gb Fibre
64 iSCSI
64 Gb Ethernet
112 ESCON
112 FICON
224 FC with 1024 virtual channels per physical port
Up to 64 channels in total, a combination of 48 ESCON, 32 FICON or 64 Fibre 224 FC (4Gb) or 112 FC (8Gb)
112 FICON
112 ESCON
32 iSCSI
Internal Bandwidth How fast can data move inside the box? The numbers quoted are marketing figures, you won't really see these numbers in practice. See the Architecture section for more information.
64 Gb/s 192 Gb/s with 8 engines 106 Gb/s 13.3 Gb/s 106 Gb/s
Disk Connectivity See the previous page for details of disk connectivity.
Switched FC-AL FC-AL 4Gb 2 port FC FC-AL FC-AL FC-AL
Cache size In theory, the bigger the cache, the better the performance, as you will get a better read-hit ratio, and big writes should not flood the cache. If the cache is segmented, it is more resilient, and has more data paths through it
32-384 GB 1TB with 8 engines 512 GB
192 concurrent control cache operations
64 concurrent data cache operations
128 GB 96 concurrent operations, mirrored 512 GB
192 concurrent control cache operations
64 concurrent data cache operations
3380/90 emulation 3380 drives are older legacy technology and most sites have now converted to 3390. 3390 comes in multiple sizes, a 3390-3 will hold 2.8 GB. The newest model is the 3390-M.
All models, including EAV All models, including EAV All models, supports up to 65,536 logical devices (the older USPs just support 16,384 Open Systems devices) All models All models, supports up to 65,536 logical devices (or 16,384 Open Systems devices)
RAID levels supported See the RAID section for details
5,6,10 1,5 (3+1 or 7+1),6 (6+2 or 14+2) 1,5,6,10 1,5,10 1,5,6
Physical disk size How big are the real, spinning disks and how fast do they run. The bigger the disks, the less you pay for a terabyte, but bigger disks might be performance bottlenecks. If you have really large disks, then there should be fewer of them on an FC-AL loop. Faster speeds less rotational delay.
146, 300 GB and 450 GB FC; all 15,000 rpm
1TB SATA

146,300GB FC; 15,000 rpm
400Gb 10,000 rpm
450GB 15,000 rpm
1TB SATAII; 7,200 rpm

146,300,450,600 GB FC; 15,000 rpm
1TB, 2TB SATA 7,200 rpm
146,300,450,600 GB FC; 15,000 rpm
1TB, 2TB SATA 7,200 rpm
146,300,450,600 GB FC; 15,000 rpm
1TB, 2TB SATA 7,200 rpm
Flash Disk support Does the subsystem support Flash disks? They can used for data that requires very fast access
73GB or 146GB SS 73GB or 146GB SS 200GB or 400GB 200GB or 400GB 73GB, 200GB or 400GB
remote copy Do you mirror data between two sites? If so you need this. There are basically two flavours of mirroring, SRDF from EMC, and PPRC from IBM. SRDF is arguably the better solution technically, but it locks you into EMC disks. PPRC is used by everyone. The remote mirroring section has more details.
Global Mirror, asynchronous
Metro Mirror (PPRC), synchronous
Synchronous(SRDF/S) and asynchronous(SRDF/A) data replication between subsystems.
SRDF/DM will migrate data between subsystems.
SRDF/AR works with TimeFinder to create remote data replicas.
SRDF products are all EMC to EMC
SRDF can emulate Metro mirror and Global mirror
Hitachi true copy, PPRC compatible and synchronous;
Hitachi Universal Replicator, asynchronous copy.
Storageworks Continuous Access, synchronous
Storageworks cluster extension for clustered servers
Hitachi True Copy for mainframe, synchronous
GDPS support for automated site failover See the GDPS pages for details
Yes Yes, including Hyperswap Yes Yes Yes
Instant copy 'Instant Copy' of volumes or datasets. Can be used for instant backups, or to create test data. Some implementations require a complete new disk, and so double the storage. Some implementations work on pointers, and just need a little more storage.
Flashcopy at volume and dataset level Timefinder at volume or dataset level. BCV version requires a complete volume be supplied, newer 'snap' version just uses pointers.
EMC Compatible Flash (FlashCopy)
Shadow Image at volume level
Copy on write snapshot
Storageworks Business Copy, full volume copy, looks similar to EMC Timefinder.
The FlashCopy version provides mainframe copies.
PAV and MA support Parallel Access Volume and Multiple Allegiance. See the implementation tips section for details. Used to permit multi-tasking to logical devices
Yes Yes , including HyperPAV support Yes Yes Yes (Hitachi PAV)
LPAR Capable Can the storage subsystem be split logically, so it appears to be several separate systems, perhaps running different levels of microcode?
No, but older models do support 2 LPARS with a 50/50 or 75/25 split. No Yes (32 LPARS, Z/OS data in single LPAR only) No Yes (32 LPARS, Z/OS data in single LPAR only)
Storage Virtualisation Server Can the storage subsystem act as a virtualisation engine in conjunction with a SAN? This enables lots of disparate storage to be controlled from one central point, including mirroring between different vendor's devices.
No No Yes No Yes
Manufacturer IBM EMC HDS HP
Device DS8700 V-MAX USP V
SUN SE9990
USP VM XP12000

Price is usually very negotiable, but be sure to make sure that the vendor quotes for a complete solution with no hidden extras. Also, make sure that you get capped capacity upgrade prices, including increased software charges as software is usually charged by capacity tiers.

back to top


Copyright © Lascon Storage Ltd. 2000 to present date. By entering and using this site, you accept the conditions and limitations of use

 

 

 

Advertising banner for Lasconet