What is an enterprise subsystem? My definition was one that supports all the major operating systems; z/OS, Unix variants, Linux variants,
Windows and Netware. However it seems only 4 vendors now support z/OS mainframes and I've had a few discussions lately with people
who say that using z/OS as a criteria is too restrictive. The 4 vendors that will support z/OS are EMC, IBM, HDS and HP. Oracle,
a disk vendor by way of their SUN purchase, does not resell HDS disks now and so does not support z/OS anymore. NetApp is now a
major player in the disk market so it seems reasonable to include them here.
The first section discusses the enterprise products from the six big enterprise vendors; EMC, HDS, IBM, HP, Oracle and NetApp. The second section is a table that compares some of their products.
EMC
History
EMC started out producing cache memory and developed solid state disks, memory devices that emulated spinning disks, but with much faster performance. These solid state disks were usually re-badged and sold by StorageTek.
Around 1988, EMC entered the storage market in its own name, selling symmetrix disk subsystems with what at that time was a very large, 256MB cache fronting 24GB of RAID 1 storage. Their mosaic architecture was the first to map IBM CKD mainframe disk format to standard FBA open system backend disks, and as such, could claim to be the first big user of storage virtualisation. In those days, EMC developed a reputation for delivering best performance, but at a price.
EMC introduced their latest addition to the symmetrix range, the V-MAX, in April 2009. This is the successor to the DMX-4, so is it just coincidence that 'V' is the Roman number for 5, as well as standing for Virtual Matrix?
Architecture
The DMX series uses the Direct Matrix architecture, now called Enginuity. The principle behind Direct Matrix is that all IO comes into the box the front-end directors. These are connected to global memory cache modules, which are in turn connected to back-end directors that drive the IO down to the physical disks. This connectivity is all done by a directly connected, point-to-point fibre-channel matrix. There is no switching or bus arbitration involved. Each DMX-4 data path runs at 4GB/s, and there are 128 concurrent data paths. In 2008, EMC became the first to use flash storage in an enterprise subsystem, for high performance applications.
The V-MAX architecture builds on the older DMX architecture. The first release of V-MAX consists of between 1 and 8 engines. Each engine is built from commodity processors, cache, host adapters and disk adapters. This makes them relatively cheap to produce and easier to upgrade later. Internally, the engine components communicate locally, so memory access is local. However, the engines must also communicate with each other and also support the Enginuity global memory concept. To achieve this, the memory is virtualised, and each engine communicates with other engines using fiber connect and RAPIDIO technology. When a director gets a memory request it then checks the location, and if it is local it is served at memory bus speeds. If it is remote, then the request is packaged up and sent off to the remote director for processing. Presumably EMC have optimised this setup to ensure that most memory accesses are local. Certainly the EMC diagrams show each engine with 2 directors, 16 host ports and 16 disk ports, but only 4 virtual matrix ports. There are two of these ports per director, and they are connected to other engines with two MIBE (Matrix Interface Boards). The Cache memory is mirrored, and in configurations with 2 or more engines, it is mirrored between engines.
This architecture extends the direct matrix principle, but now the matrix is virtual. One of the difficulties in machine hall design is leaving room for various frames to grow as cabinets are added to increase capacity. The plan is that V-MAX frames do not need to be in the same footprint, they can be 100m apart.
One interesting feature is the storage tiering, based on T0 Flash storage, T1 FC drives and T2 SATA drives.
EMC FAST, or "fully automated storage tiering" will check for data usage patterns on files and move them as required between Fibre Channel, SATA and flash drives to optimise cost effectiveness and performance requirements. Supported subsystems include the V-Max, the Clariion CX4 and the NS unified system.
FAST can also be configured manually to move application data to higher performing disk on selected days of the month or year. This could be useful for a monthly payroll application , for example.
EMC introduced FAST2 in August 2010, which introduced true LUN tiering and can data at block level.
Models
The Symmetrix DMX-4 became generally available in August 2007. It is very scalable, from 240-2,400 drives. It supports 1 TB SATAII drives and 73 or 146GB flash drives. Internally, it uses 4Gb/s communications end to end, with 4 Gb/s support for FICON or Fibre Channel host connections, internal connectivity and Fibre Channel Drives. The backend architecture is FCAL.
The V-MAX starts with the single-cabinet, entry-level Symmetrix V-MAX SE that can hold 120 disks. This can be extended by adding up to 10 more frames, each holding 240 disks. Note that in this first release, all the frames must be adjacent to each other.
Software
DMX software includes EMC Symmetrix Management Console for defining and provisioning volumes and managing replication. The Time Finder products are used for in-subsystem and PIT replication, and SRDF for remote replication. SRDF can run in full PPRC compatibility mode, and can also replicate to three sites in a star configuration.
Enginuity 5784 adds new features including SRDF/EDP (Extended Distance Protection) which is similar to cascaded SRDF except that it uses a DLDEV (DiskLess Device) for the intermediate hop.
EMC was lacking in z/OS support for some years, but they have recently licensed PAV and MA software from IBM, and have provided z/OS Storage Manager to manage mainframe volumes, datasets and replication.
Openness
The DMX in general is not an Open implementation. SRDF, for example, will only work between EMC devices, and even then, not with all of them. EMC Open Replicator has the ability to take PIT copies from selected non-EMC subsystems to DMX, or to copy from DMX to selected non-EMC devices.
The V-MAX is a closed virtual system, as it cannot connect to storage subsystems within the EMC range.
Full intersite connectivity is planned, permitting a geographically dispersed storage subsystem, but no date is announced yet.
HDS
History
Hitachi Data Systems was always known as the company that manufactured disks that were exactly compatible with IBM, but worked a little faster and cost a little less. HDS broke that mould when they introduced the 'Lightning' range of subsystems in 2000, which was a merging of telephony cross-bar technology and storage subsystem technology. They extended and developed that architecture further with the USP (Universal Storage Platform), released in September 2004. In September 2010 HDS released the Virtual Storage Platform (VSP), a purpose built subsystem that provides automated tiering between flash and spinning disk drives.
Architecture
Unlike competing storage subsystems, the VSP is not built from 'commodity' components, but uses parts designed and manufactured within HDS.
HDS claims that this allows them to make a subsystem that outperforms its rivals. There is only one model of VSP, if more performance
or capacity is required, you can 'scale up', 'scale out' or 'scale deep'.
The VSP is composed of 'racks', 'chassis' and 'boards'. The base model is a single rack and can be 'scaled up' by adding a second
control rack and up to four disk racks. The base rack contains one control chassis and one drive chassis. The control chassis contains a number of functional boards, and more boards can be added to the first chassis to improve performance, and another disk chassis added to increase capacity. This is called 'scaling out'. Like the USP, the VSP supports adding external disks behind a virtualisation unit, and this is called 'scaling deep'.
There are five different kinds of functional boards.
The Front End Director boards or FEDs provide the interface to host servers and also to any external storage that may be attached to the VSP. FEDs can be either 16 port 8Gb/s FICON or 16 port Fiber Channel.
The Back end Director boards or BEDs interface to the disk or SSD devices. Each chassis can hold two or four BED boards and BED has eight 6Gb/s SAS links, a significant departure from the USP which used F-CAL links to the disks. Two boards are normally installed and the extra two boards are added for extra performance, as that gives 32 * 6 Gb/s SAS links in a chassis. The BEDs will connect to 128 disk Small Form Factor (SSF) disk containers or DKU and 80 disk Large Form Factor (LFF) DKUs. The SFF DKU can hold 200GB SSD disks, or 146GB, 300GB or 600GB SAS disks. The LFF DKU can hold 500GB SAS SSD or 2TB SATA disks. The maximum raw capacity with 2TB SATA is 2.5 PB.
The BEDS generate RAID parity. The RAID options are very flexible, but HDS recommends Raid5 for SSD devices and RAID10 for disks. Other RAID configurations are possible, including RAID6 P+Q. You can also buy a BED with no disks, which you can use as a virtualisation engine for external disks. The maximum raw internal capacity with 3.5" SATA drives is 2.5 PB.
A Virtual Storage Director (VSD) is the central processor and data movement engine. Either two or four VSDs are installed in each chassis, four are used for added performance. The processors are now Intel, another change from USP technology. Each VSD board contains a quad-core Xeon CPU and 4GB of RAM. The VSDs are paired for failover purposes and they hold their meta-data and control data in shared system memory to make this possible. If one VSD fails then failover to the other one is automatic with no loss of service. When the failed VSD is repaired, failback is also automatic. The maximum number of processors is 32; 2 chassis with 4 VSDs, each with 4 processors. Control memory is not on a separate board anymore, but is held on the VSD.
The Cache boards or DCAs (Data Cache Adapter) hold the system memory. This contains transient user IO activity and also configuration details like RAID setup, dynamic tiering status and remote copy operations. Up to 6 DCAs can be installed per chassis, and each DCA board can hold between 8GB and 32GB, giving a maximum subsystem cache size of 1TB. Each DCA board also has either one or two 32GB SSDs to allow the board to backup configuration details and any outstanding activity if the power drops. This means the VSP does not need the heavy, expensive batteries that were required to protect from power failure on the USPs. Write blocks are mirrored, but not read blocks, which means the cache utilisation is improved.
The Grid Switch boards or GSWs are PCI express based, connected by a crossbar switch. They form a High-Star-E network with two or four GSWs in each chassis. Every GSW board has 24 1GB/s , bi-directional ports connected as follows
8 ports connect to FED and BED boards and transfer both data and meta-data
4 ports connect to VSD boards for job requests and system data transfer (like memory access requests)
8 ports connect to DCA boards for user data transfer and control memory updates
4 ports are used if an extra chassis is installed, to cross-connect to the matching GSW in the second chassis.
The switched PCI-e architecture means that internal communication in non-blocking and every input port can connect to any piece of memory and every BED port can connect to any disk. This means that data does not need to be placed behind specific ports to ensure performance.
The idea behind storage tiering is an old one - you keep your busiest data on fast, but expensive storage, then as it ages and becomes less busy you move it down the hierarchy to cheaper, slower storage. To achieve this, you had to solve two problems, first you had to run a report that identified data access profiles and use that report to work out what data was in the wrong place in this timeframe. Second, you had to move the incorrectly positioned data to the correct place in the storage hierarchy, a process that often required application downtime.
This data movement might involve whole volumes, or whole files. However in many cases files are active for part of the day and waste expensive disk space for the rest of the day. Some very large files can be have parts that are very active, and parts that are rarely accessed and moving the whole file to expensive storage is wasteful.
HDS has addressed those problems with Hitachi Dynamic Tiering (HDT). Storage inside the VSP can be either fast but expensive SSD, or slower and cheaper SAS/SATA drives. When you allocate a virtual volume on a VSP, it stripes the data over all the physical volumes in 42MB chunks or pages and that striping can go over both SSD and spinning disk. The page size is much bigger than that used by other Storage manufacturers, and HDS has used that bigger size to allow it to position parts files on different types of disk. The process is called sub-lun tiering.
Page access is checked on a regular basis, and if a page becomes 'hot' it is automatically moved up to SSD disk, while pages that have cooled down again are moved back to SAS disks. This means that active parts of files are held on high performance SSD and inactive parts on SAS disk, so optimising SSD usage. HDS claims that this effectively means that all files are on fast disk.
HDT is not just a VSP feature, it is also used on HDS NAS and Content Management storage systems.
Some other VSP features are:
Thin provisioning. Disk space is just allocated as needed, up to size of the virtual volume. When data is deleted from the virtual
volume, a Zero Page Reclaim utility returns unused storage pages returned back to spare pool
Automatic Dynamic Rebalancing. When new physical volumes are added to the subsystem virtual volume pages are re-striped to ensure they are still evenly spread over all the physical volumes.
Universal Virtualisation Layer. If you put some external storage behind the VSP then it is carved up and allocated to look the same as the internal storage. This means that mirroring, snapshot and replication software all work consistently for both internal and external storage
Virtual Ports. Up to 1024 virtual FC ports can share the same physical. Each attached server will only see its own virtual ports, which
means they don't get to access each others data. This feature allows the VSP to efficiently use the high bandwidth that is available
on an individual port.
All data stored on the VSP is hardware encrypted for security.
The USP
The older USP architecture was based on cross-bar switch connectivity, in-subsystem virtualisation, the ability to partition a subsystem into component LPARS and to ability to replicate data to externally attached subsystems.
Software
Hitachi High Availability Manager provides non-disruptive failover between VSP and USP systems and means instant data access at remote site if primary site goes down. This is aimed at non-mainframe SAN based applications.
Mainframe availability uses Truecopy synchronous remote mirroring and Universal replicator with full support for GDPS.
The Storage Command suite includes.
Hitachi device manager for disk and storage configuration
Hitachi replication manager
Hitachi Storage Capacity Reporter for usage trending
Hitachi tuning manager
Openness
The VSP is an open architecture, in that it works with disks from many other vendors and virtualises the data. The list of supported vendors includes EMC, HP, IBM and SUN, as well as older HDS devices. In general, the USP will support the hardware, but replaces the OEM replication software with its own.
IBM
History
The original IBM hard drive, the RAMAC 350, was manufactured in 1956, had a 24 inch (609mm) platter, and held 5 MB. The subsystem also weighed about 1 ton. That was a bit before my time, but when I joined IT 23 years ago, the storage market was dominated by IBM, the mainframe was king, and the standard disk type was the IBM 3380 model K which contained 1.89 GB. IBM lost their market leader position to EMC sometime in the 1990s.
IBM introduced their latest subsystem family, the DSxxxx series, in late 2004 in response to competition from EMC and HDS. They updated their internal bus architecture to increase the internal transfer speed by 200% plus over the ESxxx series, and also abandoned their SSA disk architecture for a switched FC-AL standard. The DS8300 is essentially a follow-on from the ESS disk series, and re-uses much of the ESS microcode.
Architecture
The DS8000 architecture effectively consists of two processor complexes called servers that are connected to hosts using host adaptors, and disks using device adaptors.
The processor complex consists of 2-way or 4-way p-570 servers containing two types of cache, volatile and persistent memory. Every write IO is written to volatile memory in one processor, and non-volatile in the other before the write is acknowledged as complete. The subsystem effectively works internally as two separate units, but one server can run the whole subsystem if the other fails.
If the processor complexes hold 4 way power5+ servers then the DS8K subsystem can also be split logically into two completely independent LPARS, either as a 50/50 or a 75/25 split. Each processor complex is split into two server LPARS, and then a Storage Facility Image or SFI is built using one server LPAR from each processor complex. An SFI is sometimes called a storage LPAR, but note that this is not the same as a server LPAR.
Internal connectivity between servers and device adaptors uses RI0-G connectors, the same as is used internally in the p-series servers. These links can run at a 2 GB per second sustained bandwidth and permit the sharing of host adapters between servers. Host adaptors can be either ESCON, or 4Gb/s FICON / FC. Each Host adaptor provides 4 Host connections, but has only two internal protocol engines, so there may be a degree of blocking.
Device adaptors are installed in pairs, and include the RAID controllers. The device adaptors are connected to the disks using Switched FC-AL, but they are not in an FC-AL loop. The disks are allocated FC-AL addresses to allow the switching to work, but once the connection is made, communications are point-to-point Fibre Channel.
For more detail, try http://www.redbooks.ibm.com/redpieces/pdfs/sg246786.pdf
Models
The top range DS8300 is a cabinet mounted subsystem that can support a maximum of 1024 disk drives and will hold a maximum of 460 TB raw capacity using 500GB FATA disks, or 192 TB using 300GB FC disks. The base cabinet holds 128 disk drives, up to two expansion cabinets can be added, each holding 256 disk drives. The raw disks are supplied in blocks of sixteen, but are configured in groups of eight, with each group being called an array group. All the disks in an array group must have identical size and rotation speed. The DS8300 uses two four-way p-570 servers
The DS8100 spec. is similar to the DS8300, except that it will only support one expansion frame, contains a maximum of 384 drives, and has a maximum raw capacity of 192 TB. The internal processor specification is also lower, at two dual processor p-570 servers.
The DS8000 series currently has three Turbo models available: the DS8100 Turbo Model 931, and the DS8300 Turbo Models 932 and 9B2. The 931 and 932 models do not support LPARS, and have 2-way or 4-way processors respectively. The 9B2 model does support LPARS and has 4-way processors.
The DS6800 is rack mounted, and supports up to 16 disk drives in the base unit. With seven expansion units, it can hold up to 128 drives, giving a maximum raw capacity of 38.4 TB with 300GB drives.
From April 2010, DS8000 extent pools can be a mixture of SSD and spinning disk, so individual LUNs and mainframe CKD volumes can have some extents on SSD and some on Disk. If you already have discrete SSD and disk pools you can merge them together to create a mixed pool.
You can move LUNs or volumes manually and non-disruptively between storage tiers, but the Easy Tier product enhances this. It moves data at 1 Gb storage stripe level rather than full volumes and the movement is policy based, depending on how active or hot a 1GB storage stripe extent is.
Manual movement is called ELMR or Entire-LUN Manual Relocation, while the automated striped based migration is called Easy stripe.
Software
The DS software includes Flashcopy for internal subsystem point-in-time data copies, IBM Total Storage DS Manager for configuration and Metro/Global mirror for continuous inter-subsystem data replication.
The older ESS subsystems supported two kinds of z/OS Flashcopy, a basic version that just copied disks, and an advanced version that copied disks and files. DS only supports the advanced Flashcopy.
Flashcopy versions include;
multi-relationship, will support up to 12 targets;
Incremental, can refresh an old Flashcopy to bring the data to a new point-in-time without needing to recopy unchanged data;
Remote Mirror Flashcopy, permits dataset flash operations to a primary mirrored disk; Inband Flashcopy commands, permits the transmission of flashcopy commands to a remote site through a Metro Mirror link;
Consistency Groups, flash a group of volumes to a consistent point-in-time. A consistency group can span multiple disk subsystems.
Remote mirroring versions include;
Metro Mirror, synchronous remote mirroring up to 300km, was PPRC;
Global Copy, asynchronous remote data copy intended for data migration or backup, was PPRC-XD;
Global Mirror, asynchronous remote mirroring;
Metro/Global Mirror, three site remote replication, two sites being synchronous and the third asynchronous;
z/OS Global Mirror, z/OS host based asynchronous remote mirror, was called XRC;
Z/OS Metro/Global Mirror, three site remote replication, two sites being synchronous and quite close together, the third asynchronous and remote.
Openness
The DS subsystem series is self contained and does not interface with any other vendor's storage subsystem. For Open Systems data, IBM does support mirroring and copying to other vendor's subsystems if they are fronted with SVC virtualisation.
Futures
IBM promised a lot of enhancements to the DS series when they were first announced, and has delivered SATA drive support, space efficient Flashcopy, 4Gb FICON and virtual LUN space support so far.
The DS Subsystem LPARing is currently restricted to two LPARS both of which must be the same size. IBM propose to increase the number of LPARs, allow them to be different sizes and allow them to support different microcode levels in future
XIV
In early 2008 IBM bought XIV, a small storage company based in Tel Aviv. The XIV is a different type of box for IBM, and they sell it alongside their DS8000 range as an open systems solution.
Architecture
The XIV is based on a grid architecture of up to 15 interconnected but independent units called data modules. There is no common backplane, the modules are interconnected with Ethernet switches. Each data module contains standard Linux based Intel servers, up to 12 storage disks, cache and processors. Interface modules are a special type of data module and contain the above, but can also connect to external hosts through Fibre Channel and iSCSI interfaces. They also manage external mirroring and data migration tasks. Note that as there is no Ficon connectivity there is no z/OS support, which is unusual for a mainstream IBM storage unit.
As every module contains processors, all the modules share equally in processing the workload so a single module can be lost with little performance impact.
The other two types of component are the Ethernet switches and the UPS units. The redundant Ethernet switches connect the data and interface modules together so that every module can interface directly to every other module.
The XIV can be scaled out by adding new modules and scaled up by upgrading existing modules. When a new module is added, because it contains all of storage, cache and processing power, performance and bandwith capability increases in proportion.
If a new interface module is added, Ethernet and Fibre Channel interfaces are added in proportion.
The XIV can hold a maximum of 180 physical volumes, which with 2 TB drives, gives a maximum raw capacity of 360TB. The system is designed to be able to cope with losing a whole module and three disks in other modules without losing data, so it reserves the equivalent capacity of 1*12 disk module plus 3 disks for this. It also reserves another 4% of the space for Metadata, then the available space is reduced by 50% for partition copies, so the maximum effective capacity is ((360-30)*.96))/2 = 158.4TB
Logical Volume layout
The logical volumes as presented to the hosts are made up of 1 MB data units called partitions. These partitions are striped over all the physical disks and are also duplicated, with each copy held on different modules. The partition copies are called primary copy or secondary copy.
The mapping of logical volume partitions to physical disks and primary to secondary partitions is held in a distribution table and is carried out by the system at system startup. The distribution table is obviously a very critical component as the data would be inaccessible without it, so it is replicated over every module.
You have no control over where partitions are stored and in fact, you cannot interrogate the mapping from logical volume to partition to physical volume.
The XIV calculates its space in decimal GB ( 1 decimal GB = 1,000,000,000 bytes, a 'normal' GB = 1024*1024*1024 = 1,073,741,824 bytes). This makes volume allocation a challenge as volume calculations normally use the higher value.
A logical volume is physically made up of 17 decimal GB chunks or 15.83 standard GB chunks, so it's best to define logical volume sizes as multiples of 17GB. You can define a maximum of 16,377 logical volumes including snapshots.
The data is mirrored and striped over all the disks, which can be considered a form of RAID10, but IBM say this is not really the case as the distribution follows different rules.
The 1 MB partitions are 'pseudo-randomly' spread over the disks in a way that ensures that the partition pairs never reside in the same module, the data for each volume is spread evenly over all disks, and each logically adjacent partition on a volume is distributed across a different disk.
If you add more volumes, the system creates a new goal distribution which re-balances the data distribution to make sure it is still spread evenly over all the disks. So new physical disks are quickly used and contributing to overall system performance, with no action needed from yourself
Logical volumes are 'thin provisioned', that is, the system only allocates physical space as it is required. The logical volume size is the one that is defined to the host, but the physical size is allocated in 17GB chunks as needed, until the physical size reaches the limit set by the logical size.
Snapshots, or point-in-time copies of a volume, are fundamental to the XIV design. As the partitions that make up logical volumes are already tracked by pointers in the Distribution Table, it is very easy to create a snapshot by manipulating those pointers. Once a snapshot is created it is possible to update it, or even take another snapshot of it. Up to 16,000 snapshots can be created.
The XIV uses re-direct on write to manage snapshots, that is, if data is updated, the new data is written out to a new partition. With a copy-on-write snapshot, the old data must be copied over to a snapshot space before the new data can be written to disk. The proviso is that the update is going to be applied to the whole 1MB partition, otherwise the non-updated data must also be copied to the new location.
Snapshots can be made to be consistent over several logical volumes by creating consistency groups. In this case I/O activity is suspended over all the volumes in the group until all the snapshots are created.
It is possible to partition the storage into independent groups of volumes called storage pools to simplify administration. You can set a maximum storage pool size for each pool, which could be useful for setting quotas on applications or user groups. A master volume and all of its associated snapshots are always a part of only one Storage Pool.
HP
History
HP has been a Hitachi reseller for some time, but while they buy VSP hardware from Hitachi, HP supplies its own software. I've always viewed HP as a major Intel player, but not a supplier with limited presence in the mainframe market. HP certainly uses re-badged HDS subsystems for mainframe storage where they run a managed service.
P9500 Architecture
Because the HP P9500 is a re-badged Hitachi VSP, it has the same basic architecture.
For mainframe solutions, HP seems to supply standard HDS software. For Open Systems solutions, HP supplies its own software. This includes
Storageworks Continuous Access which provides synchronous data mirroring between subsystems
Storageworks Business Copy which provides full volume copy within the subsystem. This looks similar to EMC Timefinder rather than IBM FlashCopy
Storageworks Virtualization system, an internal and external virtualisation manager, can be used for data migration and replication
Storageworks LUN configuration and Security manager which is used to configure the XP12000, to define paths, array groups, volumes and LUNs
StorageWorks Performance Advisor which monitors performance within the XP subsystem
The P9500 has the same open architecture as the HDS VSP and supports the same range of OEM devices, plus it supports HP MSA devices.
HP StorageWorks 6400/8400 Enterprise Virtual Array
Architecture
The EVA is built around a pair of HSV (Hierarchical Storage Virtualization) controllers fronting up to 37 disk enclosures, depending in the model. Every disk enclosure is connected to both controllers so any disk can be accessed from either controller.
Each HSV controller has four times 4Gb/s FC host interfaces and contain the system cache, power supplies and enough battery power to maintain cache contents for up to 96 hours. The controllers connect up to nine drive enclosures with redundant FCAL. Larger EVA models support more FCAL loops and so can connect to more disk enclosures.
A disk enclosure can hold up to 12 drives, and the base rack can hold 18 enclosures, or 216 disk drives, which gives a maximum storage capacity of 216 TB in a single rack. The EVA8400 takes an expansion cabinet with an extra 9 enclosures, and so the maximum EVA capacity is 324TB. Connectivity within the enclosure is point-to-point.
EVA models are EVA4400, EVA6400 and EVA8400. The main differences between the models are cache sizes; varying from 4GB to 22GB and total disk capacity, carrying from 400GB to 324 TB. It is possible to physically upgrade a 4400 to a 8400 by swapping out controllers and reconnecting the disk enclosures, without having to migrate data between disks.
All models have same replication capability and management software and all models support up to 256 hosts with a maximum 32TB LUN size. They support up to 2048 LUNs (up to 256 per HBA) ranging in size from 1GB to 32TB per Virtual disk, in 1GB increments.
All subsystems support 300GB, 450GB and 600GB FC disk drives running at10K or 15K rpm and 1TB FATA drives. The 6400 and 8400 also support between 6 and 8 SSD drives with 72GB, 200GB or 400GB capacity.
EVA Software
HP StorageWorks Continuous Access EVA will replicate data between subsystems, but only supports HP StorageWorks enterprise virtual arrays. It normally runs at SAN distances, up to 20km, but can replicated data over longer distances if the HP StorageWorks IP Distance Gateway is added. This uses FCIP over the WAN.
The HP StorageWorks Business Copy incorporates Virtually Capacity-free Snapshot (Vsnaps), standard snapshots and Snapclones. Snaps can be managed through an HP Replication Solutions Manager GUI, or through a scripting interface.
It is possible to use an EVA as a tape library by adding the HP StorageWorks 12000 Virtual Library System software. This emulates several different libraries types and tape drive formats, and supports deduplication.
Oracle / SUN
History
Oracle bought out SUN, who bought out StorageTek in June 2005. StorageTek was an enterprise player. StorageTek was the first major company to put solid state cache in front of disks to make them perform better. In the late 1990s they brought out the Iceberg product. From an architectural perspective this box should have cleaned up the market with its innovative design and data management facilities. However even with IBM backing it was unable to really compete with the EMC 8xxx series and HDS 77xx series.
Oracle completed its takeover of SUN in early 2010 and seems to have de-emphasised it's disk range, dropping the OEM agreement with HDS and concentrating on the old SUN ZFS subsystems and the 6780 storage array. These systems do not support mainframe data.
Architecture
Touted by SUN as a high-end midrange storage device, with the bigger devices targeted at database, VMware and e-mail support. The 6*80 range starts with the 6180 and can be upgraded all the way to the top end 6780 just be replacing components and adding extra drive enclosures.
The main differences between the models are increasing host connectivity, cache, disks and internal bus connectivity.
Replaceable Host Inteface Cards (two per Controller)
Up to 16x 4Gb or 8Gb* FC Host Channels
Up to 8x 10Gb* Ethernet Host Channels
Up to 512 servers with dedicated LUN mapping can be attached to the array
Intel Xeon 2.4 GHz processors
Up to 32GB dedicated RAID Cache
Persistent cache backup Cache battery backup Battery-backed cache destage to flash disk
Custom XOR engine for RAID parity calculations RAID-5 and RAID-6, RAID 0,1,3,5,6 (p+q),10 supported
Up to 16x/28x Drive Enclosures
16 x 4Gb Dual or Quad FC Drive Channels, every channel can connect to any disk.
PCI-e connectivity between drive channels and drives.
Drives are all 3.5 in. form factor, low profile, Dual-port Fibre Channel Interface
SATA II drives; 7,200 rpm, 3Gbps
500GB / 1TB / 2TB
600GB (not available on 6180)
Solid State Disk drives (not available on 6180)
73GB
Enterprise Features:
Snapshot (supports up to 16 snapshots per volume depending on model with a maximum total of 2047 volume copies)
Data Copy
Data Replicaton (supports up to 128 replicated volumes depending on model)
Data encryption services restricted to FDE capable drives, which at present (Feb '11) is 600GB FC drives.
Common Array Manager (CAM), browser-based management interface.
NetApp
History
Founded in 1992, NetApp is a recent player in the disk market. The original NetApp devices were filers, that is, devices based on a file system. Traditional storage as supplied by the likes of IBM and EMC is block based. Shared File Storage systems use protocols like NFS and CIFS and can connect direct to a LAN, usually with an ethernet connection while Shared Block Storage systems require a SAN, usually connected with Fibre Channel cables. NetApp added block storage capability to its range in 2002, so it could break into the SAN storage market
The filers use NetApp's own Data ONTAP operating system, which now supports NFS, CIFS, iSCSI and Fibre Channel.
Models
The NetApp hardware range includes the FAS6200 Series, of which the FAS6280 is expandable up to 2,800TB raw. This series supports both file and block storage.
The FAS3200 Series supports both block and file storage, and can expand up to 136TB raw
The FAS2000 Series is files storage only, and can expand up to 136TB raw.
Software
NetApp software range includes various SnapMirror and SnapClone products for inter subsystem and intra subsystem data copying. They have MSSQL, Oracle and Exchange agents.
Storage Subsystem Features table
The various suppliers of mainframe disks are contrasted in the tables below. The first row explains why the factor might be important, the second row just presents the facts, which were correct at time of writing, January 2011. However I'd advise you to check with your salesperson for up to date details.
Vendor
IBM
EMC
HDS
SUN
HP
NetApp
Device
DS8700
V-MAX
VSP
6780
P9500
FAS6280
Subsystem Architecture
Internal Comms Architecture
See the previous page for an
explanation of the various types of comms architecture
PCI BUS
Virtual Matrix
PCI-e
PCI-e
PCI-e
PCI-e
Internal Bandwidth
How fast can data move inside the box? The numbers
quoted are marketing figures, you won't really see these numbers in
practice. See the Architecture section for
more information.
64 Gb/s
192 Gb/s with 8 engines
106 Gb/s
not known
Same as HDS
unknown
External Connectivity
How many external cables can you connect to the box, and how fast do they run.
218 4Gb FICON or FC,
64 ESCON
Up to 128*4Gb FICON
128*4Gb Fibre
64 iSCSI
64 Gb Ethernet
112 ESCON
112 FICON
224 FC with 1024 virtual channels per physical port
16*8GB/s FC
Same as HDS
8 * 10GbE, 8-32 * 8Gb FC, 0-24 * 6Gb SAS
Protocol Support
What kind of cables you can plug into the box. A good box will support a mixture of protocols.
Ficon, Escon, Fibre Channel
Ficon, Fibre Channel, iSCSI, GB Ethernet
Ficon, Escon, Fibre Channel
Fibre Channel
Ficon, Escon, Fibre Channel
FC, FCoE, iSCSI, NFS, CIFS, HTTP, FTP
Disk Connectivity
See the previous page for
details of disk connectivity.
Switched FC-AL
FC-AL 4Gb 2 port FC
FC-AL
FC-AL
Same as HDS
6Gb SAS
Storage Virtualisation Server
Can the storage subsystem act as a virtualisation engine
in conjunction with a SAN? This enables lots of disparate storage
to be controlled from one central point, including mirroring
between different vendor's devices.
No
No
Yes
no
Yes
No
LPAR Capable
Can the storage subsystem be split logically, so it
appears to be several separate systems, perhaps running different
levels of microcode?
No, but older models do support 2 LPARS with a 50/50 or 75/25 split.
No
Yes (32 LPARS, Z/OS data in single LPAR only)
No
Yes
No
Subsystem Capacities
Maximum, and maximum effective capacity
How much data can you cram into the box? The maximum
configured capacity will be less than the rated capacity, partly due
to RAID overhead, and partly due to 3390 emulation overhead. The configured
figures are for Mainframe emulation, Open Systems emulation will be
higher. The maximum EFFECTIVE capacity for a mainframe workload running
IO intensive TP systems can be as little as 33% of the maximum capacity,
if you want adequate performance.
450 TB with 450GB FC disks
1 PB with 1TB SATA disks (raw)
Using 2,400*1 TB drives, 2.4 PB raw or 1,967TB usable with RAID7+1 on z/OS
Using 2,400*146 GB drives, 350TB raw or 287TB usable with RAID7+1 on Open Systems.
2,269B raw maximum, RAID6 usable capacities
1,690 TB; Open Systems
796 TB; z/OS
896 TB
Same as HDS
2,880 TB
Cache size
In theory, the bigger the cache, the better the performance,
as you will get a better read-hit ratio, and big writes should not
flood the cache. If the cache is segmented, it is more resilient,
and has more data paths through it
32-384 GB
1TB with 8 engines
512 GB
192 concurrent control cache operations
64 concurrent data cache operations
32GB
Same as HDS
8TB Flash
Number of LUNs supported
???
65336, LUN or CKD. 2TB max size./td>
4096
Disk types
Physical disk size
How big are the real, spinning disks and how fast do they run. The bigger the
disks, the less you pay for a terabyte, but bigger disks might be
performance bottlenecks. If you have really large disks, then there
should be fewer of them on an FC-AL loop. Faster speeds less rotational delay.
146, 300 GB and 450 GB FC; all 15,000 rpm
1TB SATA
Do you mirror data between two sites? If so you need
this. There are basically two flavours of mirroring, SRDF from EMC,
and PPRC from IBM. SRDF is arguably the better solution technically,
but it locks you into EMC disks. PPRC is used by everyone. The
remote mirroring section has more details.
Global Mirror, asynchronous
Metro Mirror (PPRC), synchronous
Synchronous(SRDF/S) and asynchronous(SRDF/A) data replication between subsystems.
SRDF/DM will migrate data between subsystems.
SRDF/AR works with TimeFinder to create remote data replicas.
SRDF products are all EMC to EMC
SRDF can emulate Metro mirror and Global mirror
'Instant Copy' of volumes or datasets. Can be used
for instant backups, or to create test data. Some implementations
require a complete new disk, and so double the storage. Some implementations
work on pointers, and just need a little more storage.
Timefinder at volume or dataset level. BCV version
requires a complete volume be supplied, newer 'snap' version just uses pointers.
EMC Compatible Flash (FlashCopy)
Shadow Image at volume level
Copy on write snapshot
up to 16 snapshot volumes
Storageworks copy software
SnapMirror
Z/OS features
3380/90 emulation
3380 drives are older legacy technology and most sites have now converted to 3390. 3390 comes in multiple sizes, a
3390-3 will hold 2.8 GB. The newest model is the 3390-M.
All models, including EAV
All models, including EAV
All models, supports up to 65,536 logical devices (the older USPs just support 16,384 Open Systems devices)
Parallel Access Volume and Multiple Allegiance. See
the implementation tips section for details.
Used to permit multi-tasking to logical devices
Yes
Yes , including HyperPAV support
Yes
N/A
Yes , including HyperPAV support
N/A
Manufacturer
IBM
EMC
HDS
SUN
HP
NetApp
Device
DS8700
V-MAX
VSP
6780
P9500
FAS6280
Price is usually very negotiable, but be sure to make sure that the vendor quotes for a complete solution with no hidden extras. Also, make sure that you get capped capacity upgrade prices, including increased software charges as software is usually charged by capacity tiers.