Parallel Running

A typical batch system, even in a medium sized shop, can easily run to several thousand jobs per night. You need someone with an intimate knowledge of how the various applications hang together, and how they share their data, to be able to investigate running more work in parallel. This task is normally assigned to a Batch Analyst, but a less skilled person can understand complex batch systems with the aid of a picture or two.

Tools such as Beta44 from Beta Systems will build up a picture of your schedule, and draw it or your PC or printer. These tools will plot out the critical path through the schedule, and can make it easier to spot bottlenecks.

If you want to analyse a small application, then it is possible to do this using an Excel spreadsheet. First you need to get hold of the start and end times for the jobs in your application. You can either get this from SMF record types 31 & 32, or just extract messages 'IEF403I' (job start) and 'IEF404I' (job end) from the SYSLOG

If your batch suite spans over midnight, then you need to specify both day and time in the spreadsheet, as Excel would consider a job starting at 01:00, as running before one starting at 23:00. However if you use SAS to extract your data, then there is no SAS format corresponding to the Excel dd/mm/yy hh:mm:ss, and no relevant Excel import format to pick this up as two columns. Try the following SAS code

FILE SASOUT NOTITLES NOPRINT; 
   PUT @2  JOB                
       @11 STARTD             
       @19 '#'                
       @20 STARTT             
       @29 ENDD               
       @37 '#'                
       @38 ENDT;

This writes the output as Jobname dd/mm/yy#hh:mm:ss dd/mm/yy#hh.mm.ss When imported into Excel, this is formatted as three columns. You then define columns B & C as format dd/mm/yy hh:mm:ss, then do a global change of '#' to ' '. Changing the cell format does not change the format of any existing data, but the global change automatically changes the data type from 'text' to 'date'.

If your applications include interim backups, then these are prime candidates for Concurrent Copy, or its variants. At a very basic level, concurrent copy works by establishing a synchronisation point for all the datasets requiring backup. It then releases ENQ locks, and allows other work to update these files, while the backup proceeds. If data is updated before it gets a backup, then copies of the original data are held in storage, until the backup job catches up. RVA2, or ICEBERG machines have a 'SNAPSHOT' equivalent, and ESS (SHARK) machines, 'FLASHCOPY', both of which work by defining a set of pointers to the old data, and maintaining these pointers if the data is overwritten by parallel applications. Use of Concurrent, or Snapshot copy, allows application work to run in parallel with backups, so saving dedicated backup time.

Database applications can also benefit from parallelism, if you use 'fuzzy' image copies. Here, once a synchronisation point is established, the backup proceeds and updates are logged as normal. The data can be recovered by restoring from the backup, and then applying the logs.

back to top

By entering and using this site, you accept the conditions and limitations of use