Recovering a recalled file from HSM

This procedure was supplied by Robin Lowe.

Data which is held on DFHSM is not on primary disk, so if you use utilities like DFDSS or FDRABR for your disk backups, your backups will eventually expire and the only copy of that data will held on DFHSM ML1 or ML2. The location of data held within HSM is recorded in a 'D' record, and when you recall a dataset from HSM, that 'D' record is deleted, any data held on ML1 is deleted and any data held on ML2 is marked for deletion. This can be a problem if you recall a dataset then delete it before the next backup run, as if it was migrated several days before, there might not be any backups for it and then you may have lost it. This often happens when an older GDG has rolled off.
In some instances it may be possible to recover a dataset from a HSM migrated copy, if that dataset has been recently recalled. If the data was on an ML2 tape, the data is intact until the cartridge is RECYCLED. If the data was on ML1, the ML1 dataset entry will be deleted upon completion of the RECALL, but is restorable, using the standard file restore procedures, provided you back up your ML1 volumes. You need an HSM D record (migrated dataset record) for the recovery, and this may or may not exist.

This process requires that you backup your ML1 disks and run a report job on a daily basis, details of which are in the link below

Keeping the data needed to recover a recalled file from HSM

Use this SAS job to create a daily record of HSM MCDS information.

// your job card goes here
//STEPSAS  EXEC ASV6  <- the name of your SAS catalogued procedure
//WORK    DD SPACE=(CYL,(100,35)),UNIT=DISK
//SASLOG  DD SYSOUT=*
//SASLIST  DD DSN=HSM.MCDS.LISTING(+1),DISP=(,CATLG),
//        SPACE=(CYL,(200,50),RLSE)
//DIN1    DD DSN=HSM.MCDS,DISP=SHR <- your MCDS file name
//SYSIN   DD *
DATA MCDSINFO(KEEP= DSNAME MIGDSN MIGVOL LOC FILENO BLOKNO);
  INFILE DIN1 ;
  FORMAT FILENO HEX4. BLOKNO HEX4.;
  INPUT @47 RECTYPE £1. @;
  IF RECTYPE ¬= '00000000'B THEN DELETE; /* TEST FOR MIG RECORD */
  INPUT @1 DSNAME    £44. @;
  IF SUBSTR(DSNAME,1,15) = 'Z99999999999999' THEN DELETE;
  INPUT @65 MIGVOL     £6.
    @71 FLAG    PIB1.
    @273 FILENO    PIB2.
    @275 BLOKNO      PIB2.
    @221 MIGDSN     £44.;
  IF FLAG = '.....1..'B THEN LOC = 'ML2';
  ELSE IF FLAG = '.....0..'B THEN LOC = 'ML1';
  ELSE LOC = '?';
  IF FLAG = '1.......'B THEN MIGCOPY = 'Y';
  ELSE DELETE;
  IF FLAG = '....1...'B THEN MIGDSN = 'SMALL-DATASET-PACKING-DATASE
  RETURN;
PROC PRINT U SPLIT='*' NOOBS UNIFORM;
  TITLE 'DATASETS IN THE HSM MCDS';
  LABEL DSNAME='DATASET*NAME'
    MIGVOL='MIGRATION*VOLSER'
    LOC='MIGRATION*LEVEL'
    FILENO = 'FAST*BLOCKID'
    BLOKNO = 'REAL*BLOCKID';

To be able to recover lost ML1 data you need to dump your ML1 data on a daily basis. How you do this will depend on what backup software you use. You will also need to decide how long you keep the ML1 dumps for. This one is really up to you, but for these purposes there is little point in keeping the dumps for longer than the longest 'keep the data set on ML1' management class.

OK, so you have ML1 backups and a daily report, and now you need to recover a deleted file. The first thing you need to do is to check to see if the D record exists, which is the starting point below. You may also use some of the following steps, depending on what conditions you find and where your date is.

   EADM Advert

Accelerate DB2 Write with zHyperWrite and "EADM™ by Improving DB2 Logs Volumes Response Time:


Check to see if the D record exists

You can check by issuing the following HSM command :

HSEND LIST DSN('dsname') MCDS TERM

If you get the following result with MIGVOL=ONLINE, it is possible to reset the HSM D record and recall the dataset immediately if it is on ML2 :

DSN=dsname       MIGVOL=ONLINE DSO=*** SDSP=NO
 LAST REF=03/01/16 MIG=03/01/16 TRKS=000000 2K BLKS=******* TIMES MIG=000
 16K BLKS=000000 LAST MIGVOL=*NONE*

Otherwise, if the dataset was on ML2, you need to recatalog the file with an entry of MIGRAT as explained in this link.

If the dataset was on ML1, the data will have to be restored from backup.

If the D record does not exist, you will get a 'NO MCDS INFORMATION FOUND FOR DATASET' message. In this case, carry on to

If you need to, you can get back to the start of the process here.

Resetting the D record

All you need to do is to reset the MIGVOL value to the original ML2 volume by changing the ASSIGNEDBIT to ON using the following command :

HSEND FIXCDS D dsname ASSIGNEDBIT(ON)

You will get the following reply :

ARC0197I TYPE D, KEY dsname, FIXCDS ASSIGNED SUCCESSFUL

Check this D record is now ready for RECALL by reissuing the command

HSEND LIST DSN('dsname') MCDS TERM

You should see a display similar to this ;

DSN=dsname MIGVOL=ML1010 DSO=PS SDSP=NO
LAST REF=03/01/07 MIG=03/01/15 TRKS=000030 2K BLKS=0000307 TIMES MIG=001
16K BLKS=****** LAST MIGVOL=******

You can now recall the dataset.

 


Find the D record information

To find the D record information, you will need to browse the GDG which contains your daily report of the MCDSDATA. See the pre-requisite actions section to find out how to create this GDG.

This is the daily snapshot of the Migration Control Dataset. The fields from the D record listed are :

  • FAST BLOCKID (ML2 ONLY x'0000' FOR ML1)
  • REAL BLOCKID (ML2 ONLY x'0000' FOR ML1)
  • z/OS DATASET NAME
  • MIGRATION VOLSER
  • HSM MIGRATED DATASET NAME
  • MIGRATION LEVEL

You may have to browse several of the GDGs until you find the entry you are looking for. Make note of the HSM dataset name, the MIGRATION LEVEL and MIGRATION VOLSER for use in the steps below.

Note the goovy number of the GDG that you need to read in the REBUILD process (e.g. your.file,name.GnnnnVnn) or it's relative generation number (e.g. your.file.name(-n)). Now you need to rebuild the D record.

 


Recover the ML1 dataset

You are pretty much on your own with this one. First thing is that you must have daily backups of your ML1 pool, and the backup you want must still exist. How you find this out will depend entirely on your own procedures, and on what backup software you run. To do this, you will need the HSM dataset name which you should have made a note of when your were checking the D record information.

Once you have the ML1 file restored, you need to rebuild the D record.

If you cannot restore the ML1 file, because the backup has expired, then I'm afraid that you can't get the file back.


Check that the ML2 tape has not been recycled

To check the ML2 volume, do the following :

Go into your tape management system, and check that the volume is not scratch, and that the dataset on the tape is HSM.HMIGTAPE.DATASET

Additionally, issue the following HSM command :

hsend list ttoc(vvvvvv) nodsi term

the result will look something like this :

VOLSER=vvvvvv UNITNAME=uuuu VOL TYPE= ML2 CAPACITY=nnnnnnnn RACF= YES
VALID BLKS=nnnnnnnn PCT VALID= 077 VOL STATUS= FULL PREV VOL=*NONE*
SUCC VOL=*NONE* NUM REC=nnn ONE FILE=YES ALTVOL=vvvvvv
LIBRARY=*NO LIB* STORAGE GROUP=**NO SG*

If the tape is still valid, proceed to

If the ML2 tape has been recycled, then sorry, but your data is gone.


Rebuild the D record

To rebuild the HSM MCDS D record, you need to enter several HSM fix commands. The easiest way to do this is to generate them with a SAS job. In the following procedure, I'm assuming you are working in a PDS called MY.PDS.FILE. Of course, you will substitute this with your own library name.

  1. In the GDG which you use to record your MCDS data, note the offset of the first variable. To do this, in browse mode, note the column (2 or 3) that the first variable begins at.
  2. Add a member ML2DSN to MY.PDS.FILE with the z/OS DATASET NAME of the dataset you wish to RECALL. You can add more than one dataset, but all dataset names must start in column 3.
  3. Add a member REBUILD in MY.PDS.FILE with the the JCL below. Remember to add the correct name of the MCDSDATA data gds you wish to input in DDNAME DIN2.

    //your jobcard standards
    //S1     EXEC ASV6 -- your SAS cataloged procedure
    //WORK     DD SPACE=(CYL,(100,50)),UNIT=DISK
    //SASLOG   DD SYSOUT=*
    //SASLIST  DD SYSOUT=J,OUTLIM=100000
    //DIN1     DD DSN=MY.PDS.FILE(ML2DSN),DISP=SHR
    //DIN2     DD DSN=GDG.WHICH.CONTAINS.MCDSDATA.GnnnnVnn,DISP=SHR
    //DOUT1    DD DSN=MY.PDS.FILE(A#DEF1),DISP=SHR
    //DOUT1A   DD DSN=MY.PDS.FILE(A#DEF2),DISP=SHR
    //DOUT2    DD DSN=MY.PDS.FILE(A#REC1),DISP=SHR
    //DOUT3    DD DSN=MY.PDS.FILE(A#REC2),DISP=SHR
    //SYSIN    DD *
    DATA FILE1;
      INFILE DIN1;
      INPUT @3 DSNAME £44. ;
      OUTPUT;
      RETURN;

    PROC SORT DATA = FILE1;
      BY DSNAME;

    DATA FILE2 ;
      ATTRIB DSNAME FORMAT=£CHAR44.;
      ATTRIB MIGDSN FORMAT=£CHAR44.;
      INFILE DIN2 ;
      INPUT  FILENO1 £ FILENO2 £ DSNAME £MVOL £ MIGDSN £ LOC £;
      OUTPUT;
    RETURN;

    PROC SORT DATA = FILE2;
      BY DSNAME;

    DATA FILE3 FILE4 FILE5;
      MERGE FILE1(IN=IN1) FILE2(IN=IN2);
      BY DSNAME;
      IF IN1=1 AND IN2 = 1 THEN DO;
        IF LOC = 'ML1' THEN OUTPUT FILE3;
        ELSE IF LOC = 'ML2' THEN OUTPUT FILE4;
      ELSE DELETE;
      END;
      ELSE IF IN1=1 AND IN2 = 0 THEN OUTPUT FILE5;
      ELSE DELETE;
    RETURN;

    PROC PRINT DATA=FILE3;
    PROC PRINT DATA=FILE4;
    PROC PRINT DATA=FILE5;

    DATA -NULL-;
      SET FILE3;
      BY DSNAME;
      FILE DOUT1 NOTITLES;
      M = -1;
      PUT @2 'DEFINE NVSAM(NAME(''' DSNAME + M ''') -'
        / @2 '    VOLUMES(MIGRAT) DEVT(3380))';
    RETURN;

    DATA -NULL-;
      SET FILE4;
      BY DSNAME;
      FILE DOUT1A NOTITLES;
      M = -1;
      PUT @2 'DEFINE NVSAM(NAME(''' DSNAME + M ''') -'
        / @2 '    VOLUMES(MIGRAT) DEVT(3490))';
    RETURN;

    DATA -NULL-;
      SET FILE3;
      BY DSNAME;
      FILE DOUT2 NOTITLES;
      M = -1;
      OFFS = 06;
      OFF1 = 64;
      OFF2 = 72;
      OFF3 = 88;
      OFF4 = 156;
      OFF5 = 208;
      OFF6 = 210;
      OFF7 = 216;

    PVOL = 'GNLS2H';
    PUT @1 'HSEND WAIT FIXCDS D ' DSNAME ' DELETE'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'ADDMIGRATEDDATASET(' MVOL +M ')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF1 HEX3. ''' -'
       / @1 'X''' PVOL HEX12. ''')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF2 HEX3. ''' -'
       / @1 'X''C7C5D5403030200E'')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF3 HEX3. ''' -'
       / @1 'X''3030200F'')' ;
     IF MIGDSN =:'SMALL' THEN DO;
       PUT @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFFS HEX3. ''' -'
       / @1 'X''8801'')';
      END;
      ELSE DO;
       PUT @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFFS HEX3. ''' -'
       / @1 'X''8001'')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF4 HEX3. ''' -'
       / @1 ''MIGDSN +M ''')' ;
       END;
       PUT @1 'HSEND RECALL ' DSNAME;

     DATA -NULL-;
       SET FILE4;
       BY DSNAME;
       FILE DOUT3 NOTITLES;
       M = -1;
       OFFS = 06;
       OFF1 = 64;
       OFF2 = 72;
       OFF3 = 88;
       OFF4 = 156;
       OFF5 = 208;
       OFF6 = 210;
       OFF7 = 216;
       PVOL = 'GNLS2H';
       PUT @1 'HSEND WAIT FIXCDS D ' DSNAME ' DELETE'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'ADDMIGRATEDDATASET(' MVOL +M ')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF1 HEX3. ''' -'
       / @1 'X''' PVOL HEX12. ''')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF2 HEX3. ''' -'
       / @1 'X''C7C5D5403010200F'')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF3 HEX3. ''' -'
       / @1 'X''78048081'')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF4 HEX3. ''' -'
       / @1 ''MIGDSN +M ''')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF5 HEX3. ''' -'
       / @1 'X''' FILENO1 +M ''')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFF6 HEX3. ''' -'
       / @1 'X''' FILENO2 +M ''')'
       / @1 'HSEND FIXCDS D ' DSNAME ' -'
       / @1 'PATCH(X''' OFFS HEX3. ''' -'
       / @1 'X''8401'')';
    PUT @1 'HSEND RECALL ' DSNAME;

  4. Check the SAS code in the SYSIN for the DATA FILE2 INPUT statement and ensure the correct set of variable definitions is coded.
    • Choices are the set that starts at offset 2, e.g. :

      INPUT @2 FILENO1 £4. @12 FILENO2 £4. @21 DSNAME £44. @69 MVOL £6.
      @80 MIGDSN £44. @126 LOC £3.;

    • or the set that starts at offset 3

      INPUT @3 FILENO1 £4. @13 FILENO2 £4. @22 DSNAME £44. @70 MVOL £6.
      @81 MIGDSN £44. @126 LOC £3.;

  5. Submit MY.PDS.FILE(REBUILD)
  6. Check job completes cc=0.
  7. The job will create two pairs of commands as follows :
    • For ML1 :

      MY.PDS.FILE(A#DEF1)
      MY.PDS.FILE(A#REC1)

    • For ML2 :

      MY.PDS.FILE(A#DEF2)
      MY.PDS.FILE(A#REC2)

Once you have rebuild the D record successfully, you need to define the MIGRAT catalog entry

 


Define the MIGRAT catalog entry

If the dataset was on ML1 and you have restored the dataset back to ML1, run the following

DEFINE NVSAM(NAME(' dsname ') VOLUMES(MIGRAT) DEVT(3390))

If the dataset was on ML2, do the following :

DEFINE NVSAM(NAME(' dsname ') VOLUMES(MIGRAT) DEVT(3490))

Your ML2 DEVT will depend on what type of tapes you are using of course. You can now carry on to the next section and recall your dataset.


Recall the dataset

If the dataset was on ML1, do the following :

If you have not had to run REBUILD, simply issue a RECALL command for the dataset.

If you have run REBUILD, Execute MY.PDS.FILE(A#REC1)
Executes the HSM commands to do the following :

  1. Delete the current D record (if one exists)
  2. Issue FIXCDS commands to create the necessary D record.

If the dataset was on ML2, do the following :

If you have not had to run REBUILD, simply issue a RECALL command for the dataset.

If you have run REBUILD, Execute MY.PDS.FILE(A#REC2)
Executes the HSM commands to do the following :

  1. Delete the current D record (if one exists)
  2. Issue FIXCDS commands to create the necessary D record.
  3. Issue the RECALL command.

back to top