Author: prino
Subject: Replace repeated JCL with looping REXX
Posted: Thu Oct 13, 2016 5:30 am (GMT 5.5)
I've got a job consisting of 54 steps, 18 of them executing a PGM, 36 of them executing a PROC, which in turn contains 26 steps executing a PGM... Needlessly to say, the job actually runs as four jobs, and that, at the moment, for 179 different input datasets... (So 716 jobs, ouch...)
Now it would be nice if I could reduce it to just 19 steps, merging the 36 PROCs into a nice looping bit of REXX, and some preliminary testing seems to confirm that this is indeed very well possible. However, the JCL uses sh*tload of SET VAR=value statements, and I don't really want to pass sh*tloads of parameters to the REXX exec, so I've been trying to find a smart way of encoding them, as there doesn't(?) seem to be any way of actually accessing them via control blocks, which makes sense as they're only needed at the JCL conversion stage.
FWIW, the job currently runs 36 versions of a program that produces 6 output files (it used to produce 4), and the steps following the EXEC PGM=version[NN] uses SuperC to zap the output datasets of "version[NN-1]" if they are the same as the current ones, or saves them into a PDS(E). For what it's worth here are the abbreviated JCL:
and PROC:
As you can see, the number of SET VAR=value statements is pretty large, and even more (my default libraries) are included in the "INCLUDE MEMBER=IMOI" member. I could obviously put all of them into a text member that is read by the replacement REXX exec, but that would mean that I need to update two datasets if I ever want to port these jobs to another system. I have thought about a dirty trick, adding a
and read that in inside the replacement REXX exec, parsing out the relevant SET statements.
Any other thoughs on this?
I'm also curious as to the efficiency of running this as a straight BATCH job (as it is at the moment), or as a job that runs ISPF in batch - at one site another smart-ass programmer thought using an edit-macro in batch was an ideal way to massage some output, but after just one run in production that approach was abandoned do to an excessive CPU usage. Here the REXX exec would only be allocating files (rather a lot...), running the program to be tested, ISRSUPC and SORT, be it under ISPF...
And as a PS, the current JCL member is submitted by an edit macro that updates the three 'XXX' strings to 001..178, the 178 input files and with a slight delay (0.5 sec) between the four jobs that make up one test, to avoid starting a later one before an earlier one.
_________________
Robert AH Prins
robert.ah.prins @ the.17+Gb.Google thingy
No programming here (yet)![icon_smile.gif]()
Subject: Replace repeated JCL with looping REXX
Posted: Thu Oct 13, 2016 5:30 am (GMT 5.5)
I've got a job consisting of 54 steps, 18 of them executing a PGM, 36 of them executing a PROC, which in turn contains 26 steps executing a PGM... Needlessly to say, the job actually runs as four jobs, and that, at the moment, for 179 different input datasets... (So 716 jobs, ouch...)
Now it would be nice if I could reduce it to just 19 steps, merging the 36 PROCs into a nice looping bit of REXX, and some preliminary testing seems to confirm that this is indeed very well possible. However, the JCL uses sh*tload of SET VAR=value statements, and I don't really want to pass sh*tloads of parameters to the REXX exec, so I've been trying to find a smart way of encoding them, as there doesn't(?) seem to be any way of actually accessing them via control blocks, which makes sense as they're only needed at the JCL conversion stage.
FWIW, the job currently runs 36 versions of a program that produces 6 output files (it used to produce 4), and the steps following the EXEC PGM=version[NN] uses SuperC to zap the output datasets of "version[NN-1]" if they are the same as the current ones, or saves them into a PDS(E). For what it's worth here are the abbreviated JCL:
| Code: |
| //PRINOXXX JOB (PRINO), // 'DIFFS DEV - EQ', // CLASS=A, // MSGCLASS=H, // MSGLEVEL=(2,0), // NOTIFY=&SYSUID - - - - - - - - - - - - - - - - 8 Line(s) not Displayed //*!WAIT00500!********************************************************* //PROCS JCLLIB ORDER=(PRINO.RAHP.PROC) //********************************************************************* // INCLUDE MEMBER=IMOI //* - - - - - - - - - - - - - - - - 4 Line(s) not Displayed // SET SRCE='TRXXX' QQQ 'WORKLIFT' // SET TID='XXX' QQQ 'ALL' //* - - - - - - - - - - - - - - - - 4 Line(s) not Displayed // SET TY='TR' -+- TRIP // SET L='L' | // SET P='P' -+ //* // SET OTEXT='PRINO.LIFT.TEMP' Must be PDSE! //* - - - - - - - - - - - - - - - - 1 Line(s) not Displayed // SET PRE='EQ' Enterprise PL/I //* // SET SYSOUT1='*' // SET SYSOUTD='*' // SET DIFFS='PRINO.RAHP.DIFFS.COPY' Must be PDSE! //* - - - - - - - - - - - - - - - - 4 Line(s) not Displayed // SET OUTSORT='DUMMY' // SET OUTSUPC='DUMMY' // SET OLDDEL='(OLD,DELETE)' //********************************************************************* //DEL EXEC PGM=IEFBR14 //* - - - - - - - - - - - - - - - - 12 Line(s) not Displayed //SUPERS DD DSN=&SYSUID..SUPER.S&TID, // DISP=(MOD,DELETE), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FB,LRECL=202,BLKSIZE=0) - - - - - - - - - - - - - - - - 18 Line(s) not Displayed //* //SYSIN DD DSN=&SYSUID..SYSIN.SUPERC.&TY.&TID, // DISP=(MOD,DELETE), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=0) //* //SYSIN DD DSN=&SYSUID..SYSIN.SUPERCS.&TY.&TID, // DISP=(MOD,DELETE), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=0) //********************************************************************* //ALLOC EXEC PGM=IEFBR14 //* - - - - - - - - - - - - - - - - 12 Line(s) not Displayed //SUPERS DD DSN=&SYSUID..SUPER.S&TID, // DISP=(,CATLG), // UNIT=&DASD, // SPACE=(TRK,(30,60),RLSE), // DCB=(RECFM=FB,LRECL=202,BLKSIZE=0) - - - - - - - - - - - - - - - - 18 Line(s) not Displayed //********************************************************************* //CARDS1 EXEC PGM=SORT //* //SORTIN DD * worksize 262144 dpline ' Totals per ' //* //SORTOUT DD DSN=&SYSUID..SYSIN.SUPERC.&TY.&TID, // DISP=(,CATLG), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=0) //* //SYSIN DD DSN=&DATA(SORTCOPY), // DISP=SHR //* //SYSOUT DD &OUTSORT //********************************************************************* //CARDS2 EXEC PGM=SORT //* //SORTIN DD DSN=&SYSUID..SYSIN.SUPERC.&TY.&TID, // DISP=SHR // DD * ochgt ':','.' nchgt ':','.' //* //SORTOUT DD DSN=&SYSUID..SYSIN.SUPERCS.&TY.&TID, // DISP=(,CATLG), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FB,LRECL=80,BLKSIZE=0) //* //SYSIN DD DSN=&DATA(SORTCOPY), // DISP=SHR //* //SYSOUT DD DUMMY //********************************************************************* //SA01 EXEC RUN2PDS, // PID='01', // OPID='01', // CALLER='SA01.',SKIPF='4095' //* //SA0201 EXEC RUN2PDS, // PID='02', // OPID='01', // CALLER='SA0201.' //* - - - - - - - - - - - - - - - 302 Line(s) not Displayed //********************************************************************* //* Between 32 & 33 SUPC must be SET to 'S' (Times to ISO 8601) //********************************************************************* //SA3332 EXEC RUN2PDS, // SUPC='S', // PID='33', // OPID='32', // CALLER='SA3332.' //* - - - - - - - - - - - - - - - - 10 Line(s) not Displayed //SA3635 EXEC RUN2PDS, // PID=, // OPID='35', // CALLER='SA3635.' //* // SET PID= // SET CALLER='SA3635.' //********************************************************************* //IF110 IF &CALLER.SA020.RC = 0 THEN //* //COPYS EXEC PGM=SORT //* //SORTIN DD DSN=&SYSUID..&TY.&TID..&L.S&PID, // DISP=&OLDDEL //* //SORTOUT DD DSN=&OTEXT(T&TID.S&P.99), // DISP=SHR //* //SYSIN DD DSN=&DATA(SORTCOPY), // DISP=SHR //* //SYSOUT DD DUMMY //********************************************************************* - - - - - - - - - - - - - - - - 65 Line(s) not Displayed //MODSUP EXEC PGM=IKJEFT01, // DYNAMNBR=50, // REGION=0M, // PARM='%ISPFBAT EHHDIFF' //* // INCLUDE MEMBER=ISPF@ICI //* //SUMM DD DSN=&SYSUID..SUPER.S&TID, // DISP=SHR - - - - - - - - - - - - - - - - 10 Line(s) not Displayed //********************************************************************* //SUPERS EXEC PGM=SORT //* //SORTIN DD DSN=&SYSUID..SUPER.S&TID, // DISP=&OLDDEL //* //SORTOUT DD DSN=&DIFFS(LS&TID), // DISP=SHR //* //SYSIN DD DSN=&DATA(SORTCOPY), // DISP=SHR //* //SYSOUT DD DUMMY - - - - - - - - - - - - - - - - 65 Line(s) not Displayed //* //IF110E ENDIF //********************************************************************* //SA120 EXEC PGM=IEFBR14 //* //SYSIN DD DSN=&SYSUID..SYSIN.SUPERC.&TY.&TID, // DISP=&OLDDEL //* //SYSIN DD DSN=&SYSUID..SYSIN.SUPERCS.&TY.&TID, // DISP=&OLDDEL |
and PROC:
| Code: |
//RUN2PDS PROC LOAD='PRINO.RAHP.LOAD', // STEP1='PRINO.RAHP.LOAD', - - - - - - - - - - - - - - - - 1 Line(s) not Displayed // CALLER=, // SUPC=, // SKIPF='0' //********************************************************************* //SA010 EXEC PGM=IEFBR14 //* //SUMMOUT DD DSN=&SYSUID..&TY.&TID..&L.S&PID, // DISP=(MOD,DELETE), // UNIT=&DASD, // SPACE=(TRK,0), // DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0) - - - - - - - - - - - - - - - - 36 Line(s) not Displayed //********************************************************************* //SA020 EXEC PGM=&PRE.01&PID //* //STEPLIB DD DSN=&STEP1, // DISP=SHR - - - - - - - - - - - - - - - - 2 Line(s) not Displayed // DD DSN=&LOAD, // DISP=SHR //* //LIFTIN DD DSN=&VEXT(&SRCE), // DISP=SHR //* //SUMMOUT DD DSN=&SYSUID..&TY.&TID..&L.S&PID, // DISP=(,CATLG), // UNIT=&DASD, // SPACE=(TRK,(10,10),RLSE), // DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0) - - - - - - - - - - - - - - - - 42 Line(s) not Displayed //********************************************************************* //IFPLI IF &CALLER.SA020.RC = 0 & &CALLER.SA020.RC = &SKIPF THEN //********************************************************************* //LS030 EXEC PGM=ISRSUPC, // PARM=(DELTAL,LINECMP, // 'DPBLKCL FMVLNS REFMOVR LONGLN NOPRTCC') //* //NEWDD DD DSN=&SYSUID..&TY.&TID..&L.S&PID,DISP=SHR //OLDDD DD DSN=&SYSUID..&TY.&TID..&L.S&OPID,DISP=SHR //* //SYSIN DD DSN=&SYSUID..SYSIN.SUPERC&SUPC..&TY.&TID, // DISP=SHR //* //OUTDD DD &OUTSUPC - - - - - - - - - - - - - - - - 5 Line(s) not Displayed //********************************************************************* //IFRUN IF &CALLER.LS030.RUN THEN //* //IF031 IF &CALLER.LS030.RC = 1 THEN //* //LS031 EXEC PGM=ISRSUPC, // PARM=(DELTAL,LINECMP, // 'DPBLKCL FMVLNS REFMOVR NOSUMS LONGLN NOPRTCC APNDLST') //* //NEWDD DD DSN=&SYSUID..&TY.&TID..&L.S&PID,DISP=SHR //OLDDD DD DSN=&SYSUID..&TY.&TID..&L.S&OPID,DISP=SHR //* //OUTDD DD DSN=&SYSUID..SUPER.S&TID, // DISP=MOD //* //IF031E ENDIF //********************************************************************* //IF040 IF &CALLER.LS030.RC = 0 | // &CALLER.LS030.RC = 28 THEN //* //LS040 EXEC PGM=IEFBR14 //S DD DSN=&SYSUID..&TY.&TID..&L.S&OPID, // DISP=(MOD,DELETE), // UNIT=&DASD, // SPACE=(TRK,1), // DCB=(RECFM=FBA,LRECL=121,BLKSIZE=0) //* // ELSE //* //IF041 IF &CALLER.LS030.RC = 1 THEN //* //LS041 EXEC PGM=SORT //* //SORTIN DD DSN=&SYSUID..&TY.&TID..&L.S&OPID, // DISP=&OLDDEL //* //SORTOUT DD DSN=&OTEXT(T&TID.S&P.&OPID), // DISP=SHR //* //SYSIN DD DSN=&DATA(SORTCOPY), // DISP=SHR //* //SYSOUT DD &OUTSORT //* // ELSE //* //IF041E ENDIF //IF040E ENDIF //* //IFRUNE ENDIF - - - - - - - - - - - - - - - 335 Line(s) not Displayed //* //IFPLIE ENDIF //* //RUN2PDS PEND |
As you can see, the number of SET VAR=value statements is pretty large, and even more (my default libraries) are included in the "INCLUDE MEMBER=IMOI" member. I could obviously put all of them into a text member that is read by the replacement REXX exec, but that would mean that I need to update two datasets if I ever want to port these jobs to another system. I have thought about a dirty trick, adding a
| Code: |
//DUMMY DSN=&CNTL(this-member),DISP=SHR |
and read that in inside the replacement REXX exec, parsing out the relevant SET statements.
Any other thoughs on this?
I'm also curious as to the efficiency of running this as a straight BATCH job (as it is at the moment), or as a job that runs ISPF in batch - at one site another smart-ass programmer thought using an edit-macro in batch was an ideal way to massage some output, but after just one run in production that approach was abandoned do to an excessive CPU usage. Here the REXX exec would only be allocating files (rather a lot...), running the program to be tested, ISRSUPC and SORT, be it under ISPF...
And as a PS, the current JCL member is submitted by an edit macro that updates the three 'XXX' strings to 001..178, the 178 input files and with a slight delay (0.5 sec) between the four jobs that make up one test, to avoid starting a later one before an earlier one.
_________________
Robert AH Prins
robert.ah.prins @ the.17+Gb.Google thingy
No programming here (yet)
