This is a very important chapter. Take the time to carefully read and assimilate it.
This chapter is common to both the Advanced REXX Telecourse and the CMS Pipelines Telecourse. If you don't know anything about CMS Pipelines yet, then you can ignore the paragraphs dealing with CMS Pipelines.
It is obviously important that tools used by the VM community perform well, are reliable and secure. Too often we see REXX procedures without an address command statement, or pipelines using the CMS device driver stage. In both cases, performance is wasted and the procedures are subject to breaking in an unpredictable manner.
In this article, we would like to explain the consequences of not using the address command or using then CMS device driver stage instead of the COMMAND device driver stage.
In order to fully understand all the implications, it is mandatory that we first review how CMS resolves commands, the so-called CMS Search Order.
CMS commands (procedures or object modules) can be found on CMS minidisks, or in storage. Let's first review how commands, and REXX procedures in particular, can reside in storage.
An exec can be loaded in the users' private storage by issuing the command
EXECLOAD fn [ft [fm [execname [exectype]]]]
The advantage is then that the file does not need to be loaded from disk each time the procedure is invoked, thus resulting in reduced disk I/O. A logical consequence is that CMS will always use the copy in storage (if it exists) instead of the one residing on a minidisk.
Since CMS Release 5, the so-called installation segment (alias the INSTSEG), permits the systems programmer to load frequently used procedures in a shared segment. The default name of the shared segment is CMSINST. Typically, you'll find procedures like FILELIST and RDRLIST in the segment, together with their accompanying XEDIT macros.
In this case, not only the load from disk is avoided, but all users on the system use the same copy in storage, because it is a shared segment, resulting both in less disk I/O and less paging.
But, in order to allow a user to execute a private (maybe enhanced) version of a procedure from its own minidisks or from its private storage (EXECLOADed procedures) this procedure should take precedence over the version residing in the INSTSEG segment.
A filemode is therefore associated with the INSTSEG segment, to let it be part of the search order based on the filemode. The default filemode is S, which can be changed via the SET INSTSEG ON mode command.
So, with the INSTSEG segment defaulting to filemode S, CMS uses the following search order when looking for an EXEC:
CMS Release 6 introduced the logical shared segments. These shared segments can contain modules, EXECs, text decks, or any other associated object, such as data files. The user can get access to the contents of the shared segment by issuing the SEGMENT LOAD segname command.
Once this command is issued, MODULEs contained in the segment are considered to be NUCXLOADed. For procedures, (EXEC, XEDIT, etc...), it further depends on what is specified when the segment is built. One can choose to make the exec part of the INSTSEG, or to consider it as simply EXECLOADed. Of course, this choice influences the search order as we have just seen.
Thus, when you enter a command in the CMS environment, CMS uses the following search order to locate the command for execution :
If the command is not known to CMS -that is, all of the above fails - it
is passed to CP, provided IMPCP has not been
set OFF.
If CP does not recognize the command either, you get the
typical Unknown CP/CMS command
, and the return code
is set to -3.
For example, if you enter the command : x sauces cookbook, CMS would search as follows :
For more information on the CMS command search order, see the VM/ESA: CMS Command Reference. For additional information on searching for a translation or synonym, see the SET TRANSLATE or SYNONYM command in the VM/ESA: CMS Application Development Guide.
As you can see from the above, the search process can be quite long and elaborated. In your daily work you probably don't realize what really happens in your system.
However, if you are responsible for developing programs or procedures that will be used by a large number of users, you must be concerned by aspects such as performance, reliability and security.
This is where the address statement of REXX and the CMS and COMMAND device drivers of CMS Pipelines come into play. We'll thereby have to make a distinction between different environments where procedures are used.
For example, it makes a difference if a procedure is called from the CMS environment. By this we mean that the command is addressed to CMS directly. Most of the time, it will be from the Ready; prompt, but it can also be from the XEDIT command line, provided the command is prefixed with the keyword CMS. In both cases, we issue a call for a CMS command, or a CMS EXEC procedure.
If, from the XEDIT command line, we call a command without the CMS prefix, the we probably call for an XEDIT subcommand or XEDIT macro, one with a filetype XEDIT.
As just stated, these procedures are executed from the CMS environment. Their filetype is EXEC. Let's discuss now how commands inside those procedures are handled and searched for.
If no address statement is coded in these REXX procedures, the default addressing is address CMS. This means that, all CP or CMS commands encountered in the procedure, are passed to CMS as if they were entered directly at the terminal (or from the CMS environment).
The consequences are as follows :
| If CMS does not recognize the command, it is passed to CP, which may also not recognize the command. In that case the procedure gets the -3 return code. For this reason, REXX procedures should include a trace N (with N for Negative return codes), either than trace Off. |
From the above, we can conclude 2 extremely important facts :
Conclusion: Your procedure is not foolproof. |
For example, you want to execute a RENAME command in your procedure, but somehow you have access to a RENAME EXEC which does an ERASE... Result : your procedure will execute the RENAME procedure, actually resulting in an ERASE !
This looks silly as example, but is perfectly feasible. In fact, if you want to create a virus on your VM system, this is one of the possibilities you have... Be prepared to have excellent backups !
System personnel frequently hear users complaining, saying things like
my procedure worked OK yesterday, but now it doesn't work anymore, what have YOU changed to the system ?
And the answer is, yesterday the user had not accessed your tools disk containing the RENAME EXEC, whereas today he got the access...
Yet another problem that you may encounter is, for example, that you can no longer use the CP LINK command with a password in the procedure. This happens if, at system generation, an option was set by which passwords are refused as parameter to the command.
| Why is it that you can't solve this problem by stacking the password ? |
If you use the statement address command in your REXX procedure, then things change completely :
Hence, with the address command you loose something (stricter coding rules) but you gain a lot :
File not found), but will return an appropriate return code instead. The use of set cmstype ht/rt is no longer required.
One last remark here. Only very recently we discovered that a user-defined synonym can still interfere with our procedure. Indeed, if you look back at the CMS search order, you'll see that CMS also looks for synonyms or translated commands in step four.
For example, suppose the user has defined LIST to be a synonym of FLIST, with possible abbreviation to one character, while on the other hand, we in our procedure have coded something like
address command 'L * * F (STACK'
then, because of the CMS search order, the users' synonym gets executed and thus results in the FLIST command that does not understand the STACK option.
If you want to be absolutely certain that the LISTFILE command gets executed (and thus that synonyms are bypassed), then you have to code the CMS commands without abbreviation, as follows:
address command 'LISTFILE * * F (STACK' /* instead of 'L * * F (STACK' */ 'QUERY DISK (STACK' /* instead of 'Q DISK (STACK' */
Now, even if the user defines LISTFILE as to be a synonym of FLIST, the real CMS LISTFILE command will be executed, as step three will result in an immediate match, and synonyms are ignored. This is not fully explained in the chapter about CMS search order in the manuals.
To make your procedures better performing and safer, code an
address command at the beginning of it.
Then :
Note that coding address ' ' is absolutely equivalent to address command, it's just a useful short notation. The only instance where address CMS is justified is when your procedure accepts commands from the user and has to execute it. Most users do not always make the distinction between the different environments. |
The default environment for an XEDIT macro (filetype XEDIT) is XEDIT itself. It means that an implicit address XEDIT is used.
This is good, but sometimes you may want to execute a CMS or CP command in an XEDIT macro. Then the rules described above still apply, but there is an extra player in the game - XEDIT itself.
If you enter a command on XEDIT's command line (or code it in an XEDIT macro), without being explicit about the environment you want to address, then XEDIT will take this command for itself first. If XEDIT does not understand the command, then it passes it along to CMS - this is only true if IMPCMSCP (Implied CMS and CP) is set ON (which is the default, but the user may have changed it). From there on, CMS treats the command as with an address CMS, hence not foolproof any more. If it's not recognized by CMS, it will pass it to the CP level (depending on the IMPCP setting).
Now, let's take the example of the SET command. XEDIT, CMS and CP all have some form of SET command. So, if we aren't explicit in our command, then XEDIT will take it for itself. If the parameters are not recognized by XEDIT (because we actually wanted to execute a CMS or CP SET command), then XEDIT will return a bad return code.
So, on the XEDIT command line, we all have learned to use the CP or CMS prefixes in order to address the correct environment if the command is not exclusive to a specific environment.
In our XEDIT macros, we can temporarily switch from the implied address XEDIT status to the address COMMAND status. This can be done in 2 ways :
address command 'CP SPOOL CONSOLE START' 'ERASE TEMP FILE A' address 'XEDIT'
address command 'CP SPOOL CONSOLE START'
While we're on the subject, remember that XEDIT also makes a distinction between basic commands and macros. Furthermore, XEDIT also allows defining synonyms for its commands. So, we can have similar problems in XEDIT - executing macros when we want a basic command - as we had with CMS. This time however, we can control the execution of the commands or macros by explicitly using either the COMMAND or MACRO prefix in our statements. Note that the COMMAND prefix here has nothing to do with our address command, it's entirely an XEDIT built-in command. For example:
'COMMAND DELETE 2' 'MACRO ALL /this/ ! /that/'
An alternative to this, although not so easy to control and to code, is to use the SET MACRO ON/OFF and SET SYNONYM ON/OFF commands to control the execution of macros, synonyms or commands. Also, don't forget to restore the settings before returning to the user.
For XEDIT macros :
|
One further remark here, again in favour of the address command, is that XEDIT truncates all commands to 255 characters. With an address command, REXX does not send the commands to XEDIT, and thus, truncation does not occur. CMS Pipelines, for example, can become very large strings.
More and more products include an interface to REXX, allowing procedures to use the functions of the products. Examples are ISPF, SQL, GDDM, CPI-C, and many others (also on other platforms like OS/2).
In most cases, these products define themselves as sub-environments (or to use the exact term, sub-commands or SUBCOMs) to CMS. So, just as for XEDIT, they define their proper environment that can be 'reached' by REXX via the address statement.
So for example, to address ISPF, you would use an address ISPEXEC statement, while CPI-C would be reached via an address CPICOMM statement.
Most of what we've said above applies also somehow to CMS Pipelines. First of all, PIPE is a regular CMS command, so use address command whenever it is appropriate.
CMS Pipelines provides the CMS, COMMAND and CP device drivers to execute CMS or CP commands.
We will discuss these stages in more detail later. Just remember now that these drivers execute the commands and, instead of displaying the result to the terminal, pump the results into the pipeline.
So, what's the magic ? For CMS commands, the CMS or COMMAND stages intercept the CMS terminal output before it reaches the screen and pump the lines into the pipeline. For CP commands, the CP device driver issues a regular diagnose x'8', but asks CP to return the results in a buffer instead of displaying it at the terminal (just like REXX does it via the diag() function). |
As both REXX and CMS Pipelines use the same internal interface to pass commands to CMS, it is not a surprise that the CMS device driver has the same drawbacks as REXX's address CMS, and that the COMMAND device driver behaves as with an address COMMAND in REXX.
It's a shame that both REXX and CMS Pipelines manuals are still primarily showing address CMS and CMS stages. We presume this is because they want to open to the broadest public who may not know the difference between CP and CMS commands or procedures.
So, let's repeat once more the advantages of the COMMAND driver :
This last item is much more important for CMS Pipelines than it is for REXX procedures. These drivers pump the output of the commands into the pipeline, and so will the error messages if you don't choose the right option. But more on this later.
|
A typical pipeline using COMMAND looks like this :
Coding pipelines in REXX procedures. | |
---|---|
/* Coding pipelines */ address command /* hope you're convinced now */ 'PIPE' , /* need uppercase due to address command */ '!command LISTFILE * * A', /* pipeline parameters can be lowercase, */ /* but parameters of COMMAND stage must */ /* be coded in uppercase again... */ '!specs w2 1' , /* pipe parameter, thus can be lowercase */ '!console' , /* pipe parameter, thus can be lowercase */ exit rc |
The example shows that you can create a mess. We personally prefer to follow our coding style that says
code all commands and their parameters with upper case characters (unless lower case is important), and code all REXX statements and variables with mixed case characters.
In the previous example we would therefore code the whole pipeline in uppercase.
But, on the other hand, pipelines can become rather long and complex, and then use of mixed case can enhance readability. It's not possible to give an absolute rule, and it can be a matter of taste.
We have one last problem to solve. We said that due to the COMMAND stage, many CMS commands will not produce error messages. But there may be cases where we would like to get these error messages in the pipeline anyway, to display them at the terminal to inform the user.
Do we have to revert to the CMS stage to get the error message back ?
No, there is another useful CMS command - namely CMDCALL. This command takes other CMS commands as parameters and lets CMS execute these commands as if they came from the terminal (and thus error messages and headers are produced as normal).
If we combine the COMMAND stage (or address command as a matter of fact), with this CMDCALL command, as follows:
pipe command CMDCALL LISTDIR ! > LISTDIR OUTPUT A
then :
However, in this case, the output is pumped into the next pipeline stage, from where you can further process it. For example:
'PIPE COMMAND CMDCALL LISTDIR' somedir, '!VAR EMSG', /* any error message ? */ '!DROP', /* drop the header */ '!STEM DIR.' /* save result... */ if rc<>0 then do say 'Unable to list directory' somedir say 'Error message is:' say emsg exit rc end do i=1 to dir.0 .... end
Pipe stages written in REXX have a default addressing so that PIPE subcommands can be recognized (e.g. PEEKTO, OUTPUT, etc.).
An illustration of everything we have said in this article will probably further help you understand the different options. The procedure below will accept any command as parameter and execute it once for all of the following cases:
This procedure might also be useful in order to analyse the returned information (headers, error messages, etc.):
Analyze results of commands - TCMD EXEC © | |
---|---|
/* TCMD EXEC - Analyze results of commands */ parse upper arg cmd Say '*** Case 1 : Result of "address CMS' cmd'" ***' address CMS cmd Say 'Retcode:' rc Say '*** Case 2 : Result of "address COMMAND' cmd'" ***' address COMMAND cmd Say 'Retcode:' rc Say '*** Case 3 : Result of "address COMMAND CMDCALL' cmd'" ***' address COMMAND 'CMDCALL' cmd Say 'Retcode:' rc address command Say '*** Case 4 : Result of "PIPE CMS' cmd'!CONSOLE" ***' 'PIPE CMS' cmd'!CONSOLE' Say 'Retcode:' rc Say '*** Case 5 : Result of "PIPE COMMAND' cmd'!CONSOLE" ***' 'PIPE COMMAND' cmd'!CONSOLE' Say 'Retcode:' rc Say '*** Case 6 : Result of "PIPE COMMAND CMDCALL' cmd'!CONSOLE" ***' 'PIPE COMMAND CMDCALL' cmd'!CONSOLE' Say 'Retcode:' rc |
A sample output can be seen below :
> tcmd erase no file *** Case 1 : Result of "address CMS ERASE NO FILE" *** DMSERS002E File NO FILE not found Retcode: 28 *** Case 2 : Result of "address COMMAND ERASE NO FILE" *** Retcode: 28 *** Case 3 : Result of "address COMMAND CMDCALL ERASE NO FILE" *** DMSERS002E File NO FILE not found Retcode: 28 *** Case 4 : Result of "PIPE CMS ERASE NO FILE!CONSOLE" *** DMSERS002E File NO FILE not found Retcode: 28 *** Case 5 : Result of "PIPE COMMAND ERASE NO FILE!CONSOLE" *** Retcode: 28 *** Case 6 : Result of "PIPE COMMAND CMDCALL ERASE NO FILE!CONSOLE" *** DMSERS002E File NO FILE not found Retcode: 28 Ready;
Only cases one, three, four and six display the error messages. In cases one and four, an ERASE EXEC could break your procedure.
> tcmd state no file w *** Case 1 : Result of "address CMS STATE NO FILE W" *** DMSSTT069E Filemode W not accessed Retcode: 36 *** Case 2 : Result of "address COMMAND STATE NO FILE W" *** Retcode: 36 *** Case 3 : Result of "address COMMAND CMDCALL STATE NO FILE W" *** DMSSTT069E Filemode W not accessed Retcode: 36 *** Case 4 : Result of "PIPE CMS STATE NO FILE W!CONSOLE" *** DMSSTT069E Filemode W not accessed Retcode: 36 *** Case 5 : Result of "PIPE COMMAND STATE NO FILE W!CONSOLE" *** Retcode: 36 *** Case 6 : Result of "PIPE COMMAND CMDCALL STATE NO FILE W!CONSOLE" *** DMSSTT069E Filemode W not accessed Retcode: 36 Ready;
The same remarks apply here as in the first run.
> tcmd Listfile profile exec *** Case 1 : Result of "address CMS LISTFILE PROFILE EXEC" *** PROFILE EXEC A0 Retcode: 0 *** Case 2 : Result of "address COMMAND LISTFILE PROFILE EXEC" *** PROFILE EXEC A0 Retcode: 0 *** Case 3 : Result of "address COMMAND CMDCALL LISTFILE PROFILE EXEC" *** PROFILE EXEC A0 Retcode: 0 *** Case 4 : Result of "PIPE CMS LISTFILE PROFILE EXEC!CONSOLE" *** PROFILE EXEC A0 Retcode: 0 *** Case 5 : Result of "PIPE COMMAND LISTFILE PROFILE EXEC!CONSOLE" *** PROFILE EXEC A0 Retcode: 0 *** Case 6 : Result of "PIPE COMMAND CMDCALL LISTFILE PROFILE EXEC!CONSOLE" *** PROFILE EXEC A0 Retcode: 0 Ready;
The all produce the same result, but we were lucky that there wasn't a Listfile EXEC procedure...
> tcmd access .telecour t *** Case 1 : Result of "address CMS ACCESS .TELECOUR T" *** Retcode: 0 *** Case 2 : Result of "address COMMAND ACCESS .TELECOUR T" *** Retcode: 0 *** Case 3 : Result of "address COMMAND CMDCALL ACCESS .TELECOUR T" *** Retcode: 0 *** Case 4 : Result of "PIPE CMS ACCESS .TELECOUR T!CONSOLE" *** Retcode: 0 *** Case 5 : Result of "PIPE COMMAND ACCESS .TELECOUR T!CONSOLE" *** Retcode: 0 *** Case 6 : Result of "PIPE COMMAND CMDCALL ACCESS .TELECOUR T!CONSOLE" *** Retcode: 0 Ready;
This gives the same output in all cases, but, if we issue the same command with an invalid directory name, then we get:
> tcmd listdir .wrongdir *** Case 1 : Result of "address CMS LISTDIR .WRONGDIR" *** DMSJLD1184E Directory IECSYSU:MAINT.WRONGDIR not found or you are not authorized for it Retcode: 28 *** Case 2 : Result of "address COMMAND LISTDIR .WRONGDIR" *** Retcode: 28 *** Case 3 : Result of "address COMMAND CMDCALL LISTDIR .WRONGDIR" *** DMSJLD1184E Directory IECSYSU:MAINT.WRONGDIR not found or you are not authorized for it Retcode: 28 *** Case 4 : Result of "PIPE CMS LISTDIR .WRONGDIR!CONSOLE" *** DMSJLD1184E Directory IECSYSU:MAINT.WRONGDIR not found or you are not authorized for it Retcode: 28 *** Case 5 : Result of "PIPE COMMAND LISTDIR .WRONGDIR!CONSOLE" *** Retcode: 28 *** Case 6 : Result of "PIPE COMMAND CMDCALL LISTDIR .WRONGDIR!CONSOLE" *** DMSJLD1184E Directory IECSYSU:MAINT.WRONGDIR not found or you are not authorized for it Retcode: 28 Ready;
Having said all this, we now have a clear conscience !
This concludes Lesson 1. You are now ready to proceed with the coding exercises for this lesson.
Footnotes:
(1)
The question remains, why would you define a password if you hard-code
it in readable format in your procedure anyway ? Even compiling
your EXEC will not completely hide the password.
Back to text