2. Dataset Pipes Installation and Configuration

2.1 Dataset Pipes Installation by Use Case

The installation steps vary depending on the usage of the Dataset Pipes commands.

  • z/OS Unix System Services Integration:

    Follow the instructions for Co:Z Toolkit for z/OS. Because the Dataset Pipes commands are only running on z/OS, no additional configuration is needed. There is no specific Dataset Pipes configuration.

  • z/OS Hybrid Batch:

    When Dataset Pipes commands are run with the Co:Z Launcher in a z/OS Hybrid Batch use case, installation is required for the Co:Z Toolkit for z/OS and the Co:Z Target System Toolkits. No specific Dataset Pipes configuration is required, but see the Co:Z Launcher User's Guide for additional configuration options.

  • z/OS Remote Services:

    In addition to the Co:Z Toolkit for z/OS and Co:Z Target System Toolkits, the dspipes subsystem and dspipes session config options must also be configured. See the following sections for more detail.

2.2 z/OS Remote Services Quick Start

After completing the installation of the Co:Z Toolkit for z/OS and the Co:Z Target System Toolkit on the remote system, the following are the minimum steps to get started using z/OS Remote Services. Replace <COZ_INST> with the installation directory of the toolkit. For more detailed information, see the remaining sections in this chapter.

On z/OS:

  1. Edit /etc/ssh/sshd_config and add the following line along with other subsystem configuration.
    Subsystem dspipes <COZ_INST>/bin/dspipes.sh
    Restart SSHD.
    kill -HUP `cat /var/run/sshd.pid`
  2. Copy the sample configuration files to /etc/ssh:
    cp <COZ_INST>/samples/dspipes.site.rc /etc/ssh/dspipes.rc
    chmod 755 /etc/ssh/dspipes.rc  
    cp <COZ_INST>/samples/dspipes_site_config /etc/ssh/dspipes_config
    chmod 0644 /etc/ssh/dspipes_config 
  3. Edit /etc/ssh/dspipes_config. Uncomment and set server-host to an externally recognized IP address or hostname for the z/OS server. Uncomment and set server-ports, if the default is not applicable.

On the remote system:

Assuming the Co:Z Target System Toolkit has been installed on a unix server with the PATH configured to the Dataset Pipes command executables:

  1. Verify that the remote system can connect to the z/OS server using SSH.
    ssh user@host uname -a
  2. Interactively test a Dataset Pipes command with a single SSH connection. This command lists the files in the user's z/OS home directory.
    cozclient -ssh user@host ls -alt
  3. Interactively start a durable connection using cozcontrol, execute a few Dataset Pipes commands on z/OS, and end the session.
    cozcontrol start -ssh user@host
    cozclient ls -alt 
    fromdsn 'SYS1.MACLIB(ACB)' > /tmp/test.txt
    todsn 'user.test.dataset' < /tmp/test.txt 
    cozcontrol stop

For more examples, see Appendix A, Command Reference - Dataset Pipes and Chapter 4, z/OS Remote Services Examples.

2.3 z/OS Remote Services Configuration

This section provides additional detail about the steps summarized in the quick start section above.

Configuring the Dataset Pipes subsystem

To run Dataset Pipes commands initiated by a remote client, a subsystem must be configured in your z/OS OpenSSH server.[1]

This is done by updating the sshd_config file, typically located at /etc/ssh/sshd_config.[2]

Find the line "Subsystem" which defines the sftp subsystem. Immediately following the sftp line add this:

Subsystem dspipes <COZ_INST>/bin/dspipes.sh

(where <COZ_INST> is the directory where Co:Z Toolkit is installed).

If OpenSSH sshd was running prior to editing sshd_config, it should be reinitialized. This can be done by sending SIGHUP to the running process. The pid for this process is typically in the file /var/run/sshd.pid:

kill -HUP `cat /var/run/sshd.pid`

Dataset Pipes Configuration overview

The following table describes how a Co:Z Dataset Pipes Server session is started and outlines the sequence of configuration steps that occur prior to the establishment of the session. Details on these configuration steps follow the table.

Table 2.1. Dspipes Server initialization steps

StepConfigurationNotes
1$COZ_HOME/bin/dspipes.shThis shell script is executed by z/OS OpenSSH sshd upon a request for an Dataset Pipes server subsystem. This file should not be modified by the installation, but you may want to review the comments at the beginning of the script. This script will execute the site-wide and user-specific rc scripts and configuration files (see following steps).
2/etc/ssh/dspipes.rcSite-wide environment variable configuration.
3$HOME/.ssh/dspipes.rcUser specific environment variable configuration. Can contain customized log file location, logging and tracing options, etc. The location of this file may be changed by setting the $DSPIPES_USER_RC environment variable.
4$HOME/.ssh/dspipes_configUser-specific configuration settings. User customized options may be specified here. The location of this file may be changed by setting the $DSPIPES_USER_CONFIG environment variable.
5/etc/ssh/dspipes_configSite-wide configuration settings.

Sitewide Dataset Pipes configuration

The Dataset Pipes server can be configured with system-wide defaults by creating and configuring the file /etc/ssh/dspipes.rc. A sample file (dspipes.site.rc) is provided in <COZ_INST>/samples, and should be copied to the /etc/ssh directory:

cp <COZ_INST>/samples/dspipes.site.rc /etc/ssh/dspipes.rc
chmod 755 /etc/ssh/dspipes.rc

Sample site dspipes.rc file

#!/bin/sh
# Set site-wide environment variables for dspipes server.
# Place this sample as an executable script in file: /etc/ssh/dspipes.rc

# The following are the default locations for user level configuration files.
#DSPIPES_USER_RC=$HOME/.ssh/dspipes.rc 1
#DSPIPES_USER_CONFIG=$HOME/.ssh/dspipes_config 2

# The following defines a directory name (without trailing slash) where
# log files will be created, rather than /tmp or $TMPDIR. Setting this 
# variable and regular cleanup of this directory are recommended.
#export DSPIPES_LOGDIR= 
1

In some cases, users may not have access to individual $HOME directories or it may be desirable to have all user configuration files centralized. In this case, the environment variable DSPIPES_USER_RC can be specified to provide an alternate file name for the user .rc file in a common, readable location. For example, to specify a common directory for all user configuration files, set the following: DSPIPES_USER_RC=/usr/share/coz/$LOWER_LOGNAME.dspipes.rc

To disable the usage of user specific dspipes.rc files for all users, DSPIPES_USER_RC can be set to a dummy file name (e.g: /dummy).

Note that the z/OS Unix System Services $LOGNAME environment variable holding the current username is in uppercase. As this is not always consistent with other POSIX style usage, the dspipes.sh script exports an enviroment variable named $LOWER_LOGNAME that downcases the value in $LOGNAME.

2

Additionally, individual user server config files can be similarly located. To learn more about config files, refer to section: Section 2.4, “z/OS Remote Services Session Config Files”. By default, user server config files are located at $HOME/.ssh/dspipes_config.

Note: The /etc/ssh/dspipes.rc, if present, must be marked executable, as must the individual user files.

User specific Dataset Pipes customization

Individual users can override system wide Dataset Pipes properties by creating a profile script, dspipes.rc, in their home .ssh directory:

# if the user's .ssh does not exist:
mkdir $HOME/.ssh
chmod 700 $HOME/.ssh

cp <COZ_INST>/samples/dspipes.user.rc $HOME/.ssh/dspipes.rc
chmod u+x $HOME/.ssh/dspipes.rc

Sample user dspipes.rc file

#!/bin/sh
# Place this sample as an executable script in file: $HOME/.ssh/dspipes.rc

# Product support personnel may request that you uncomment one or more of 
# the following variables to enable tracing. These options should be set
# in user specific dspipes.rc files rather than globally in the site-wide 
# dspipes.rc 

# Setting COZ_LOG enables tracing for CozServer session level tracing. 
# The default is N, Notice.
#export COZ_LOG=T 1

# Setting COZ_LOG_CMD enables tracing for dspipes commands running on the 
# server (fromdsn, cozclient, etc). The default is N, Notice. Command 
# tracing can alternately be enabled with the -L option on most dataset
# pipes commands.  
#export COZ_LOG_CMD=F 2

# Setting COZ_LOG_CMD_DUP to true (default is false), duplicates tracing enabled by 
# COZ_LOG_CMD to the session log. This is recommended when requesting support from
# Co:Z support personnel because all logging for a problem is captured in a single 
# file.  
#export COZ_LOG_CMD_DUP=true 3
1

When logging is enabled by setting this variable, log files are created for every Dataset Pipes server session. Each session is an SSH session. See the section called “Logging Configuration for Dataset Pipes” for additional information.

2

When command logging is enabled by setting this variable, Co:Z logging messages are redirected back to the client for all Dataset Pipes commands executed by the client. The optional -L switch can be specified on Dataset Pipes commands to enable logging for a single command.

3

When set to true, command logging is redirected back to the client and duplicated to the session log.

Logging Configuration for Dataset Pipes

Log files are created for every Dataset Pipes server session. When specifying -ssh on an individual Dataset Pipes command such as fromdsn, the log file contains logging only for the single command. When using cozcontrol to define a durable session used by multiple Dataset Pipes commands, the log file contains logging for all commands using the durable session. If the log file remains empty due to the logging level set, the file is deleted when the session ends. The log file is of particular interest when a problem is encountered and additional error detail is needed. The following sections define how to control the logging destination as well as logging levels.

Logging Destination

By default, log files are written to the /tmp directory (or the directory specified by the TMPDIR environment variable, if it is set). To change this directory default for all users, modify /etc/ssh/dspipes.rc as needed. Exporting DSPIPES_LOGDIR changes the directory log files are written to. The session log file name is generated with the following pattern: dspipes.<userid>.<...>.log.

The directory containing the log files must be cleaned up and monitored for space; however, it is important to keep these files for some period of time in order to allow support personnel to review the session log file for diagnostic information when investigating a problem.

In order to assist with log file maintenance, the environment variable DSPIPES_LOG_KEEP_DAYS can be exported from either the site or individual user's dspipes.rc. When this variable is specified, the argument must be an integer greater than zero. Log files older than the argument will be removed when the user next establishes a Co:Z Dataset Pipes server session. The log files to be deleted must belong to the the connecting user and reside in the immediate directory specified by DSPIPES_LOGDIR (or $TMPDIR if DSPIPES_LOGDIR is not specified). Note that if DSPIPES_LOGFILE is specified, DSPIPESP_LOG_KEEP_DAYS (if set) will be ignored.

Logging Level

The logging level is controlled by exporting the following environment variables: COZ_LOG, COZ_LOG_CMD, and COZ_LOG_CMD_DUP. In order to diagnose a problem for an individual user, create a dspipes.rc file in their individual .ssh directory, setting these variables as directed by product support personnel.

  • COZ_LOG

    When set, enables Dataset Pipes server level logging, excluding logging of the server side execution of Dataset Pipes commands (fromdsn, todsn, cozclient, etc.).

    When the COZ_LOG environment variable is not specifically set, the default is N which logs error, warning and notice messages. The Co:Z support team may direct setting this variable to one of E, W, N, I, D, T or F for Error, Warning, Notice, Informational, Debug, Trace, or Fine level tracing.

  • COZ_LOG_CMD

    When set, enables logging of the server side execution of Dataset Pipes commands (fromdsn, todsn, cozclient, etc.). Logging for Dataset Pipes commands is redirected back to the client as stderr. Alternatively, logging can be configured for a specific Dataset Pipes command by specifying the -L option on the command. See Appendix A, Command Reference - Dataset Pipes for additional information.

    When the COZ_LOG_CMD environment variable is not specifically set, the default is N which logs error, warning and notice messages. The Co:Z support team may direct setting this variable to one of E, W, N, I, D, T or F for Error, Warning, Notice, Informational, Debug, Trace, or Fine level tracing.

  • COZ_LOG_CMD_DUP

    The default for this variable is false. Setting this variable to true duplicates Dataset Pipes command stderr to the client and the server session log. Setting this option to true in a user's dspipes.rc file captures the most complete set of information in one server session log file.

2.4 z/OS Remote Services Session Config Files

The file /etc/ssh/dspipes_config can be used to customize the options available for a cozcontrol durable session. The permissions for this file should be 0644. While user's can override these properties, they are intended to be site-wide properties.

A sample file (dspipes_site_config) is provided in <COZ_INST>/samples, and should be copied to the /etc/ssh directory:

cp <COZ_INST>/samples/dspipes_site_config /etc/ssh/dspipes_config
chmod 644 /etc/ssh/dspipes_config

Table 2.2. Durable session Dataset Pipes server properties

PropertyDescription
server-portsDefault: 8050-8058. Identifies the ports the server can use to set up its socket listener. This property is not used when tunneling is configured on cozcontrol start.
server-hostDefault: gethostname(). The external address of the Dataset Pipes server running on z/OS. Used by the cozcontrol and client Dataset pipes commands to send messages to the socket listener established when the Dataset Pipes server is started. This option is not used for tunneled connections.
inactivity-interval-minutesDefault: 15 minutes. The Dataset Pipes server will shutdown its socket server and end the SSH connection when the inactivity-interval-minutes expires due to no Dataset Pipes command activity during the durable session.



[1] SSH user subsystems are, like all SSH remote commands, executed in a process under the authenticated client userid, so normal z/OS user security determines what resources can be accessed.

[2] It is sometimes convenient to set up a test OpenSSH server where this subsystem can be easily added. Instructions for doing this can be found in the Co:Z Installation and Release Notes.


Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.

Saint Charles, Missouri
info@coztoolkit.com
+1 636.300.0901

Copyright© 2009 - 2023 Dovetailed Technologies, LLC. All rights reserved. Co:Z® is a registered trademark and Co:Z Toolkit™ is a trademark of Dovetailed Technologies, LLC.