Lock Processing

There are several ways to browse and edit Enterprise Server Cluster locks.

When a job is submitted, it gets placed into the Input queue. If an initiator is available to run the JCL for the specified class, the job is moved to the Active queue at which point the job should execute, unless the job requires locks on files that are already locked. If this is the case, the JCL is placed into a Waiting mode.

You can browse/display the locks in the GLM and the cluster clients. When an Enterprise Server Cluster is active you may browse all global locks held on the Global Lock Manager (GLM), and you can also identify locks for a specific JCL, initiator or client. On a cluster client you can browse all the global and its own local locks. You can do this using ESMAC or via the caslock command.

Browsing active locks with ESMAC

Both GLM and Enterprise Server Cluster clients have tools to allow you to browse active locks. On the GLM, all cluster-wide global locks can be browsed. A cluster client can only display its own global locks and all of its local locks.

Browsing locks on a GLM

On the GLM, you can view all global locks and their pertinent information. A lock (ENQ) is defined by the target resources detail as well as the chain of its lock owners and the characteristics of the lock(s) they have requested.

A resource is defined by a QNAME and an RNAME:

QNAME: The QNAME defines the type of target resource for an ENQ or a DEQ. The only QNAMEs currently supported by an Enterprise Server Cluster are SYSZJOBD, SYSZJOBN and SYSZDSN.
RNAME: The RNAME defines the target resource name for an ENQ or DEQ.

The lock owner chain contains information such as:

Lock type - exclusive or shared.
Lock owner status - controlled, waiting or pending.

To view the locks on a GLM, visit the corresponding ESMAC page. In the Resources pane, select Active, then select Locks (CASRDO33). The display will resemble the following:

Screenshot of an active Locks screen in ESMAC, showing all locks

PID: PID of the GLM's communication server (MFCS)
res-ID: Identifies the resource ID
Type: Type of lock (shared or exclusive)
State: Lock state - Controlled, Waiting, Pending

The above screenshot illustrates a situation where job JOB01364 (jobname MYLOCK) running in a batch initiator (PID 8716) in Enterprise Server Cluster client ESCLSLV1 owns locks on several datasets, including a dataset identified by res-ID 1. In the Resource Details group, res-ID 1 is defined as belonging to Queue SYSZDSN with Resource Name SYSI.CLUSTER.MYFILE.

At the same time, job number JOB01365 (for which the job name is not yet known) running in a batch initiator (PID 12228) in Enterprise Server Cluster client ESCLSLV1 is in a wait state as it, too, requires an exclusive lock on the same resource: res-ID 1, SYSI.CLUSTER.MYFILE.

Note: In the Resource Details group, the scope is not displayed. This is because that on the GLM, all cluster-wide locks are GLM local locks.

Browsing locks on an Enterprise Server Cluster client: To view the cluster client locks, visit the corresponding ESMAC page. In the Resources pane, select Active, then select Locks (CASRDO33). The display will resemble the following, which is similar to the preceding screenshot, except that only the locks owned by this client are displayed:

Viewing active locks with the `caslock` command

The caslock command is self-documented - simply enter caslock -h at the Enterprise Server command prompt and the following information is displayed:

Usage: caslock [/a /l /r /u /p /c]

   /a[client_applid]        Remove locks for specified APPLID of the client region
   /l                       List all active locks
   /r[region_name]         	The name of the region to send the request to
   /u[username]             The user's username
   /p[password]             The user's password
   /c[group]                The user's group

The command: caslock -rESCLMGR -l will display all active global locks in the Enterprise Server Cluster:

Lock Ownership      
PID     res-ID    Type        State         Persistence         Seq
8720    t:001    JOB01485 JRX0033  ( MYLOCK   ) (ESCLSLV1/     10392)
           1     Exclusive   Controlled    Task                 1
           2     Exclusive   Controlled    Task                 1
           3     Exclusive   Controlled    Task                 1
           4     Exclusive   Controlled    Task                 1
           5     Exclusive   Controlled    Task                 1
           6     Exclusive   Controlled    Task                 1
           7     Exclusive   Controlled    Task                 1
10660    t:002    ES Subsystem
           8     Exclusive   Controlled    Server               1
Resource Details
      res-ID     Queue        Length    Resource Name
         1      SYSZDSN           19    SYSI.CLUSTER.MYFILE
         2      SYSZDSN           20    SYSI.CLUSTER.SPEC002
         3      SYSZDSN           20    SYSI.CLUSTER.SPEC001
         4      SYSZDSN           20    SYSI.CLUSTER.PATT001
         5      SYSZDSN           20    SYSI.CLUSTER.GENE002
         6      SYSZDSN           20    SYSI.CLUSTER.GENE001
         7      SYSZJOBN           8    JRX0033
         8      SYSZJOBD           8    JRX0034

Note:

You can only run the caslock command against the GLM.
You cannot target a specific APPLID - all cluster-wide global locks are displayed.

Removing global locks

Lock removal functionality is provided to enable troubleshooting in specific circumstances. Global locks should only be removed when a permanent connection failure occurs between an Enterprise Server Cluster client and the Enterprise Server Cluster manager, and locks are held preventing jobs from executing in other cluster clients still active in the cluster.

If a connection between the GLM and one of the Enterprise Server Cluster clients has been lost for longer than the duration defined by the ES_GLM_TIMEOUT environment variable, the cluster client layer will mark the connection to the GLM as disabled. Once the client's connection state is marked as disabled it cannot be recovered. All successive attempts to connect to the cluster will be rejected with the following messages:

TXCS3032S Connection to ES Cluster manager GLM APPLID (GLM SYSID) is disabled, verify and release
 global locks on ES cluster manager

TXCS3033S Attempt to connect to a disabled ES Cluster manager GLM APPLID (GLM SYSID) , verify and 
release global locks on ES cluster manager

The system administrator needs to take action, which must include identifying whether this region holds global locks that will prevent work units running in other active cluster clients from executing successfully. If this is the case, the system administrator needs to use ESMAC on the GLM or the caslock command to remove all the locks held by the failing Enterprise Server Cluster client.

CAUTION:

When locks are being removed, the target Enterprise Server Cluster client is implicitly removed from the cluster. Attempts to execute further JCL on this client could lead to unexpected behavior.

Removing locks with ESMAC: To remove locks with ESMAC, visit the GLM's ESMAC page. In the Resources pane, select Active, then select Locks (CASRDO33).
At the bottom of the page, the Remove Locks button will allow the removal of all the locks for an APPLID. In the client APPLID entry field, enter the APPLID of the Enterprise Server Cluster client you want to remove the locks for.

Lock Processing

Browsing active locks with ESMAC

Viewing active locks with the caslock command

Removing global locks

Viewing active locks with the `caslock` command