Skip to content

Clean Up Production Backup Libraries

Backup libraries are associated with ChangeMan ZMF production libraries. When a package is installed, the current version of a component is copied from the production library to the backup library so it can be restored if package is backed out.

There are three ways to handle cleanup of backup libraries, and thereby manage the growth of components that accumulate in these data sets. By specifying the appropriate option as a PARM parameter on the JCL EXEC statement when running program CMNBATCH, you can do one of the following:

  • Bypass cleanup of backup libraries (the default).

  • Perform package processing.

  • Perform commit processing.

Bypass Cleanup of Backup Libraries

To bypass cleanup of backup libraries, specify the parameter 'BACKUPLIB=N' for program CMNBATCH. This is the default, so specifying the BACKUPLIB parameter is optional. When CMNBATCH runs, the CMN920 transactions delete the staging libraries, but no attempt is made to delete components from the backup libraries:

    //CMNBATCH EXEC PGM=CMNBATCH,`
    // PARM='SUBSYS=?,BACKUPLIB=N'

Perform Package Processing for Backup Library Cleanup

If cleanup processing is required for your site and you plan to process backup libraries by package association, use package processing for cleanup of backup libraries. You can do this by specifying 'BACKUPLIB=P' in program CMNBATCH when executing this program for housekeeping purposes, as in the following JCL:

    //CMNBATCH EXEC PGM=CMNBATCH,
    // PARM='SUBSYS=?,BACKUPLIB=P'

During package processing, CMNBATCH calls the program CMNCLNUP. The component being deleted in the backup library is first compared to its corresponding component in the staging library. If the directory entries for both components match, CMNCLNUP deletes the component from the backup library. Because this only happens to packages that have been ‘memo deleted’ or have ‘aged’, you ensured that a component deleted from the backup library came only from a specific staging library in a specific package. The following discussion describes this type of clean up operation:

  1. Assume that you created a package named PKG$000010 that contains the uniquenamed component 'XYZ0000'. When package PKG$000010 is installed, ChangeMan ZMF attempts to copy component ‘XYZ0000’ from the production library associated with this package to the backup library in the CMN20 job. Because this component is unique and never resided in the production library, the IEBCOPY step that copies the component from the production library to the backup library ends with an RC=0004. At this time, the backup library has no component named ‘XYZ0000’. When CMN20 runs, the component ‘XYZ0000’ is copied from the staging library to the production library. When CMN20 completes successfully and CMN30 executes, the baseline ripple happens.

  2. Assume that you created a package named PKG$000020 that contains component ‘XYZ0000’ in the staging library. You make changes to the component, and schedule the package for installation. When the package is installed, the CMN20 job attempts to copy the component ‘XYZ0000’ from the production library associated with this package to its corresponding backup library. Because a same-named component was previously-placed in the same production library by package PKG$000010, the IEBCOPY step backs up the component by performing a copy from the production library to the backup library. When IEBCOPY ends with an RC=0000, the backup library contains a component named ‘XYZ0000’ from package PKG$000010. In the install step of CMN20, the next component ‘XYZ0000’ from package PKG$000020 is copied from its staging library to the production library. As a result, the component‘XYZ0000’ from package PKG$000020 overlays the same-named component that was there previously from package PKG$000010. When CMN20 completes successfully and CMN30 executes, baseline ripple happens.

  3. Assume that package PKG$000010 in BAS status aged, so that the package is included in the CMN920 report, thereby creating the delete Staging Libraries transaction record for this package. This record becomes part of the SYSIN input for program CMNBATCH. PKG$000020 remains because it was installed at a later time and has not yet aged.

    During housekeeping, the CMN920 report generates a delete staging library transaction record for PKG$000010. When CMNBATCH reads the delete StagingLibraries transaction record, it builds a table from the package master listing all the components in package PKG$000010 that reside in a single staging library. Thereafter, the data set names for the staging library, the production library, and the backup library (associated with package PKG$000010 for one library type) are placed in another table. Then, CMNBATCH calls program CMNCLNUP to pass the component list and library list tables. Program CMNCLNUP then opens (1) the staging data set for package PKG$000010 under the DDname of SYSUT1, and (2) the backup library for package PKG$000010 under the DDname of SYSUT2.

    Next, program CMNCLNUP obtains the component name 'XYZ0000' from the component list and compare the directory entry for component 'XYZ0000' (from the staging library of package PKG$000010) to the directory entry of component 'XYZ0000' (in the backup library). If they match, you have ensured that component 'XYZ0000' in the backup library could only have arisen from package PKG$000010. Component 'XYZ0000' is deleted from the backup library. Alternatively, if the entries do not match, no deletion occurs. This ensures that any component being deleted from the backup library is directly associated only with the specific package.

    Once all components are processed, CMNCLNUP returns to the CMNBATCH program, which builds a component list for the next library type of package PKG$000010, and calls CMNCLNUP again until all library types have been processed. Thereafter, the Staging Libraries are deleted for the package, and the next transaction record is read, and housekeeping resumes.

    Because a component is tied directly to the package that is scheduled for deletion, this method of cleaning up the Backup Libraries is termed ‘package processing’.

    When a site has been running ChangeMan ZMF for an extended period of time and cleanup of Backup Libraries has not been performed, the Backup Libraries might have grown substantially and they are populated with many components from many packages. By running prior housekeeping jobs without ‘clean up,’ many packages and their corresponding Staging Libraries have probably been deleted, leaving many of their associated components in their respective Backup Libraries. This might result in retaining many orphan components (in the backup library) that are not deleted when you run a cleanup. This is because before cleanup processing became available as a new feature, prior housekeeping jobs may have deleted the package records and staging library for an old component in the backup library.

    As future packages are installed with the same named components as those in the backup library, the CMN20 job overlays old components in the backup library with the components being copied from the production library. Over time, the cleanup process matches components in the Backup Libraries from recently aged packages. This could take a long time, and if a component in a backup library never gets replaced it will always remain there.

Perform Commit Processing for Backup Library Cleanup

If your site requires that the cleanup of Backup Libraries to be handled by ‘commit’ processing, specify the keyword 'BACKUPLIB=C' for program CMNBATCH when executing this program for housekeeping, as follows:

    //CMNBATCH EXEC PGM=CMNBATCH,
    // PARM='SUBSYS=?,BACKUPLIB=C'

Commit processing differs from Package processing in that the latter is a two-library compare and delete operation whereas former is a three-library operation. Commit processing means that once a component has been installed in the production library and the package for that component has reached its aging criteria, you intend to commit that component to production and to remove any same-named components in the backup library regardless of the originating package. You are essentially committing the component in the production library. Provided the components for all aged packages in production are functionally stable, committing the components enables you to manage growth in the Backup Libraries for your ChangeMan ZMF system.

During commit processing, a component in the staging library is compared to the samenamed component in the production library. If the directory entries for these components match, then the component is arbitrarily deleted from the backup library. During package cleanup, in contrast, the staging library and the backup library are compared to determine if the component is in the backup library.

The following steps describe commit processing.

  1. Assume that you create a package called PKG@000100 that contains component 'ABC00001' in the staging library. Next, you change the component and schedule the package for installation. When the package is installed, the CMN20 job attempts to copy a component named 'ABC00001' from the production library associated with this package to its corresponding backup library.

    If a same-named component was previously-placed in the same production library by a previously installed package, the IEBCOPY step backs up the component by copying it from the production library to the backup library. The backup library then has a component named 'ABC00001' from a previous package. When CMN20 runs, the same-named component 'ABC00001' from package PKG@000100 is copied from its staging library to the production library. As a result, the component 'ABC00001' from package PKG@000100 overlays the same-named component that was there previously. When CMN20 completes successfully, the CMN30 job executes a baseline ripple for package PKG@000100, with the following result:

  2. Assume that package PKG@000100 in BAS status has reached its aging criteria so that it is included in the CMN920 report that creates the delete Staging Libraries transaction record for this package. Next, program CMNBATCH is executed with the parm keyword of 'BACKUPLIB=C' in the EXEC statement. Then, the package that will be cleaned up is identified from the delete Staging Libraries transaction record. Next, program CMNBATCH extracts information from the package master to build a table listing all the components for package PKG@000100 that reside in a single staging library. The component name list only contains components that are known to ChangeMan ZMF for that package.

    In this example, the component list only contains 'ABC00001' because it was the only component in package PKG@000100. Next, the data set names for the staging library, the production library, and the backup library associated with package PKG@000100 (for one library type) are placed in another table. Finally, CMNBATCH calls program CMNCLNUP and passes the component list and library names list.

    Once CMNCLNUP accesses the data set names table, it opens the following:

    • Staging data set for package PKG@000100 under the ddname of SYSUT1.

    • Production library for package PKG@000100 under the ddname of SYSC$DSN.

    • Backup library data set for package PKG@000100 under the ddname SYSUT2.

    Next, CMNCLNUP accesses the component names list table to obtain the component 'ABC00001’. Program CMNCLNUP compares the directory entry for component ‘ABC00001’ from the SYSUT1 staging library of package PKG@000100 to the directory entry of the same-named component in the SYSC$DSN production library. If the entries match, you have verified that the component 'ABC00001' in the production library must have come from package PKG@000100, and component ‘ABC00001' is deleted from the backup library represented by ddname SYSUT2 regardless of the source package.

    If the directory entries do not match, then deletion does not occur. This means that any component residing in the backup library with the same name as the component being compared in the Staging and Production Libraries is deleted if the components match.

Considerations for Cleaning up Backup Production Libraries

Cleanup to the Backup Libraries whether by package processing or commit processing results in you being unable to perform a Backout. In deciding whether or not to use package processing or commit processing, consider that the time packages are aged and the length of time that packages remain in development before they are eventually installed in production. Commit processing of the Backup Libraries is the most conservative implementation of a cleanup process because deletion occurs based on aging of the package currently in production. In setting the aging of the staging library by number of days, the installation can judge when the component in the production library has reached a point of functional stability such that the component in the backup library is no longer required and can be safely removed. This means stability in a component is based on its execution in production over a period of time with no problems after which you can release space occupied by the Staging Libraries and delete the backup component from the backup library. In commit processing whatever number of days you specify to age the Staging Libraries is the number of days that a backup component can be expected to remain in the backup library.

With package processing, deletion occurs based only on aging of the package. Package processing is used less often than commit processing. Low activity packages with long aging criteria and short install intervals with infrequent housekeeping runs can use package processing to manage the Backup Libraries. With package processing there is the possibility that a package could start its 90 day aging on June 1 when it is installed. A second package containing the same component name could be created on June 1 and installed on August 30 resulting in the old component from the first package being copied from the production library to the backup library. Housekeeping could run that night where the first package installed June 1 has now aged resulting in the backup component being deleted within a day of the second package being installed. This would impact your ability to back out from the second package. For high activity applications containing many packages in motion with varying install intervals and short aging criteria where you run housekeeping frequently, use commit processing to ensure that the backup for a component in production remains in the backup library for at least the number of days specified in Application Administration for the aging of the Staging Libraries.

Cleanup performs all comparisons between the directory entry for each member in their respective library. Every field in the directory entry for a member is compared to its corresponding field in the directory entry for the same named member in the other library. Cleanup does not read the entire PDS directory for the staging library and compare for all members found. Cleanup gets its member list from the package master for the package that has aged and searches the staging library for the member name. If you allow the staging library to be used as a repository for members outside of ChangeMan ZMF, the cleanup process does not include them. Since only components known to ChangeMan ZMF are included, the comparison process between members based on only the directory entry is accurate.

Consider when a site has been running ChangeMan ZMF for an extended period of time and the cleanup of the Backup Libraries has not been implemented. The Backup Libraries may have experienced substantial growth over this time period, and the Backup Libraries are populated with many components from many packages. With the running of prior housekeeping jobs without cleanup of the Backup Libraries many packages and their corresponding Staging Libraries have probably been deleted leaving their associated components residing in their respective Backup Libraries. This results in the possibility of leaving orphan components in the backup library which are not deleted when you start running a new cleanup process with your housekeeping task.

This is because there is no package record or staging library available to perform a compare against what resides in the backup library for a package that prior housekeeping has deleted. As packages are installed in the future with the same named components as those that exist in the backup library, the result is old components are overlaid in the backup library with the components being copied from the production library as part of the CMN20 job. Over time the cleanup process catches up with delete processing as it encounters more hits in the Backup Libraries from recently aged packages. This catch up process could take an extended period of time, and if a component in a backup library never gets replaced it will always remain.

The cleanup process follows the parameters specified at Application Administration,

Option B, Baseline Configuration, Panel CMNBAS1 where the customer site sets the ‘Install in Prod’ indicators to ‘N’, ‘Y’, or ‘C’. During the cleanup process a read is performed to the package master for General Record 4 (library types) to identify all of the library types defined just to the package that has aged or is in memo delete status. Then a read is performed to Local Record 4 (baseline libraries) where the baseline install indicator is checked. If the library type is defined to the package but the ‘Install in Prod’ indicator is set to ‘N’, then cleanup excludes the library. The cleanup process picks up only the library types defined in General Record 4 that have a corresponding library type in Local Record 4 with the install indicator set to ‘Y’ or ‘C’. Next Local Record 5 (Production Libraries) is read to obtain the data set names of the production libraries.

When cleaning up the Backup Libraries (for packages) with either package or commit processing, consider the following:

  • From the Baseline Configuration Panel (CMNCBAS1) during Application Administration, check the "Install in Prod" indicator setting. If the baseline library type is defined to the package but the indicator is set to N, then cleanup excludes the backup library for that same library type.

  • From the Production Libraries Panel (CMNCPRDL) during Application Administration, check whether or not the production libraries are defined. The cleanup process checks for the baseline install indicator and for the presence of production libraries defined in the package master to decide which Backup Libraries are subject to cleanup processing.

  • From the Application Parameters Panel (CMNGLP02), during Application Administration, check the aging criteria established for Installed Packages and Staging Data Sets.

  • Changing this number might generate two separate transactions on different dates, one to remove package records from the package master and one to delete the Staging Libraries. Cleanup processing reads the package records and requires the Staging Libraries for the comparison.

  • The package records and the Staging Libraries must both exist during cleanup processing. So, for proper housekeeping, the Installed Packages value must be greater than or equal to the Staging Data Sets value.

  • Because the cleanup process deletes components from the backup library, you have no backout capability. Furthermore, you must maintain unique component names if you choose to perform commit processing and share the same backup library among different applications.