https://support.oracle.com/epmos/faces/DocumentDisplay?_afrLoop=221156612954064&id=2081655.1&_adf.ctrl-state=wze2k6dq0_53

purgeLogs: Archive & Cleanup traces, logs in one command (Doc ID 2081655.1)

In this Document

Manual cleanup, what is needed to do?

purgeLogs - Onebutton automation script

How to Install purgelogs

PurgeLogs Usage Examples

purgeLogs - GI/RDBMS cleanup

purgeLogs - OSWatcher/ExaWatcher archive & OAK/DCS cleanup (on ODA - Oracle Database Appliance)

purgeLogs - Files from "extra" given folders

purgeLogs - GI/RDBMS, TFA, OSWatcher & OAK cleanup

purgeLogs - Cleanup listener logs

Setting up purgeLogs as crontab job

PurgeLogs Considerations

PurgeLogs inside TFA

References

APPLIES TO:

Exadata Database Machine V2
Gen 2 Exadata Cloud at Customer
Oracle Database - Enterprise Edition
Real Application Clusters
Oracle Database Appliance - Version All Versions and later
Linux x86-64

ABSTRACT

This document explains how to use "purgeLogs" - a tool used to archive on OCI bucket, cleanup, traces and logs created by different Exadata/ODA (Oracle Database Appliance) components such Grid Infrastructure, RDBMS, TFA (Trace File Analyzer Collector), OSWatcher, OAK (Oracle Appliance Kit) or DCS.

HISTORY

Author: Ruggero Citton - RACPack Team
Create Date jan-2015
Update Date May-2023
Latest purgeLogs Version: 20230503 - $Revision: 2.0.1.6 $

What's new

What's new in 2.0.1-6 release:
- audit files under 'ORACLE_BASE/admin/*/adump/*' are now removed/archived
What's new in 2.0.1-5 release:
- skipping '/opt/oracle/dcs/log/jobfiles/.gi_rhp_done_tasks_<version>' file
What's new in 2.0.1-4 release:
- on Exadata, orcl cleanup/archive is cleaning/archiving up "/u02/app/oracle/cfgtoollogs/sqlpatch"
What's new in 2.0.1-3 release:
- "orc"l cleanup/archive is cleaning/archiving up "$OH/rdbms/log"
- "oda" cleanup/archive is cleaning/archiving up "/u01/app/odaorabase/oracle/cfgtoollogs/sqlpatch"
- GI cleanup is managing not grid user files under "diag/crs/<hostname>/crs/cdump"

DETAILS

### -------------------------------------------------------------------
### Disclaimer:
###
### EXCEPT WHERE EXPRESSLY PROVIDED OTHERWISE, THE INFORMATION, SOFTWARE,
### PROVIDED ON AN \"AS IS\" AND \"AS AVAILABLE\" BASIS. ORACLE EXPRESSLY DISCLAIMS
### ALL WARRANTIES OF ANY KIND, WHETHER EXPRESS OR IMPLIED, INCLUDING, BUT NOT
### LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
### PURPOSE AND NON-INFRINGEMENT. ORACLE MAKES NO WARRANTY THAT: (A) THE RESULTS
### THAT MAY BE OBTAINED FROM THE USE OF THE SOFTWARE WILL BE ACCURATE OR
### RELIABLE; OR (B) THE INFORMATION, OR OTHER MATERIAL OBTAINED WILL MEET YOUR
### EXPECTATIONS. ANY CONTENT, MATERIALS, INFORMATION OR SOFTWARE DOWNLOADED OR
### OTHERWISE OBTAINED IS DONE AT YOUR OWN DISCRETION AND RISK. ORACLE SHALL HAVE
### NO RESPONSIBILITY FOR ANY DAMAGE TO YOUR COMPUTER SYSTEM OR LOSS OF DATA THAT
### RESULTS FROM THE DOWNLOAD OF ANY CONTENT, MATERIALS, INFORMATION OR SOFTWARE.
###
### ORACLE RESERVES THE RIGHT TO MAKE CHANGES OR UPDATES TO THE SOFTWARE AT ANY
### TIME WITHOUT NOTICE.
###
### Limitation of Liability:
###
### IN NO EVENT SHALL ORACLE BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
### SPECIAL OR CONSEQUENTIAL DAMAGES, OR DAMAGES FOR LOSS OF PROFITS, REVENUE,
### DATA OR USE, INCURRED BY YOU OR ANY THIRD PARTY, WHETHER IN AN ACTION IN
### CONTRACT OR TORT, ARISING FROM YOUR ACCESS TO, OR USE OF, THE SOFTWARE.
### --------------------------------------------------------------------------------
### The tool has been tested and appears to work as intended.
### --------------------------------------------------------------------------------

NOTE: In the images and/or the document content below, the user information and environment data used represents fictitious data from the Oracle sample or bulit-in schema(s), Public Documentation delivered with an Oracle database product or other training material. Any similarity to actual environments, actual persons, living or dead, is purely coincidental and not intended in any manner.

PurgeLogs concept

On-premises systems such Exadata, ODA (Oracle Database Appliance) and on-cloud DBCS, ExaCS, ExaC@C, several components such Grid Infrastructure, RDBMS, TFA (Trace File Analyzer Collector), OSWatcher, OAK (Oracle Appliance Kit) or DCS are running producing several traces and logs.

Since a patch update may fail due to filesystem full or some other unexpected behavior, using purgeLog tool can be helpful to cleanup old files. If you need to keep traces & logs for a while on inexpensive storage, purgelogs is offering the possibility to automatically archive them on OCI Storage Bucket

Manual cleanup, what is needed to do?

In order to cleanup all different trace/log files, several different manual actions are needed:

From GI prospective:
- For every single “diag homes”, manually clean several different folders: Alert,Incident,Trace,Cdump,HM
- TFA repository cleanup
- GPNPClient logs
- ACFS logs (from 18c and above)

From RDBMS prospective:
- For every single “diag homes”, manually clean several different folders: Alert,Incident,Trace,Cdump,HM
- TFA repository cleanup
From OSWatcher perspective, remove manually files into

/opt/oracle/oak/oswbb/archive/
|-- oswiostat
|-- oswmeminfo
|-- oswmpstat
|-- oswnetstat
|-- oswprvtnet
|-- oswps
|-- oswslabinfo
|-- oswtop
`-- oswvmstat

In case of ExaWatcher: "/opt/oracle.ExaWatcher/archive"

From OAK perspective, remove manually files into "/opt/oracle/oak/log"

/opt/oracle/oak/log/
|-- fishwrap
|-- test
| |-- client
| |-- dcli
| |-- oak
| |-- odabaseagent
| `-- tools
`-- node
|-- adapters
|-- client
|-- dcli
|-- oak
|-- oakvmclient
|-- odabaseagent
|-- tools
`-- utils

purgeLogs - Onebutton automation script

purgeLogs script provides the ability to archive & cleanup automatically in one single command, all trace/log/core/... files above based on file ‘age’ and its able to manage files from “extra folders” too. The default retention period is 30 days for all components. purgeLog needs to be executed on all nodes.

purgeLogs - Verbs

Archive operation
- archive
Cleanup operation
- cleanup
Make encrypted password file:
- encpass
Get help info
- help

purgeLogs - Usage

Archive function

Usage:
purgelogs.bin archive --bucketname <target bucket name>
                     [--nocli
                        --ociregion <OCI region>
                        --ocinamespace <OCI namespace>
                        --ociidname <OCI user name>
                        --ociatoken <Auth Tokens password file>
                       [--proxyhost <host> --proxyport <port> [--proxyid <user ID> --proxypass <password file>]]
                     ]
                     [--parallel <max parallel upload process>]
                     [--ocicli <oci-cli path>]
                     [--ociprofile <oci-cli profile name>]
                     [--cksum]
                     [--days <days> [--aud] [--lsnr]] |
                     [--orcl <days> [--aud] [--lsnr]] |
                     [--osw <days>] |
                     [--oda <days>] |
                     [--extra '<folder>':<days> | [, '<folder>':<days>]]
                     [--rootlabel <root label>]
                     [--onefile [--filename]]
                     [--dryrun]
purgeLogs archive OPTIONS
--bucketname   <target bucket name>           OCI Bucket name
--nocli                                       OCI restAPI usage
--ociregion    <OCI region>                   Without oci-cli, you need to specify OCI region name
--ocinamespace <OCI namespace>                Without oci-cli, you need to specify OCI namespace
--ociidname    <OCI user name>                Without oci-cli, you need to specify OCI User name
--ociatoken    <Auth Tokens password file>    Without oci-cli, you need to specify OCI User Authentication token password file
--parallel     <max parallel upload process>  Maximum parallel upload process (Default: #CPUs)
--ocicli       <oci-cli path>                 OCI client path (Default: from PATH)
--ociprofile   <oci-cli profile name>         OCI client profile name (Default: DEFAUL
--cksum                                       Verify the object checksum with the local file
--rootlabel    <root label>                   Root folder on OCI Bucket (Default: /yyyymmdd/hostname/)
--onefile                                     It will make a single tar.gz file
--filename     <filename>                     Archive tar.gz filename (Default: yyyymmdd_hostname.tar.gz)
--days         <days>                         Archive orcl,tfa,osw,oda components logs & traces older then # days
--orcl         <days>                         Archive only GI/RDBMS logs & traces (Default 30 days)
--osw          <days>                         Archive only OSW archives older then # days (Default 30 days)
--oda          <days>                         Archive only ODA logs and trace older then # days (Default 30 days)
--extra       '<folder>':<days>               Archive only files in user specified folders (Default 30 days)
--aud                                         Archive Audit logs based on '-orcl <days>' option
--lsnr                                        Archive of listeners log
--dryrun                                      It will show the Archive commands w/o execute them
--help                                        Display this help and exit

Parameters "--ociatoken" and/or "--proxypass" are expecting a file generated with 'encpass' function described below. The content is the encrypted "Auth Tokens" and/or the "proxy user password"
In case of restAPI usage (--nocli) more parameters are required such:
- --ociregion
- --ocinamespace
- --ociidname
- --ociatoken
By default purgelogs archive will upload the files one by one (in parallel) and preserving the original path adding a root folder '/date/hostname/', example

File upload convention

You could override such behavior using '--onefile' option for which purgelogs will upload a single tar.gz file created with following name convention: <date>_<hostname>.tar.gz, you could override such filename with '--filename' option

Cleanup function

Usage:
purgelogs.bin cleanup [--days <days> [--aud] [--lsnr]] |
                      [--orcl <days> [--aud] [--lsnr]] |
                      [--tfa <days>] |
                      [--osw <days>] |
                      [--oda <days>] |
                      [--extra '<folder>':<days> | [, '<folder>':<days>]]
                      [--automigrate]
                      [--dryrun]
purgeLogs cleanup OPTIONS
--days   <days>           Purge orcl,tfa,osw,oda components logs & traces older then # days
--orcl   <days>           Purge only GI/RDBMS logs & traces (Default 30 days)
--tfa    <days>           Purge only TFA repository older then # days (Default 30 days)
--osw    <days>           Purge only OSW archives older then # days (Default 30 days)
--oda    <days>           Purge only ODA logs and trace older then # days (Default 30 days)
--extra '<folder>':<days> Purge only files in user specified folders (Default 30 days)
--aud                     Purge Audit logs based on '-orcl <days>' option
--lsnr                    It will force the cleanup of listeners log independently by the age
--automigrate             It will run the adrci schema migrate commands in case of DIA-49803
--dryrun                  It will show the purge commands w/o execute them
--help                    Display this help and exit

Encapass function

Usage:
purgelogs.bin encpass --file <password file>
purgeLogs encpass OPTIONS
--filename <filename> File name to store encrypted password (for '--ociatoken' and/or '--proxypass')

How to Install purgelogs

purgelogs can be installed using the RPM (RedHat Package Manager) command as following:

# rpm -i purgelogs-2.0.1-X.el7.x86_64.rpm
(*) X=version number

or updating an installed version, issuing:

# rpm -Uvh purgelogs-2.0.1-X.el7.x86_64.rpm
(*) X=version number

Following files are created under ‘/opt/purgelogs‘:

/opt/purgelogs/
└── purgelogs.bin
0 directories, 1 file

Note: installing purgelogs a softlink link is created under '/usr/local/sbin' (/usr/local/sbin/purgelogs.bin --> /opt/purgelogs/purgelogs.bin)

PurgeLogs Usage Examples

purgeLogs - GI/RDBMS cleanup

Cleanup Alert,Incident,Trace,Cdump,HM, GPNPClient and ACFS log (from 18c and above) for the Grid Infrastructure and RDBMS homes removing all file older then 15 days

# /opt/purgelogs/purgelogs.bin cleanup --orcl 15
2015-11-24 01:58:06: I adrci GI purge started
2015-11-24 01:58:06: I adrci GI purging diagnostic destination diag/asm/+asm/+ASM1
2015-11-24 01:58:06: I ... purging ALERT older than 15 days
2015-11-24 01:58:06: I ... purging INCIDENT older than 15 days
2015-11-24 01:58:06: I ... purging TRACE older than 15 days
2015-11-24 02:04:10: I ... purging CDUMP older than 15 days
2015-11-24 02:04:10: I ... purging HM older than 15 days
2015-11-24 02:04:10: I adrci GI purging diagnostic destination diag/crs/oda101/crs
2015-11-24 02:04:10: I ... purging ALERT older than 15 days
2015-11-24 02:04:10: I ... purging INCIDENT older than 15 days
2015-11-24 02:04:10: I ... purging TRACE older than 15 days
2015-11-24 02:04:23: I ... purging CDUMP older than 15 days
2015-11-24 02:04:23: I ... purging HM older than 15 days
2015-11-24 02:04:23: I adrci GI purging diagnostic destination diag/tnslsnr/oda101/listener_scan2
2015-11-24 02:04:23: I ... purging ALERT older than 15 days
2015-11-24 02:04:24: I ... purging INCIDENT older than 15 days
2015-11-24 02:04:24: I ... purging TRACE older than 15 days
2015-11-24 02:04:24: I ... purging CDUMP older than 15 days
2015-11-24 02:04:24: I ... purging HM older than 15 days
2015-11-24 02:04:24: I adrci GI purging diagnostic destination diag/tnslsnr/oda101/listener
2015-11-24 02:04:24: I ... purging ALERT older than 15 days
2015-11-24 02:04:24: I ... purging INCIDENT older than 15 days
2015-11-24 02:04:25: I ... purging TRACE older than 15 days
2015-11-24 02:04:25: I ... purging CDUMP older than 15 days
2015-11-24 02:04:25: I ... purging HM older than 15 days
2015-11-24 02:04:25: I adrci GI purging diagnostic destination diag/tnslsnr/oda101/asmnet1lsnr_asm
2015-11-24 02:04:25: I ... purging ALERT older than 15 days
2015-11-24 02:04:25: I ... purging INCIDENT older than 15 days
2015-11-24 02:04:26: I ... purging TRACE older than 15 days
2015-11-24 02:04:26: I ... purging CDUMP older than 15 days
2015-11-24 02:04:27: I ... purging HM older than 15 days
(...)
2015-11-24 02:13:56: I adrci RDBMS purging diagnostic destination diag/rdbms/s2test/S2TEST
2015-11-24 02:13:56: I ... purging ALERT older than 15 days
2015-11-24 02:13:56: I ... purging INCIDENT older than 15 days
2015-11-24 02:13:57: I ... purging TRACE older than 15 days
2015-11-24 02:13:57: I ... purging CDUMP older than 15 days
2015-11-24 02:13:57: I ... purging HM older than 15 days
2015-11-24 02:13:57: I Purging completed succesfully!

You can include the Grid Infrastructure & RDBMS Audit files using "-aud" option, example:

# /opt/purgelogs/purgelogs.bin cleanup --orcl 15 --aud

purgeLogs - OSWatcher/ExaWatcher archive & OAK/DCS cleanup (on ODA - Oracle Database Appliance)

Cleanup OSWatcher/ExaWatcher archive files and OAK logs, removing all file older then 35 days

# /opt/purgelogs/purgelogs.bin cleanup --osw 35 --oda 35
2015-11-24 02:05:10: I Purging OSW archives older then 35 days
2015-11-24 02:05:10: I Purging completed successfully!

purgeLogs - Files from "extra" given folders

Cleanup files/traces/cores/... from some given "extra" folders such '/tmp', '/var/log', '/opt/app' with different retention period

# /opt/purgelogs/purgelogs.bin cleanup --extra '/tmp':10,'/var/log':20,'/opt/app':30
2015-11-24 02:06:13: I Purging files older then 10 days on user specified folder '/tmp'
2015-11-24 02:06:13: I Purging files older then 20 days on user specified folder '/var/log'
2015-11-24 02:06:13: I Purging files older then 30 days on user specified folder '/opt/app'
2015-11-24 02:06:13: I Purging completed succesfully!

Use this option with caution, a wrong path might result in data loss.

purgeLogs - GI/RDBMS, TFA, OSWatcher & OAK cleanup

Cleanup Alert,Incident,Trace,Cdump,HM for the Grid Infastructure and RDBMS homes + TFA archives + OSWatcher archive files + OAK logs, removing all file older then 30 days (default)

# /opt/purgelogs/purgelogs.bin cleanup
2015-11-24 03:35:35: I adrci GI purge started
(...)
2015-11-24 03:35:40: I adrci RDBMS purge started
(...)
2015-11-24 03:35:44: I Purging TFA archives older then 30 days
2015-11-24 03:35:48: I Purging OSW archives older then 30 days
2015-11-24 03:35:48: I Purging ODA logs older then 30 days
2015-11-24 03:35:49: I Purging completed succesfully! 

purgeLogs - Cleanup listener logs

As described by "Adrci Purge Does Not Clear the Text-formatted Alert.Log located in the Trace Folder (Doc ID 751082.1)", adrci is not cleaning up listener logs. In order to force the cleanup of the following files:

/u01/app/grid/diag/tnslsnr/<hostname>/asmnet1lsnr_asm/trace/mgmtlsnr.log
/u01/app/grid/diag/tnslsnr/<hostname>/mgmtlsnr/trace/asmnet1lsnr_asm.log
/u01/app/grid/diag/tnslsnr/<hostname>/listener/trace/listener.log
/u01/app/grid/diag/tnslsnr/<hostname>/listener_scan1/trace/listener_scan1.log
/u01/app/grid/diag/tnslsnr/<hostname>/listener_scan2/trace/listener_scan2.log

you should use the '-lsnr' option, example:

/opt/purgelogs/purgelogs.bin cleanup --orcl 50 --lsnr

Note: with '-lsnr' option, purgelogs will force the cleanup of listeners log independently by the age

Setting up purgeLogs as crontab job

As example, we are setting up purgelogs to be executed every day at 2.00am purging GI/Rdbms traces older then 20 days and keeping TFA, OSWatcher and ODA logs not older then 10 days

00 02 * * * /opt/purgelogs/purgelogs.bin cleanup --orcl 20 --tfa 10 --osw 10 --oda 10

Note: As many ODA patches could be staged and applied, the correct method to clean up the repository "/opt/oracle/oak/pkgrepos", outside the purdeLogs scope, is described by the Note:1681062.1

PurgeLogs Considerations

For systems without Grid Infrastructure installed (non-RAC), the '/etc/orab' file must be in use.
purgeLogs need to be executed by root user (as purgelogs will cleanup GI(grid user) and RDBMS(oracle user) staff)
purgeLogs can be used with crontab
"extra" command line option could have potentially destroying behavior as it will delete all files on related folders, use this option with caution, a wrong path might result in data loss.
purgeLogs archive function supports: oci-cli and/or restAPI
purgeLogs archive uses Automatic Diagnostic Repository Command Interpreter (ADRCI) utility and for each "ADR Homes" will collect traces & logs using "show tracefiles" command. purgeLogs cleanup for each "ADR Homes" will purge each traces/logs type such 'ALERT, INCIDENT, TRACE, CDUMP, STAGE, SWEEP, HM, UTSCDMP'
purgelogs archive is considering the 'change date' (the file creation date on almost all filesystems is not available). Then for files such the alert.log that is always modified (the age can no be more then 'days'...), the archive most likely will skip it. A possible workaround could be to configure the logrotate (see 'man logrotate'), example having '/u02/app/oracle/diag/rdbms/mydb/mydb1/trace/alert_mydb.log' you could make a file 'alert_mydb' under /etc/logrotate.d/ such

/u02/app/oracle/diag/rdbms/mydb/mydb1/trace/alert_mydb.log {
su oracle oinstall
daily
notifempty
missingok
sharedscripts
copytruncate
rotate 10
size 10M
compress
}

in such way we are going to create a copy of the alert and later purgelogs archive could archive it as the age condition could be satisfied

PurgeLogs inside TFA

Limited PurgeLogs has been included into TFA (Oracle Trace File Analyzer) as 'managelogs' tool feature.

You can turn the "auto purge on" by setting

tfactl set manageLogsAutoPurge=ON

The usage of managelogs:

tfactl [run] managelogs [ -purge [[-older <n><m|h|d>] | [-gi] | [-database <all|d1,d2..>] | [-dryrun] ]] [ -show [usage|variation] [ [-older <n><d>] | [-gi] | [-database <all|d1,d2..>] ] ] [-node <all|local|node1,node2..>]Options:
-purge Purge logs
-older Timeperiod for log purge
-gi Purge Grid Infrastructure logs(all ADR homes under GIBASE/diag and crsdata(cvu dirs))
-database Purge Database logs (Default all else provide list)
-dryrun Estimate logs which will be cleared by purge command
-show Print usage/variation details
-older Timeperiod for change in log volume
-gi Space utilization under GIBASE
-database Space utilization for Database logs (Default all else provide list)
-older <n><m|h|d> Files from past 'n' [d]ays or 'n' [h]ours or 'n' [m]inutes
e.g:
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl managelogs -purge -older 30d -dryrun
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl managelogs -purge -older 30d
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl managelogs -show usage
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl run managelogs -purge -older 30d -dryrun
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl run managelogs -purge -older 30d
/u01/app/18.1.0.0/grid/tfa/tls/tfa_home/bin/tfactl run managelogs -show usage