ORA_Archive_Disk_Check
1.About check_mk metric:ORA_Archive_Disk_Check
This is a custom monitoring metric on check_mk platform,
the threshold is defined in /usr/lib/check_mk_oracle/MAIN/conf1/perf_ORCL_arc_disk.json
[oracle@pcnoradb5 conf1]$ cat perf_ORCL_arc_disk.json
{"target_info":[{
"perf_min":"0",
"chk_interval":5000,
"output_cmd":"echo",
"post_action":"",
"perf_crit":"90",
"chk_cmd":"sh run_arc_disk.sh Bakparameters_ORCL",
"perf_max":"0",
"chk_result_separator":";",
"perf_warn":"85",
"database":"ORCL"
}]}
notice the two threshold value:“perf_warn”:“85”,“perf_crit”:“90”, mean that the disk usage ratio that archive log located reach 85% will trigger warning alarm and 95%trigger critical alarm
2.Solution
2.1 get archive log location
[oracle@pcnoradb5 MAIN]$ sqlplus /nolog
SQL*Plus: Release 10.2.0.5.0 - Production on �P���� 3�� 17 14:23:47 2023
Copyright (c) 1982, 2010, Oracle. All Rights Reserved.
SQL> connect / as sysdba
Connected.
SQL> show parameter log_archive_dest_1
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
log_archive_dest_1 string location=/u2/archive/orcl
2.2 archive log located in /u2, right now we check disk usage by linux df command
[oracle@pcnoradb5 orcl]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 45G 39G 4.2G 91% /
/dev/sda1 2.0G 52M 1.8G 3% /boot
tmpfs 12G 0 12G 0% /dev/shm
/dev/sdb 296G 261G 20G 93% /u1
/dev/sdc1 74G 62G 8.8G 88% /u2
172.19.5.100:/database/Oracle/005127
6.5T 5.6T 894G 87% /DBBACKUP
here will observed that the space usage ration was up to 88% on /u2, the warning alarm will be triggered by check_mk platform at this time
2.3 more further, we need to investigate who occupied most disk space on /u2
[oracle@pcnoradb5 orcl]$ du -sh /u2/*
62.1G /u2/archive
16K /u2/lost+found
The output clearly tell us that the archived logs consumed most of the disk space
next, how to do? it is not recommend to delete these archive logs directly, because these archive log will involve database disaster recovery in the future, it’s possible that will be unable to recovery database to specific time if miss these archive logs
2.4 fortunately, have a script file named rman_backup_archive.sh , which is provided by HQ DBA, in general, it is located /backup/script,
1.backup archivelog to NAS device,the NAS device was mounted in /DBBACKUP,as as you saw in previous output in step 2.2, you can manually mount it if there is not the folder in server
2.after delete archivelog under /u2/archive folder
/backup/script/rman_backup_archive.sh /backup/script/Backup_ORCL
notice, Backup_ORCL is a parameter file, localeted the same location with script file.
2.5 use df command to check again, you will be found that the disk space on /u2 already be released
[oracle@pcnoradb5 ~]$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda2 45G 39G 4.2G 91% /
/dev/sda1 2.0G 52M 1.8G 3% /boot
tmpfs 12G 0 12G 0% /dev/shm
/dev/sdb 296G 261G 20G 93% /u1
/dev/sdc1 74G 28G 43G 40% /u2
172.19.5.100:/database/Oracle/005127
6.5T 5.6T 898G 87% /DBBACKUP
if on the server did not exist the script file , you can copy from other server, or manually execute the following command in rman command line:
RUN {
ALLOCATE CHANNEL ch00 TYPE DISK;
BACKUP
AS COMPRESSED BACKUPSET
filesperset 20
FORMAT '/DBBACKUP/RMAN/ARCHIVE/al_%s_%p_%T'
ARCHIVELOG ALL DELETE INPUT;
RELEASE CHANNEL ch00;
}