一、概述
zabbix是一款非常强大,同时也是应用最为广泛的开源监控软件,本文将给大家介绍如何利用zabbix监控oracle数据库。
二、环境介绍
以下是我安装的环境,实际部署时并不需要跟我的环境一样。
1. 监控机 Readhat 6.5 + Zabbix server 3.4.10 + Python 2.6.6 (操作系统自带) + Oracle Client 11.2 (x86_64)
2. 被监控机 Oracle Server 11.2.0.4
三、选择监控方式
zabbix监控的方式主要有以下三种类型
Zabbix agent
在被监控机上面安装zabbix agent,zabbix agent将被监控机采集到的数据发送给zabbix server。这种方式最常用,一般用来采集服务器的cpu、内存等信息。
SNMP
一些网络设备如交换机,上面无法安装zabbix agent,所以只能通过snmp的方式收集监控数据如端口状态,流量等。
External check
在zabbix server上面运行脚本,直接查询被监控机上的数据。此种方式在被监控机上面不需要做任何部署,所有查询全部从zabbix server上面发出,所以对zabbix server的性能要求较高,官方不推荐大量使用该方式。对于少量的oracle数据库服务器,可以采用该方式。在zabbix server上面新建python脚本,连接被监控机的数据库,查询相关数据。
四、规划监控项
监控数据库的目的是为了保障数据库稳定运行,一旦出现故障,dba能够及时发现并介入处理,监控项分为以下两类
1. 数据库空间不足或数据库发生故障,DBA需要立即处理。
监控项包括表空间、用户状态、实例状态、锁、大量等待事件、闪回区使用率等。此类监控项需要给其设置触发器,一旦出现异常,及时告警。
2. 数据库运行状态的一些统计信息,为DBA定位数据库性能问题发生的时间和类别提供参考。
监控项包括常见的等待事件发生的次数,命中率、硬解析比例等。
下面表格中列出附件中模板的监控项
类别 |
函数名 |
描述 |
间隔时间 |
报警阈值 |
说明 |
实例状态 |
check_active |
检查实例状态是否可用 |
10m |
<>1 |
|
空间 |
dbfilesize |
数据文件占用的总大小 |
6h |
|
方便规划空间 |
dbsize |
数据库已经使用的大小 |
6h |
|
方便规划空间 |
|
show_tablespaces |
得到所有表空间名字 |
6h |
|
zabbix自动发现表空间 |
|
tablespace_used |
表空间已使用的大小 |
6h |
|
方便规划空间 |
|
tablespace |
表空间使用率 |
10m |
>80% |
|
|
show_tablespaces_temp |
得到所有临时表空间名字 |
6h |
|
zabbix自动发现临时表空间 |
|
tablespace_temp |
临时表空间使用率 |
10m |
>80% |
|
|
show_asm_volumes |
得到所有ASM磁盘组名字 |
6h |
|
zabbix自动发现磁盘组 |
|
asm_volume_use |
ASM磁盘组使用率 |
6h |
>80% |
|
|
fra_use |
闪回区使用率 |
20m |
>80% |
|
|
用户 |
show_users |
得到所有状态为open的用户名 |
6h |
|
zabbix自动发现用户名 |
user_status |
用户是否被锁 |
5m |
<>1 |
|
|
会话和锁 |
query_sessions |
查看活动会话 |
5m |
>300 |
|
query_processes |
查看最大的processes/limit |
5m |
>80% |
|
|
query_lock |
超过十分钟的表锁或行锁 |
5m |
>0 |
|
|
实例开启以来的统计信息 |
deadlocks |
死锁发生的次数 |
10m |
|
|
redowrites |
lgwr写日志文件的次数 |
10m |
|
|
|
commits |
用户提交的次数 |
10m |
|
|
|
rollbacks |
用户回滚的次数 |
10m |
|
|
|
hparsratio |
硬解析比例 |
10m |
|
|
|
rcachehit |
data buffer命中率 |
10m |
|
|
|
dsksortratio |
磁盘排序比例 |
10m |
|
|
|
tblscans |
大表全表扫描的次数 |
10m |
|
一般情况下,大表即占用的buffer数量超过buffer cache的2% |
|
indexffs |
索引快速全扫描的次数 |
10m |
|
|
|
netsent |
服务器发送给客户端的字节数 |
10m |
|
|
|
netresv |
客户端发送给服务器的字节数 |
10m |
|
|
|
cpu time |
query_sysmetrics |
cpu消耗在用户级别调用上的比例 |
5m |
|
参数Database CPU Time Ratio |
query_sysmetrics |
cpu消耗在非空闲等待上的比例 |
5m |
|
参数Database Wait Time Ratio |
|
query_sysmetrics |
cpu的运行的时间,单位毫秒/每秒 |
5m |
|
参数Database Time Per Sec |
|
常见等待事件的统计信息 |
maxwaitevents |
超过20个会话发生同一等待事件 |
5m |
not null |
|
freebufwaits |
free buffer waits |
5m |
|
当数据库要在buffer cache中寻找空闲空间来放置数据,但发现空间不足时,就会产生这个等待 |
|
bufbusywaits |
buffer busy waits |
5m |
|
热点块,高并发的DML操作造成 |
|
logswcompletion |
log file switch completion |
5m |
|
redo日志太小,切换过于频繁造成 |
|
logfilesync |
log file sync |
5m |
|
用户提交回滚太频繁造成 |
|
logprllwrite |
log file parallel write |
5m |
|
磁盘io效率低下造成 |
|
dbseqread |
db file sequential read |
5m |
|
sql的索引选择率不高造成 |
|
dbscattread |
db file scattered read |
5m |
|
全表扫描或索引快速全扫描造成 |
|
dbsnglwrite |
db file single write |
5m |
|
检查点更新文件头,磁盘io效率低下造成 |
|
dbprllwrite |
db file parallel write |
5m |
|
dbwn写数据,磁盘io效率低下造成 |
|
directread |
direct path read |
5m |
|
大表的全表扫描造成 |
|
directwrite |
direct path write |
5m |
|
通常由数据加载造成 |
|
latchfree |
latch free |
5m |
|
通常由sql硬解析造成 |
五、安装
上面闲话扯完,进入正式安装环节,我假定你已经安装了zabbix server,因此这里略过zabbix server的安装步骤。
以下所有操作均在zabbix服务器上面执行
安装oracle客户端
从官网下载如下三个rpm包
oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm
oracle-instantclient11.2-devel-11.2.0.4.0-1.x86_64.rpm
oracle-instantclient11.2-sqlplus-11.2.0.4.0-1.x86_64.rpm
使用root安装oracle客户端
rpm -ivh oracle-instantclient11.2-basic-11.2.0.4.0-1.x86_64.rpm
rpm -ivh oracle-instantclient11.2-devel-11.2.0.4.0-1.x86_64.rpm
rpm -ivh oracle-instantclient11.2-sqlplus-11.2.0.4.0-1.x86_64.rpm
配置环境变量
vi + /etc/profile
---------------------------------------------
export ORACLE_HOME=/usr/lib/oracle/11.2/client64
export LD_LIBRARY_PATH=$ORACLE_HOME/lib
export PATH=$PATH:$ORACLE_HOME/bin
执行下面的命令让配置生效
source /etc/profile
添加动态库配置文件
vi + /etc/ld.so.conf.d/oracle.conf
---------------------------------------
/usr/lib/oracle/11.2/client64/lib
执行命令ldconfig
连接oracle进行测试
SQL> sqlplus scott/[email protected]:1521/orcltest
SQL*Plus: Release 11.2.0.4.0 Production on Wed Apr 24 18:24:28 2019
Copyright (c) 1982, 2013, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
With the Partitioning, OLAP, Data Mining and Real Application Testing options
SQL>
出现上面的提示证明oracle client安装成功
安装python相关包
安装cx_Oracle(python连接oracle的包)
wget http://downloads.sourceforge.net/project/cx-oracle/5.1.2/cx_Oracle-5.1.2-11g-py26-1.x86_64.rpm
rpm -ivh cx_Oracle-5.1.2-11g-py26-1.x86_64.rpm
安装argparse
wget https://bootstrap.pypa.io/2.6/get-pip.py --no-check-certificate
python get-pip.py
pip install argparse
上传python脚本
将附件中的pyora.py脚本放入/usr/lib/zabbix/externalscripts/目录下
赋权限,让zabbix用户能够执行该脚本
chmod 755 /usr/lib/zabbix/externalscripts/pyora.py
[先在被监控机的oracle数据库中创建监控用户
create user zabbix identified by zabbix;
grant connect, select any dictionary to zabbix;]
测试脚本
python pyora.py --username zabbix --password zabbix --address 10.30.10.32 --port 1521 --database office show_tablespaces
参数说明
username: 用户名
password: 密码
address: 被监控机ip地址
port: 端口号
database: oracle service name
有返回结果表示脚本能正常运行
上传template文件
将附件中的Pyora_ExternalCheck_11G.xml模板导入到zabbix server中
在zabbix页面中,依次点击Configuration – Templates – Import – 选择文件 – Import,即完成了导入
添加机器,并链接到模板
在zabbix页面中,依次点击Configuration – Hosts – Create host – Hostname (输入ip地址) – groups (选Linux servers) – Templates (选择Pyora_ExternalCheck_11G) – 点击上面的Add – Macros – 点击上面的Add添加宏,全部添加完毕后,点击下面的Add,主机即添加完毕
查看监控数据
Monitoring – Latest Data – Host (选择对应的主机),则可以看到监控的数据了
添加自定义监控项
如果你想监控其它的数据,则只需要在pyora.py中添加一个函数,然后在模板中创建一个监控项,Key末尾对应相应的函数名即可,十分方便
上图中函数名为check_active,则下图对应的Key为pyora.py[--username,{$USERNAME},--password,{$PASSWORD},--address,{$ADDRESS},--database,{$DATABASE},--port,{$PORT},check_active]
参考: https://github.com/bicofino/Pyora
附件:
pyora.py
#!/usr/bin/env python # coding: utf-8 """ Author: Danilo F. Chilene Modified: Bao Yang """ import argparse import cx_Oracle import inspect import json class Checks(object): # check instance status def check_active(self): """Check Intance is active and open""" sql = "select count(*) retvalue from v$instance where status = 'OPEN' and logins = 'ALLOWED' \ and database_status = 'ACTIVE'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) # check size & space def dbfilesize(self): """Total size(MB) of all datafiles (without temp)""" sql = "select to_char(sum(bytes/1024/1024), 'FM9999999990') retvalue from dba_data_files" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dbsize(self): """Total size(MB) of all datafiles have been used (without temp)""" sql = "select to_char(sum(a.bytes - f.bytes)/1024/1024, 'FM9999999990') retvalue from \ (select tablespace_name, sum(bytes) bytes from dba_data_files group by tablespace_name) a, \ (select tablespace_name, sum(bytes) bytes from dba_free_space group by tablespace_name) f \ where a.tablespace_name = f.tablespace_name" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def show_tablespaces(self): """List tablespace names in JSON format for Zabbix auto discover""" sql = "select tablespace_name from dba_tablespaces where contents <> 'TEMPORARY'" self.cur.execute(sql) res = self.cur.fetchall() key = ['{#TABLESPACE}'] lst = [] for i in res: d = dict(zip(key, i)) lst.append(d) print (json.dumps({'data': lst})) def tablespace_used(self, name): """Get tablespace size(MB) has been used""" sql = "select to_char(sum(a.bytes - f.bytes)/1024/1024, 'FM9999999990') retvalue from \ (select tablespace_name, sum(bytes) bytes from dba_data_files group by tablespace_name) a, \ (select tablespace_name, sum(bytes) bytes from dba_free_space group by tablespace_name) f \ where a.tablespace_name = f.tablespace_name and a.tablespace_name = '{0}'".format(name) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def tablespace(self, name): """Get tablespace usage""" sql = "select to_char((a.bytes_alloc - f.bytes_free)*100/a.bytes_total, 'FM9990') retvalue from \ (select tablespace_name, sum(bytes) bytes_alloc, sum(greatest(bytes,maxbytes)) bytes_total \ from dba_data_files group by tablespace_name) a, \ (select tablespace_name, sum(bytes) bytes_free from dba_free_space group by tablespace_name) f \ where a.tablespace_name = f.tablespace_name and a.tablespace_name = '{0}'".format(name) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def show_tablespaces_temp(self): """List temporary tablespace names in JSON format for Zabbix auto discover""" sql = "select tablespace_name from dba_tablespaces where contents = 'TEMPORARY'" self.cur.execute(sql) res = self.cur.fetchall() key = ['{#TABLESPACE_TEMP}'] lst = [] for i in res: d = dict(zip(key, i)) lst.append(d) print (json.dumps({'data': lst})) def tablespace_temp(self, name): """Get temporary tablespace usage""" sql = "select nvl(to_char(used*100/total, 'FM9990'), '0') retvalue from \ (select tablespace_name, sum(greatest(bytes, maxbytes)) total from dba_temp_files \ group by tablespace_name) a, \ (select tablespace, sum(blocks*8192) used from v$tempseg_usage group by tablespace) f \ where a.tablespace_name = f.tablespace(+) and a.tablespace_name = '{0}'".format(name) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def show_asm_volumes(self): """List ASM volumes in JSON format for Zabbix auto discover""" sql = "select name from v$asm_diskgroup" self.cur.execute(sql) res = self.cur.fetchall() key = ['{#ASMVOLUME}'] lst = [] for i in res: d = dict(zip(key, i)) lst.append(d) print (json.dumps({'data': lst})) def asm_volume_use(self, name): """Get ASM volume usage""" sql = "select to_char(100 * (1 - usable_file_mb*(decode(type,'EXTERN',1,'NORMAL',3,'HIGH',5))/total_mb), 'FM9990') \ retvalue from v$asm_diskgroup where name = '{0}'".format(name) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def fra_use(self): """Query the Fast Recovery Area usage""" sql = "select to_char(space_used*100/space_limit, 'FM9990') retvalue from v$recovery_file_dest" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) # check user def show_users(self): """List open user in JSON format for Zabbix auto discover""" sql = "select username from dba_users where account_status = 'OPEN'" self.cur.execute(sql) res = self.cur.fetchall() key = ['{#DBUSER}'] lst = [] for i in res: d = dict(zip(key, i)) lst.append(d) print (json.dumps({'data': lst})) def user_status(self, dbuser): """If user status not change, then 1, otherwise 0""" sql = "select decode(account_status, 'OPEN', '1', '0') retvalue from dba_users \ where username = '{0}'".format(dbuser) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) # session & lock def query_sessions(self): """Query active sessions""" sql = "select count(*) retvalue from v$session where status = 'ACTIVE'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def query_processes(self): """Query max processes""" sql = "select to_char(max_utilization*100/limit_value, 'FM9990') retvalue \ from v$resource_limit where resource_name = 'processes'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def query_lock(self): """Query table is locked over 10 minites""" sql = "select count(*) retvalue from v$lock where type in('TM', 'TX') and ctime > 600" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) # performance def deadlocks(self): """Deadlocks""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'enqueue deadlocks'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def redowrites(self): """Redo Writes""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'redo writes'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def commits(self): """User Commits""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'user commits'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def rollbacks(self): """User Rollbacks""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'user rollbacks'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def hparsratio(self): """Hard parse ratio""" sql = "select to_char(h.value/t.value*100,'FM9990') retvalue from v$sysstat h, v$sysstat t \ where h.name = 'parse count (hard)' and t.name = 'parse count (total)'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def rcachehit(self): """Read Cache hit ratio""" sql = "select to_char((1 - (phy.value - lob.value - dir.value)/ses.value) * 100, 'FM9990') retvalue \ from v$sysstat ses, v$sysstat dir, v$sysstat lob, v$sysstat phy \ where ses.name = 'session logical reads' \ and dir.name = 'physical reads direct' \ and lob.name = 'physical reads direct (lob)' \ and phy.name = 'physical reads'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dsksortratio(self): """Disk sorts ratio""" sql = "select to_char(d.value/(d.value + m.value) * 100, 'FM9990.99') retvalue \ from v$sysstat m, v$sysstat d \ where m.name = 'sorts (memory)' and d.name = 'sorts (disk)'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def tblscans(self): """Table scans (long tables)""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'table scans (long tables)'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def indexffs(self): """Index fast full scans (full)""" sql = "select to_char(value, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'index fast full scans (full)'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def netsent(self): """Bytes(MB) sent via SQL*Net to client""" sql = "select to_char(value/1024/1024, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'bytes sent via SQL*Net to client'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def netresv(self): """Bytes(MB) received via SQL*Net from client""" sql = "select to_char(value/1024/1024, 'FM99999999999999990') retvalue from v$sysstat \ where name = 'bytes received via SQL*Net from client'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) # query cpu time model def query_sysmetrics(self, name): """Query v$sysmetric parameters""" sql = "select value from v$sysmetric where metric_name = '{0}' and \ intsize_csec > 3000".format(name.replace('_', ' ')) self.cur.execute(sql) res = self.cur.fetchall() for i in res: print i[0] # event statistics def maxwaitevents(self): """Get session event name which more than 20""" sql = "select event from (select wait_class, event, count(*) from v$session \ where wait_class <> 'Idle' group by wait_class, event having count(*) > 20 \ order by count(*) desc) where rownum <= 1" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def freebufwaits(self): """Free buffer waits""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'free buffer waits'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def bufbusywaits(self): """Buffer busy waits""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'buffer busy waits'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def logswcompletion(self): """Log file switch completion""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'log file switch completion'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def logfilesync(self): """Log file sync""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'log file sync'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def logprllwrite(self): """Log file parallel write""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'log file parallel write'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dbseqread(self): """DB file sequential read""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'db file sequential read'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dbscattread(self): """DB file scattered read""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'db file scattered read'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dbsnglwrite(self): """DB file single write""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'db file single write'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def dbprllwrite(self): """DB file parallel write""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'db file parallel write'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def directread(self): """Direct path read""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'direct path read'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def directwrite(self): """Direct path write""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'direct path write'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) def latchfree(self): """latch free""" sql = "select nvl(to_char(time_waited, 'FM99999999999999990'), '0') retvalue \ from v$system_event se, v$event_name en \ where se.event(+) = en.name and en.name = 'latch free'" self.cur.execute(sql) res = self.cur.fetchall() for i in res: print (i[0]) class Main(Checks): def __init__(self): parser = argparse.ArgumentParser() parser.add_argument('--username') parser.add_argument('--password') parser.add_argument('--address') parser.add_argument('--database') parser.add_argument('--port') subparsers = parser.add_subparsers() for name in dir(self): if not name.startswith("_"): p = subparsers.add_parser(name) method = getattr(self, name) argnames = inspect.getargspec(method).args[1:] for argname in argnames: p.add_argument(argname) p.set_defaults(func=method, argnames=argnames) self.args = parser.parse_args() def db_connect(self): a = self.args username = a.username password = a.password address = a.address database = a.database port = a.port if a.port else 1521 self.db = cx_Oracle.connect("{0}/{1}@{2}:{3}/{4}".format( username, password, address, port, database)) self.cur = self.db.cursor() def db_close(self): self.cur.close() self.db.close() def __call__(self): try: a = self.args callargs = [getattr(a, name) for name in a.argnames] self.db_connect() try: return self.args.func(*callargs) finally: self.db_close() except Exception as err: print (err) if __name__ == "__main__": main = Main() main()
Pyora_ExternalCheck_11G.xml地址
https://files.cnblogs.com/files/ddzj01/Pyora_ExternalCheck_11G.xml