python snmp 自动化2-在python中使用snmp

python snmp 自动化2-在python中使用snmp

#2012-02-23 磁针石

#承接软件自动化实施与培训 验证码破解 软件破解 脚本开发 测试和python培训等

#gtalk： ouyangchongwu#gmail.com qq 37391319 博客:testing.blog.chinaunix.net

#版权所有，转载刊登请来函联系

#自动化测试和python群组： http://groups.google.com/group/automation_testing_python

#python qq group: 深圳自动化测试python群：113938272

#武冈深圳qq群：66250781 都梁深圳湖南户外群：49494279

# 参考资料

为什么不使用pysnmp？

         Pysnmp是一个纯python的snmp模块，不过我们的mib文件，大概有20%的，Pysnmp不能成功读取，报错如下：

# build-pysnmp-mib -o fsp150cm-sa.mib.txt fsp150cm-sa.mib

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

WARNING: empty MIB module name seen in smidump output at CM-SA-MIB

Traceback (most recent call last):

  File "/usr/bin/libsmi2pysnmp", line 5, in

    pkg_resources.run_script('pysnmp==4.2.1', 'libsmi2pysnmp')

  File "/usr/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 489, in run_script

  File "/usr/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg/pkg_resources.py", line 1214, in run_script

  File "/usr/lib/python2.7/site-packages/pysnmp-4.2.1-py2.7.egg/EGG-INFO/scripts/libsmi2pysnmp", line 435, in

   

  File "/usr/lib/python2.7/site-packages/pysnmp-4.2.1-py2.7.egg/EGG-INFO/scripts/libsmi2pysnmp", line 147, in __genTypeDef

   

KeyError: 'syntax'

smidump -k -f python fsp150cm-sa.mib | /usr/bin/libsmi2pysnmp fails

make sure you are using libsmi version > 0.4.5 (or svn)

 

         网上有篇netsnmp和pysnmp比较的评论：

This isn't so much an answer as much as me sharing my experience testing these libraries.

 

I ran some snmpwalk benchmarks on PySNMP 4.x and net-snmp using python's multiprocessing.Pool. Benchmarks were pretty dirty (basically scaled up the thread pool until I didn't get any performance gains), so reader be-ware. Here's some observations:

 

netsnmp's python bindings:

 

    * Could pull down 11958 oids/sec on the quad core test box.

    * Didn't consume a lot of CPU. Seemed to be waiting on network I/O most of the time (keep reading).

    * Didn't support snmpbulkwalk, unfortunately. So this generates more network traffic and a single thread of execution tends to be slow.

    * It has some annoying MIB look up behavior. I had to specify 'ifName' instead of 'IF-MIB::ifName', which could lead to some ambiguity. I also couldn't seem to find a way to control MIB lookups very well.

    * Threading will not work well. Even if you want one thread of execution, run it in a separate process so you don't starve other threads. This library is thread safe, but not thread friendly.

 

PySNMP4:

 

    * Came in at 5560 oids/sec on the same box.

    * Very CPU intensive. I attribute this to the packet parsing being done in python.

    * MIB lookups I thought were really nice.

    * snmpwalks would leak some unrelated OIDs. For example, I'd walk IF-MIB::ifXTable and at the end I'd get IF-MIB::ifStackTable. IF-MIB dump.

    * I'd almost certainly tailor a wrapper for my application instead of using this library directly. Specifically I'd wrap all the error handling to use Exceptions.

    * I'm not a big fan of writing/reading asynchronous code, so I'd just ignore all the async bits and run big SNMP operations in a separate process.

 

Overall, I'm really kind of disappointed. There's really not a "best overall" library. Apart from the API and performance, PySNMP4 is great. Apart from having some strange MIB/oid lookup handling behavior and not supporting many bulk operations, NetSNMP's python bindings were great.

         可见pysnmp强在mib解析等方面，性能方面不能和netsnmp媲美。既然我们的mib pysnmp无法解析，只好放弃，期待pysnmp尽快开发出自己的好的mib解析器。只有选择netsnmp。

        

为什么不使用subprocess？

         Subprocess会经常出现超时。

 

Python Get示例：

单个get：

print "Get ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0')

oidList = netsnmp.VarList(oid)

resultList = netsnmp.snmpget(oid ,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

以下是通过类的方法进行Get：

 

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oidList = netsnmp.VarList(oid)

resultList = session.get(oidList)

print resultList         

 

多个get：

print "Get ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0')

oid1 =  netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpget(oid ,oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

以下是通过类的方法进行Get：

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid2 = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oid3 = netsnmp.Varbind('.1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1',20,'INTEGER')

oidList = netsnmp.VarList(oid3,oid2)

resultList = session.get(oidList)

print resultList

 

注意类和函数的方式的区别。类方法始终要求传入的参数为oid列表，函数的方式则可以把单个或者多个oid作为参数传入，不需要oid列表。通过代码可以看到函数的方式其实调用类方法，为此，可能效率要比类方法低。

Python Set示例

 

调用netsnmp.snmpset一次set多个值：

print "Set ntpClientEnabled"

oid = netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.2.1.10.1','0','2','INTEGER')

oid1 =  netsnmp.Varbind('1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpset(oid, oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

         上面的1.3.6.1.4.1.前面是否加点号没有关系。Varbind的init方法如下，tag可以用tid，比如ntpClientEnabled，iid表示后面带的序号，比如0,1.1.1.1。val表示具体的值，type表示类型，更详细的参见python netsnmp库的README。Val值'2'和'15'可以不加引号，以为这里是整型，建议还是全部加上引号。

         通过tcpdump抓包分析，snmpset多个值也发送了一次请求，因此多个set集合在一起发送，会明显提高效率。

class Varbind(object):

    def __init__(self, tag=None, iid=None, val=None, type=None):

        

         以下是通过类的方法进行set：

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid2 = netsnmp.Varbind('1.3.6.1.4.1','2544.1.12.2.1.10.1.0',2,'INTEGER')

oid3 = netsnmp.Varbind('.1.3.6.1.4.1.2544.1.12.8.1.1.1.5','1.1.1.1',20,'INTEGER')

oidList = netsnmp.VarList(oid3,oid2)

resultList = session.set(oidList)

print resultList         

 

：

Mib翻译

通过snmptranslate -Tso >toid.txt可以导出tid和oid的对应表。

通过snmptranslate –Td 可以获取oid的详细定义。

# snmptranslate -Td .1.3.6.1.4.1.2544.1.12.2.1.10.1

CM-SYSTEM-MIB::ntpClientEnabled

ntpClientEnabled OBJECT-TYPE

  -- FROM       CM-SYSTEM-MIB

  -- TEXTUAL CONVENTION TruthValue

  SYNTAX        INTEGER {true(1), false(2)}

  MAX-ACCESS    read-write

  STATUS        current

  DESCRIPTION   "This allows to enable/disable the NTP client."

::= { iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) advaMIB(2544) products(1) fsp150cm(12) cmSystemMIB(2) cmSystemObjects(1) cmTimeObjects(10) 1 }

为了提高可读性，我们需要在测试中使用true来表示1，false来表示2，并用ntpClientEnabled来表示.1.3.6.1.4.1.2544.1.12.2.1.10.1，遗憾的是，netsnmp的python库有如下限制“Access to the parsed MIB database is not yet implemented.”为此MIB的转换需要自行完成。为此拟定如下数据结构：

{ntpClientEnabled：.1.3.6.1.4.1.2544.1.12.2.1.10.1}

{ ntpClientEnabled:{ true:1, false:2}}

第2个字典只针对有别名的部分oid。

如果mib库不大，可以通过如下的脚本获取：

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import subprocess

import re

import multiprocessing

import pickle

import time

import random

 

def producer(sequence, input_p):

    for item in sequence:

        # Put the item on the queue

        input_p.send(item)

 

# 消费者

def consumer(lock,pipe):

    output_p, input_p = pipe

    input_p.close() # 关闭管道输入口

    while True:

             lock.acquire()

             try:

                   time.sleep(1)

                   item = output_p.recv()

                   # 管道读取失败的预防，重试一次

             except Exception:

                   item = output_p.recv()

             lock.release()

             if item == None:

                   break

        

             tid = item

             oid = snmpDict[item]

        

             # 处理部分

             try:

                   result = subprocess.check_output(['snmptranslate', '-Td',oid])

                   # subprocess失败的预防，重试一次

             except Exception:

                   time.sleep(random.randint(1,8))

                   result = subprocess.check_output(['snmptranslate', '-Td',oid])

            

             syntaxLine = syntax.search(result)

             if  syntaxLine:               

                   syntaxLine = syntaxLine.group(1)       

                   valueitems = items.findall(syntaxLine)

                   snmpValueDict[tid] = dict(valueitems)         

             else:

                   lock.acquire()

                   print(tid,"None")

                   lock.release()            

                   continue

            

             lock.acquire()

             print(tid,dict(valueitems))

             lock.release()

       

   

 

if __name__ == '__main__':

   

    snmpDict = dict()

    snmpValueDict = dict()

    # 读取语法部分

    syntax = re.compile(r'^  SYNTAX.*?\{(.*?)\}',re.MULTILINE)

    # 读取字符串 和对应的值

    items = re.compile('(\w*)\((\w*)\)')

    f = open ("snmpValueDict.txt",'w')

   

    # 生成tid和oid对应的字典

    for line in open("tid_oid.txt"):

         tid, oid = line.strip().split(',')

         snmpDict[tid] = oid   

 

    pickle.dump(snmpValueDict, open("snmpTidOid.txt", "w"))     

        

    print "length of snmpDict:" + str(len(snmpDict))

   

    # 进程数、创建管道，锁等

    p_num = 10

    process = []   

    (output_p, input_p) = multiprocessing.Pipe()

    lock = multiprocessing.Lock()

   

    # 定义消费进程

    for i in range(p_num):

         t =multiprocessing.Process(target=consumer,args=(lock,(output_p, input_p),))     

         t.daemon=True

         process.append(t)   

 

    # 启动消费进程

    for i in range(p_num):

         process[i].start()

        

    # 关闭输出管道，以往管道填充数据

    output_p.close()

    sequence = snmpDict.values() + [None]*p_num

    producer(snmpDict.keys(), input_p)   

    # 数据填充完毕，打开输入管道

    input_p.close()

   

    # 等待结束

    for i in range(p_num):

         process[i].join()

   

    pickle.dump(snmpValueDict, open("snmpValueDict.txt", "w"))   

    print "length of snmpDict:" + str(len(snmpDict))   

    print "length of snmpValueDict:" + str(len(snmpValueDict))     

 

以上也是一个多进程程序的演示，如果mib的oid超过1000条，建议采用如下脚本来提取：

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import re

import glob

import pickle

 

typeDict = dict()

snmpDict = dict()

snmpValueDict = dict()

 

# tid and oid dict

for line in open("tid_oid.txt"):

    tid, oid = line.strip().split(',')

    snmpDict[tid] = oid   

   

pickle.dump(snmpDict, open("snmpTidOid.txt", "w"))   

 

# type Dict

for mibFile in glob.glob(r"/home/share/andrew/soft/mib/110/MIBs/*.mib"):

    files = open(mibFile).read()

    text =re.sub("\s+"," ",files)

    result = re.findall(r'(\w*)\s+OBJECT-TYPE SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

 

    for dictName,values in result:

        splitValues = re.findall('(\w+)\s*?\((\w+)\)',values)

        typeDict[dictName] = dict(splitValues)

 

pickle.dump(snmpDict, open("typeDict.txt", "w"))               

 

# snmp Value Dict

for mibFile in glob.glob(r"D:\soft\mib\110\MIBs\*.mib"):

    print mibFile

    files = open(mibFile).read()

    text =re.sub("\s+"," ",files)

    result = re.findall(r'\s(\w*?) ::= TEXTUAL-CONVENTION STATUS current DESCRIPTION.*?SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

    print result   

    for dictName,values in result:

        splitValues = re.findall('(\w+)\s*?\((\w+)\)',values)

        snmpValueDict[dictName] = dict(splitValues)

       

    result = re.findall(r'\s(\w*?) ::= TEXTUAL-CONVENTION STATUS current DESCRIPTION.*?SYNTAX\s+\w+\s\{(.*?\(\w\).*?)\}',text)

 

pickle.dump(snmpValueDict, open("snmpValueDict.txt", "w"))   

 

上面的脚本需要根据mib文件格式的不同进行相应的调整。

现在就可以加载存储在文件中的字典，利用tid代替oid进行测试了。

 

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

 

import netsnmp

import re

import pickle

 

snmpTOidDict = pickle.load(open(r"snmpTidOid.txt"))

snmpValueDict = pickle.load(open(r"snmpValueDict.txt"))

 

print "Set ntpClientEnabled"

print snmpValueDict['ntpClientEnabled']

oid = netsnmp.Varbind(snmpTOidDict['ntpClientEnabled'], '0', snmpValueDict['ntpClientEnabled']['true'],'INTEGER')

oid1 =  netsnmp.Varbind(snmpTOidDict['ecpaControlDuration'],'1.1.1.1','15','INTEGER')

resultList = netsnmp.snmpset(oid, oid1,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

下面搞个复杂点的用例，在某产品中创建和删除保护组：

命令行执行如下：

# snmpset -v2c -c private   172.23.192.44 cmFacProtGroupSwitchMode.1.1.1.1 i 1 cmFacProtGroupWorkPort.1.1.1.1 o 1.3.6.1.4.1.2544.1.12.4.1.7.1.1.1.1.1.1 cmFacProtGroupProtPort.1.1.1.1 o 1.3.6.1.4.1.2544.1.12.4.1.7.1.1.1.1.1.2 cmFacProtGroupRowStatus.1.1.1.1 i 4

 

CM-PROTECTION-MIB::cmFacProtGroupSwitchMode.1.1.1.1 = INTEGER: oneplusone(1)

CM-PROTECTION-MIB::cmFacProtGroupWorkPort.1.1.1.1 = OID: CM-FACILITY-MIB::cmEthernetNetPortIndex.1.1.1.1

CM-PROTECTION-MIB::cmFacProtGroupProtPort.1.1.1.1 = OID: CM-FACILITY-MIB::cmEthernetNetPortIndex.1.1.1.2

CM-PROTECTION-MIB::cmFacProtGroupRowStatus.1.1.1.1 = INTEGER: createAndGo(4)

 

# snmpset -v2c -c private  172.23.192.44 cmFacProtGroupRowStatus.1.1.1.1 i 6 CM-PROTECTION-MIB::cmFacProtGroupRowStatus.1.1.1.1 = INTEGER: destroy(6)

 

通过python实现如下

#!/usr/bin/python

# -*- coding: utf-8 -*-

# Function: Ssh to remote server

# Author:         Andrew Xu

# CreateDate: 2012/02/28

 

import netsnmp

import re

import pickle

 

snmpTOidDict = pickle.load(open(r"snmpTidOid.txt"))

snmpValueDict = pickle.load(open(r"snmpValueDict.txt"))

 

print  snmpValueDict['cmFacProtGroupRowStatus']

 

oid = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupSwitchMode'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupSwitchMode']['oneplusone'],'INTEGER')

oid1 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupWorkPort'], '1.1.1.1',

                      snmpTOidDict['cmEthernetNetPortIndex'] + '.1.1.1.1','OBJECTID')

oid2 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupProtPort'], '1.1.1.1',

                      snmpTOidDict['cmEthernetNetPortIndex'] + '.1.1.1.2','OBJECTID')

oid3 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupRowStatus'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupRowStatus']['createAndGo'],'INTEGER')

 

resultList = netsnmp.snmpset(oid, oid1,oid2, oid3,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

 

oid3 = netsnmp.Varbind(snmpTOidDict['cmFacProtGroupRowStatus'], '1.1.1.1',

                      snmpValueDict['cmFacProtGroupRowStatus']['destroy'],'INTEGER')

resultList = netsnmp.snmpset( oid3,Version=2,DestHost='172.23.192.44',Community='private')

print resultList

Normal 0 7.8 pt 0 2 false false false EN-US ZH-CN X-NONE /* Style Definitions */ table.MsoNormalTable {mso-style-name:"Table Normal"; mso-tstyle-rowband-size:0; mso-tstyle-colband-size:0; mso-style-noshow:yes; mso-style-priority:99; mso-style-parent:""; mso-padding-alt:0cm 5.4pt 0cm 5.4pt; mso-para-margin:0cm; mso-para-margin-bottom:.0001pt; mso-pagination:widow-orphan; font-size:10.5pt; mso-bidi-font-size:11.0pt; font-family:"Calibri","sans-serif"; mso-ascii-font-family:Calibri; mso-ascii-theme-font:minor-latin; mso-hansi-font-family:Calibri; mso-hansi-theme-font:minor-latin; mso-bidi-font-family:"Times New Roman"; mso-bidi-theme-font:minor-bidi; mso-font-kerning:1.0pt;}

当然上面这种调用方式，参数显得有点多，正式形成库时会再次封装。

 

性能

session = netsnmp.Session(Version=2,DestHost='172.23.192.44',Community='private')

oid = netsnmp.Varbind('1.3.6.1.4.1.2544.1.12','',2,'INTEGER')

oidList = netsnmp.VarList(oid)

resultList = session.walk(oidList)

print resultList         

print len(resultList)

 

通过上述代码，执行time ./test.py 。walk的效率：耗时：5m38.919s oid个数：13385

平均秒只能获取约40个oid。和参考资料中测试得5560 oids/sec有百倍左右的差距。不过我们的设备已经吃到近90%的cpu，这部分有待以后继续研究优化。