【python3-4】Jenkins pipline集成参数自动化执行python脚本

背景简述

结合前面三篇文章
【python3-1】读取jmeter报告内容
【python3-2】读取html报告返回值，作为接口传参调用
【python3-3】argparse命令行添加参数
本章主要是通过Jenkins集成脚本进行自动化执行测试

Jenkins集成脚本

新建 流水线（Pipeline）工程

流水线（Pipeline）工程

配置基础配置，保存7天，保留6条记录

基础配置

配置参数化构建过程，添加变量参数化，可以选择和填写变量

参数化构建

参数化构建

参数化构建

参数化构建

参数化构建

编辑Pipline脚本

参考pipline官网
参见邮件配置说明Jmeter接口测试（十三）Jenkins_Pipeline邮件配置

node('master') {
    
    stage('Init'){
        git '[email protected]:dz-qa/auto_api.git'
    }
    stage('Excute'){
        echo '开始执行脚本'
        withAnt{
            sh 'ant -file /var/lib/jenkins/workspace/AutoTest-soa-${parameter}/conf/${parameter}_build.xml'
        }
    }
    stage('Deployment Init') {
        script {
            sh 'ls /opt/htdocs/jenkins/reports/soa/html/${parameter}/ | egrep *.html > cart_listFiles.txt'
            def cart_files = readFile("cart_listFiles.txt").split("\\r?\\n");
            sh 'rm -f ${parameter}_listFiles.txt'

            for (i = 0; i < cart_files.size(); i++) {
                publishHTML target: [
                    allowMissing:false,
                    alwaysLinkToLastBuild: false,
                    keepAll:true,
                    reportDir: '/opt/htdocs/jenkins/reports/soa/html/${parameter}/',
                    reportFiles: cart_files[i],
                    reportName: 'HTML Report'
                ]
            }
        
        }

    }
    stage('Report&Email'){
        echo '获取测试报告统计'
        withAnt{
            sh 'python3 /var/lib/jenkins/workspace/AutoTest-${parameter_name}/py-api/demo01.py -v1=${parameter} -v2=${parameter_name} -v3=${execute_type} -v4=${execute_stage}'
                   
            echo '发送报告'
            emailext (
                body: '''
                        

                            

                                


                                构建信息
                                

                            
                            

                                

                                    

                                    

                                        
构建名称：${PROJECT_NAME}
                                        
构建结果:  Successful 
 
                                        
构建编号：${BUILD_NUMBER}
                                        
触发原因：${CAUSE}
                                        
部署分支：${gitBranch}
                                        
构建地址：${BUILD_URL}
                 
                                        
构建日志：${BUILD_URL}console
                                        
环境: ${environment}
                                        
测试阶段：${execute_stage}
                                        
变更概要：${CHANGES}
                    
                                        
测试报告地址：${BUILD_URL}HTML_20Report
                                        
变更集:${JELLY_SCRIPT}
                                        
测试结果报告 ${FILE, path="/opt/htdocs/jenkins/reports/soa/html/${parameter}/${parameter_name}_Report.html"}
                                    
                                    
                                
                            
                        
                    
                    ''',
                subject: '${gitName}执行结果:Successful - ${PROJECT_NAME} - Build # ${BUILD_NUMBER} !', 
                to: '${noticeEmail}',
            ) 
        }
    }

}

配置完成，应用保存退出！

python脚本集成

把已经完成的脚本，通过命令行方式，把外部变量参数引入脚本内调用，方法是调用argparse库（【python3-3】argparse命令行添加参数）

    import urllib
import http.cookiejar
import requests
import json
import argparse
import sys,re
import time
from urllib import request
from bs4 import BeautifulSoup

'''引入外部参数'''
if __name__ == "__main__":
    # 创建命令行解析器句柄，并自定义描述信息
    parser = argparse.ArgumentParser(description="test the argparse package")
    # 定义必选参数 positionArg
    # parser.add_argument("project_name")
    # 定义可选参数module
    parser.add_argument("--module", "-v1", help="模块")
    # 定义可选参数case_suit_name
    parser.add_argument("--case_suit_name", "-v2", help="模块名称")
    # 定义可选参数execute_type
    parser.add_argument("--execute_type", "-v3", help="CI/Normal")
    # 定义可选参数execute_stage
    parser.add_argument("--execute_stage", "-v4", help="FeatureTest / RegressionTest / PreReleaseTest / ReleaseTest")
    args = parser.parse_args()  # 返回一个命名空间
    print(args)
    params = vars(args)  # 返回 args 的属性和属性值的字典
    parameter=[]
    for k, v in params.items():
        parameter.append(v)
        # print(v)
    print(parameter[0])
    print(parameter[1])
    print(parameter[2])
    print(parameter[3])

'''获取报告返回数值'''
path = '/opt/htdocs/jenkins/reports/soa/html/'+parameter[0]+'/'+parameter[1]+'_Report.html'

with open(path, 'r',encoding='utf-8') as f:
    Soup = BeautifulSoup(f.read(), 'lxml')
    titles = Soup.select('html > body > table > .Failure > td')
lists = []
for title in titles:
     lists.append(title.text)
print("api_total:",lists[0],",","api_pass_total:",lists[1],",","api_pass_rate:",lists[3],",","excute_time:",lists[7])

'''转换时间'''
ms = lists[7]
listss =re.compile(r'\d+').findall(ms)
ss = int(listss[0])/1000  #毫秒转为秒
print(r'将毫秒转换为秒：',ms,'-->',ss,'s')

'''执行接口调用方法'''
test_time = time.strftime ("%Y-%m-%d %H:%M:%S", time.localtime ()) #获取当前时间
url = " http://godeye.hqygou.com/api/receiveAutoTestResultTest"
# postdata =urllib.parse.urlencode({
header = {"Content-Type":"application/json"}
raw={
     "api_key": "2b0e740f99f20906a54d04ebe9816d9b",
     "api_sign": "sTest123!@#",
     "project_id": 10007,
     "project_name": "SOA",
     "platform": "OTHER",
     "level": "Core",
     "case_total": 0,
     "case_pass_total": 0,
     "case_pass_rate": 0,
     "api_total": lists[0],
     "api_pass_total": lists[1],
     "api_pass_rate": lists[3],
     "execute_detail": [{
            "case_suit_id": "1",
            "case_suit_name": parameter[1],
            "module": parameter[0],
            "case_total": "0",
            "case_pass_total": "0",
            "case_pass_rate": "0",
            "api_total": lists[0],
            "api_pass_total": lists[1],
            "api_pass_rate": lists[3]
        }],
     "execute_stage": parameter[3],
     "execute_scene": "BaselineRelease",
     "execute_type": parameter[2],
     "autotest_type": "Interface",
     "autotest_source": "Jmeter",
     "excute_start_time": test_time,
     "excute_end_time": test_time,
     "excute_time": ss,
     "report_url": "test",
     "test_result": 1,
     "execute_result": 1,
     "desc": ""
}
data = json.dumps(raw)
data1 = bytes(data,"utf8")
print(data1.decode('unicode_escape'))
req = urllib.request.Request(url,data1,header)
# print(urllib.request.urlopen(req).read().decode('utf-8'))

# #自动记住cookie
cj = http.cookiejar.CookieJar()
opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(cj))
r = opener.open(req)
print(r.read().decode('unicode_escape'))

保存脚本，上传到git库

Jenkins执行自动化结果

Jenkins执行界面进度结果

执行进度

查看测试报告

测试报告

查看控制台（Console Output）执行日志

Started by user admin
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] node
Running on Jenkins in /var/lib/jenkins/workspace/AutoTest-soa-goods
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Init)
[Pipeline] git
Cloning the remote Git repository
Cloning repository [email protected]:xxxx/auto_api.git
 > git init /var/lib/jenkins/workspace/AutoTest-soa-goods # timeout=10
Fetching upstream changes from [email protected]:xxxx/auto_api.git
 > git --version # timeout=10
 > git fetch --tags --progress [email protected]:xxxx/auto_api.git +refs/heads/*:refs/remotes/origin/*
 > git config remote.origin.url [email protected]:xxxx/auto_api.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url [email protected]:xxxx/auto_api.git # timeout=10
Fetching upstream changes from [email protected]:xxxx/auto_api.git
 > git fetch --tags --progress [email protected]:xxxx/auto_api.git +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/master^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/master^{commit} # timeout=10
Checking out Revision 72d35f846f005b7955e26442ce4a23683c0791a1 (refs/remotes/origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 72d35f846f005b7955e26442ce4a23683c0791a1
 > git branch -a -v --no-abbrev # timeout=10
 > git checkout -b master 72d35f846f005b7955e26442ce4a23683c0791a1
Commit message: "更新配置文件"
 > git rev-list --no-walk 261d9419bd5c8104662bbce83ec873f10b68c072 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Excute)
[Pipeline] echo
开始执行脚本
[Pipeline] withAnt
[Pipeline] {
[Pipeline] sh
+ ant -file /var/lib/jenkins/workspace/AutoTest-soa-goods/conf/goods_build.xml
Buildfile: /var/lib/jenkins/workspace/AutoTest-soa-goods/conf/goods_build.xml

all:

商品:
   [jmeter] Executing test plan: /var/lib/jenkins/workspace/AutoTest-soa-goods/soa-goods/soa-goods_2.jmx ==> /opt/htdocs/jenkins/reports/soa/jtl/goods/soa-goods_Report201903010745.jtl
   [jmeter] Creating summariser 

   [jmeter] Created the tree successfully using /var/lib/jenkins/workspace/AutoTest-soa-goods/soa-goods/soa-goods_2.jmx
   [jmeter] Starting the test @ Fri Mar 01 19:45:26 CST 2019 (1551440726705)
   [jmeter] Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
   [jmeter] summary +      3 in 00:00:03 =    1.1/s Avg:   848 Min:   324 Max:  1648 Err:     1 (33.33%) Active: 1 Started: 1 Finished: 0
   [jmeter] summary +     83 in 00:00:14 =    5.7/s Avg:    69 Min:     1 Max:   775 Err:    14 (16.87%) Active: 0 Started: 1 Finished: 1
   [jmeter] summary =     86 in 00:00:17 =    5.0/s Avg:    96 Min:     1 Max:  1648 Err:    15 (17.44%)
   [jmeter] Tidying up ...    @ Fri Mar 01 19:45:44 CST 2019 (1551440744837)
   [jmeter] ... end of run

report:
     [xslt] Processing /opt/htdocs/jenkins/reports/soa/jtl/goods/soa-goods_Report201903010745.jtl to /opt/htdocs/jenkins/reports/soa/html/goods/soa-goods_Report.html
     [xslt] Loading stylesheet /usr/local/jmeter/apache-jmeter-4.0/extras/jmeter-results-new-report.xsl

BUILD SUCCESSFUL
Total time: 22 seconds
[Pipeline] }
[Pipeline] // withAnt
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Deployment Init)
[Pipeline] script
[Pipeline] {
[Pipeline] sh
+ ls /opt/htdocs/jenkins/reports/soa/html/goods/
+ egrep '*.html'
[Pipeline] readFile
[Pipeline] sh
+ rm -f goods_listFiles.txt
[Pipeline] publishHTML
[htmlpublisher] Archiving HTML reports...
[htmlpublisher] Archiving at BUILD level /opt/htdocs/jenkins/reports/soa/html/goods to /var/lib/jenkins/jobs/AutoTest-soa-goods/builds/9/htmlreports/HTML_20Report
[Pipeline] }
[Pipeline] // script
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Report&Email)
[Pipeline] echo
获取测试报告统计
[Pipeline] withAnt
[Pipeline] {
[Pipeline] sh
+ python3 /var/lib/jenkins/workspace/AutoTest-soa-goods/py-api/demo01.py -v1=goods -v2=soa-goods -v3=Normal -v4=RegressionTest
Namespace(case_suit_name='soa-goods', execute_stage='RegressionTest', execute_type='Normal', module='goods')
goods
soa-goods
Normal
RegressionTest
api_total: 101 , api_pass_total: 78 , api_pass_rate: 77.23% , excute_time: 14254 ms
将毫秒转换为秒： 14254 ms --> 14.254 s
b'{"api_key": "2b0e740f99f20906a54d04ebe9816d9b", "api_sign": "sTest123!@#", "project_id": 10007, "project_name": "SOA", "platform": "OTHER", "level": "Core", "case_total": 0, "case_pass_total": 0, "case_pass_rate": 0, "api_total": "101", "api_pass_total": "78", "api_pass_rate": "77.23%", "execute_detail": [{"case_suit_id": "1", "case_suit_name": "soa-goods", "module": "goods", "case_total": "0", "case_pass_total": "0", "case_pass_rate": "0", "api_total": "101", "api_pass_total": "78", "api_pass_rate": "77.23%"}], "execute_stage": "RegressionTest", "execute_scene": "BaselineRelease", "execute_type": "Normal", "autotest_type": "Interface", "autotest_source": "Jmeter", "excute_start_time": "2019-03-02 19:29:44", "excute_end_time": "2019-03-02 19:29:44", "excute_time": 14.254, "report_url": "test", "test_result": 1, "execute_result": 1, "desc": ""}'
{"code":200,"msg":"请求成功","result":{"base_result":"303","detail_result":["35"]}}
[Pipeline] echo
发送报告
[Pipeline] emailext
Sending email to: [email protected]
[Pipeline] }
[Pipeline] // withAnt
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS

执行成功，并发送报告

邮件报告

查看数据库生成数据

生成数据