shell脚本定时推送钉钉战报
一、目的与演示效果
解析服务端返回的如下字符串，获取今日数据、历史数据，实现钉钉定时推送战报效果。
{
"code": "00000",
"msg": "",
"success": true,
"data": {
"2023-07-19": [
"1673945805117063168",
"1671561317942689792",
"1658787124574552064",
"1675750158471659520"
],
"2023-07-20": [
"1673945805117063168",
"1671561317942689792",
"1658787124574552064",
"1675750158471659520"
]
},
"ref": null
}
二、请求网络
1、curl获取数据并赋值变量
在shell中直接使用curl http://www.baidu.com会出现下载器，因此我们拼接一个-s。
对于接收数据的变量api_result：其后面的等号前后都不要有空格。
对于要执行的函数：用反单引号包裹起来
api_result=`curl -s http://www.baidu.com`
也可以使用eval函数来执行。以下为等价写法：
api_result=`eval 'curl -s http://www.baidu.com'`
2、带参数
对于整个地址用引号（单、双都可）拼接起来，要不然不识别拼接的参数。示例：
api_result=`eval 'curl -s "http://www.baidu.com?s=1&c=ANDROID"'`
等价eval写法：
api_result=`curl -s 'http://www.baidu.com?s=1&c=ANDROID'`
3、封装网络请求方法
对于请求网络，往往是为了拿到里层的data数据。因此可以考虑封装方法来复用。这里含有以下知识点：
1、方法参数传递使用$1 $2 $3等，代表第一个参数、第二个参数的意思。
调用的时候把参数放方法后面，中间以空格连接即可。
2、echo为返回，不要用return，return只能返回数字类型。
#!/bin/bash
# 请求网络，拿到里层data数据
# {"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392"]},"ref":null}
checkUploadResult(){
api_result=`eval $1` # 执行api_result，拿到返回的结果
data=$(echo $api_result | jq -r '.data')
echo $data
}
# 调用方法
androidTotalTodo=`checkUploadResult "curl -s 'http:/www.baidu.com?s=1&c=ANDROID'"`
三、json解析
shell的json解析，这里以jq解析为主，以封装的getJsonValuesByAwk解析为辅。主要是jq解析获取key的时候要求key不能是数字
1、根据key获取value (key非数字)
【key非数字】从json数据api_result中取data，即可这样写
api_result='{"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392"]},"ref":null}'
data=$(echo $api_result | jq -r '.data')
echo $data
# 打印数据：{ "2023-07-12": [ "1679009176166203392" ] }
【key为数字】这个时候使用jq解析会报错
json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
data=$(echo $json | jq -r '.2023-07-12')
echo $data
# 打印数据：-18.7977
然后我们使用getJsonValuesByAwk解析就正常了
### 方法简要说明：
### 1. 是先查找一个字符串：带双引号的key。如果没找到，则直接返回defaultValue。
### 2. 查找最近的冒号，找到后认为值的部分开始了，直到在层数上等于0时找到这3个字符：,}]。
### 3. 如果有多个同名key，则依次全部打印（不论层级，只按出现顺序）
### @author lux feary
###
### 3 params: json, key, defaultValue
function getJsonValuesByAwk() {
awk -v json="$1" -v key="$2" -v defaultValue="$3" 'BEGIN{
foundKeyCount = 0
while (length(json) > 0) {
# pos = index(json, "\""key"\""); ## 这行更快一些，但是如果有value是字符串，且刚好与要查找的key相同，会被误认为是key而导致值获取错误
pos = match(json, "\""key"\"[ \\t]*?:[ \\t]*");
if (pos == 0) {if (foundKeyCount == 0) {print defaultValue;} exit 0;}
++foundKeyCount;
start = 0; stop = 0; layer = 0;
for (i = pos + length(key) + 1; i <= length(json); ++i) {
lastChar = substr(json, i - 1, 1)
currChar = substr(json, i, 1)
if (start <= 0) {
if (lastChar == ":") {
start = currChar == " " ? i + 1: i;
if (currChar == "{" || currChar == "[") {
layer = 1;
}
}
} else {
if (currChar == "{" || currChar == "[") {
++layer;
}
if (currChar == "}" || currChar == "]") {
--layer;
}
if ((currChar == "," || currChar == "}" || currChar == "]") && layer <= 0) {
stop = currChar == "," ? i : i + 1 + layer;
break;
}
}
}
if (start <= 0 || stop <= 0 || start > length(json) || stop > length(json) || start >= stop) {
if (foundKeyCount == 0) {print defaultValue;} exit 0;
} else {
print substr(json, start, stop - start);
}
json = substr(json, stop + 1, length(json) - stop)
}
}'
}
json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
data=`getJsonValuesByAwk "$json" '2023-07-12' '没有数据'`
echo $data
# 打印数据：["1679009176166203392"]
2、获取数组长度
list='["1679009176166203392"]'
arr=($(echo $list | tr -d '[],'))
echo "${#arr[@]}"
# 打印数据：1
3、删除json中的某个key
json='{"2023-07-12":["1679009176166203392"],"2023-07-13":["1679009176166203392"]}'
json=$(echo "$json" | jq 'del(.["2023-07-12"])')
echo "$json"
# 打印数据：
#{
# "2023-07-13": [
# "1679009176166203392"
# ]
#}
4、删除空格与引号
钉钉要求文案中不能有空格与引号，否则推送报错。
# 删除空格
dingtalkResult="${dingtalkResult// /}"
# 删除引号"
dingtalkResult=${dingtalkResult//"\""/}
5、判断数据是否为空
比如对未声明的变量json。前面判断是否是字符串null，后边"-z"选项来检查变量是否为空。
#!/bin/bash
# null字符串或者空
if [ "$json" = "null" ] || [ -z "$json" ]; then
echo '数据空'
else
echo '数据不空'
fi
# 打印数据：数据空
四、钉钉推送
提供两种，一种是带At一种不带
# 钉钉通知
notifyDingTalk(){
curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
-H 'Content-Type: application/json' \
-d '{
"msgtype": "text",
"text": {
"content": "'$1'"
},
}'
}
# 钉钉通知带at
notifyDingTalkWithAt(){
curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
-H 'Content-Type: application/json' \
-d '{
"msgtype": "text",
"text": {
"content": "'$1'"
},
"at": {
"atMobiles": [
"1875819063x",
"1580366544x"
],
"isAtAll": false
}
}'
}
使用如下
notifyDingTalk '这是发给钉钉的文案'
notifyDingTalkWithAt '这是发给钉钉的文案'
配置安全文案即可
五、jenkins配置
jenkins配置定时器如下
H 17 * * *
会提示构建时间
六、源码
敏感信息部分以xxx打码，替换自己的后不影响运行。
#!/bin/bash
# 当前时间（示例：2023-07-15）
cur_time=`date +"%F"`
#钉钉key
DKEY="65f276f7747278212d5fb85729117037bb62485b7573eb6b1ba028acf2ef6xxx"
# 默认为null字符串
emptyValue=null
### 方法简要说明：
### 1. 是先查找一个字符串：带双引号的key。如果没找到，则直接返回defaultValue。
### 2. 查找最近的冒号，找到后认为值的部分开始了，直到在层数上等于0时找到这3个字符：,}]。
### 3. 如果有多个同名key，则依次全部打印（不论层级，只按出现顺序）
### @author lux feary
###
### 3 params: json, key, defaultValue
function getJsonValuesByAwk() {
awk -v json="$1" -v key="$2" -v defaultValue="$3" 'BEGIN{
foundKeyCount = 0
while (length(json) > 0) {
# pos = index(json, "\""key"\""); ## 这行更快一些，但是如果有value是字符串，且刚好与要查找的key相同，会被误认为是key而导致值获取错误
pos = match(json, "\""key"\"[ \\t]*?:[ \\t]*");
if (pos == 0) {if (foundKeyCount == 0) {print defaultValue;} exit 0;}
++foundKeyCount;
start = 0; stop = 0; layer = 0;
for (i = pos + length(key) + 1; i <= length(json); ++i) {
lastChar = substr(json, i - 1, 1)
currChar = substr(json, i, 1)
if (start <= 0) {
if (lastChar == ":") {
start = currChar == " " ? i + 1: i;
if (currChar == "{" || currChar == "[") {
layer = 1;
}
}
} else {
if (currChar == "{" || currChar == "[") {
++layer;
}
if (currChar == "}" || currChar == "]") {
--layer;
}
if ((currChar == "," || currChar == "}" || currChar == "]") && layer <= 0) {
stop = currChar == "," ? i : i + 1 + layer;
break;
}
}
}
if (start <= 0 || stop <= 0 || start > length(json) || stop > length(json) || start >= stop) {
if (foundKeyCount == 0) {print defaultValue;} exit 0;
} else {
print substr(json, start, stop - start);
}
json = substr(json, stop + 1, length(json) - stop)
}
}'
}
# 请求网络，拿到里层data数据
# {"code":"00000","msg":"","success":true,"data":{"2023-07-12":["1679009176166203392","1676130723649683456","1675059618608447488","1674345317023219712","1672586086360154112","1673586172305211392","1673950865947492352","1675757864595095552","1670766831293562880","1674976668499968000","1676513810044813312","1676512377715163136","1675146378722017280","1673687264389103616","1675283959501684736","1674213349681922048","1675545501522591744","1674269196629966848","1669551312678813696","1675366041963855872","1675445816296341504","1668133451263508480","1673610261883387904","1673945805117063168","1671561317942689792","1658787124574552064","1675750158471659520"]},"ref":null}
checkUploadResult(){
api_result=`eval $1` # 执行api_result，拿到返回的结果
data=$(echo $api_result | jq -r '.data')
echo $data
}
# 获取今天的数据
getTodayList(){
json=`getJsonValuesByAwk "$1" $cur_time $emptyValue`
echo $json
}
getArrayLengthFromString(){
# null或者null字符串
if [ "$1" = "null" ] || [ -z "$1" ]; then
echo 0
else
arr=($(echo "$1" | tr -d '[],'))
echo "${#arr[@]}"
fi
}
# 删除今日数据
deleteTodayData(){
# null或者null字符串
if [ "$1" = "null" ] || [ -z "$1" ]; then
echo {}
else
# 转正常json
# 使用 jq 命令将 JSON 字符串变量转换为 JSON 变量
# json=$(echo "$1" | jq .)
# 删除今日数据
json=$(echo "$json" | jq 'del(.["'$cur_time'"])')
echo "$json"
fi
}
# 总计
androidTotalTodo=`checkUploadResult "curl -s 'https://pre-api.xxx.com/na/api/manual/v2/getVideoInfo?s=1&c=ANDROID'"`
androidTotalFinish=`checkUploadResult "curl -s 'https://pre-api.xxx.com/na/api/manual/v2/getVideoInfo?s=2&c=ANDROID'"`
# 今日
androidTodayTodo=`getTodayList "$androidTotalTodo"`
androidTodayFinish=`getTodayList "$androidTotalFinish"`
# 总计
iosTotalListTodo=`checkUploadResult "curl -s 'https://pre-api.xxx.com/na/api/manual/v2/getVideoInfo?s=1&c=IOS'"`
iosTotalListFinish=`checkUploadResult "curl -s 'https://pre-api.xxx.com/na/api/manual/v2/getVideoInfo?s=2&c=IOS'"`
# 今日
iosTodayTodo=`getTodayList "$iosTotalListTodo"`
iosTodayFinish=`getTodayList "$iosTotalListFinish"`
dingtalkResult+="一、今日待上传"
dingtalkResult+="\n1、安卓今日待上传：【"`getArrayLengthFromString "$androidTodayTodo"`"个】\n"$androidTodayTodo
dingtalkResult+="\n2、ios今日待上传：【"`getArrayLengthFromString "$iosTodayTodo"`"个】\n"$iosTodayTodo
dingtalkResult+="\n\n二、今日完成"
dingtalkResult+="\n1、安卓完成上传：【"`getArrayLengthFromString "$androidTodayFinish"`"个】"
dingtalkResult+="\n2、ios完成上传：【"`getArrayLengthFromString "$iosTodayFinish"`"个】"
dingtalkResult+="\n\n三、历史待上传"
androidTotalTodo=`deleteTodayData $androidTotalTodo`
iosTotalListTodo=`deleteTodayData $iosTotalListTodo`
# 使用 jq 命令将 JSON 对象转换为字符串
sum_androidTotalTodo=$(echo "$androidTotalTodo" | jq -c .)
# 判断整数值是否为 0
if [ "${#sum_androidTotalTodo}" -eq 0 ]; then
dingtalkResult+="\n1、安卓历史待上传统计：{}"
else
dingtalkResult+="\n1、安卓历史待上传统计："$androidTotalTodo
fi
# 使用 jq 命令将 JSON 对象转换为字符串
sum_iosTotalListTodo=$(echo "$iosTotalListTodo" | jq -c .)
# 判断整数值是否为 0
if [ "${#sum_iosTotalListTodo}" -eq 0 ]; then
dingtalkResult+="\n1、ios历史待上传统计：{}"
else
dingtalkResult+="\n1、ios历史待上传统计："$iosTotalListTodo
fi
# 删除空格
dingtalkResult="${dingtalkResult// /}"
# 删除引号"
dingtalkResult=${dingtalkResult//"\""/}
# 钉钉通知
notifyDingTalk(){
curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
-H 'Content-Type: application/json' \
-d '{
"msgtype": "text",
"text": {
"content": "'$1'"
},
}'
}
# 钉钉通知带at
notifyDingTalkWithAt(){
curl "https://oapi.dingtalk.com/robot/send?access_token=$DKEY" \
-H 'Content-Type: application/json' \
-d '{
"msgtype": "text",
"text": {
"content": "'$1'"
},
"at": {
"atMobiles": [
"18758190xxx",
"15803665xxx"
],
"isAtAll": false
}
}'
}
if test "$androidTodayTodo" != null && test "$iosTodayTodo" != null; then
notifyDingTalk $cur_time"：今天andriod、ios视频都未上传完！以下是具体数据："
notifyDingTalk $dingtalkResult
elif test "$androidTodayTodo" != null; then
notifyDingTalk $cur_time"：今天andriod视频未上传完！以下是具体数据："
notifyDingTalk $dingtalkResult
elif test "$iosTodayTodo" != null; then
notifyDingTalk $cur_time"：今天ios视频未上传完！以下是具体数据："
notifyDingTalk $dingtalkResult
else
notifyDingTalk $cur_time"今日视频上传完毕！辛苦了！"
fi