##【预备知识】
假如我有个日志文件,每天会往里追加。日志摘要如下——
2018-06-12 09:08:56,105 - hdfs.client - INFO - Renaming ‘/apps/wps_prt_feature/business_info/2018-06-11/android_app.temp-1528765731’ to ‘/apps/wps_prt_feature/business_info/2018-06-11/android_app’.
… …
2018-06-12 09:09:29,392 - hdfs.client - INFO - Renaming ‘/apps/wps_prt_feature/business_info/2018-06-11/ios_app.temp-1528765764’ to ‘/apps/wps_prt_feature/business_info/2018-06-11/ios_app’.
我想在里面找到带有当天日期和Renaming字样的内容,以便确认我跑的作业成功了,
那么我就想用shell脚本从日志文件中过滤出来。如何实现呢?下面是思考的过程——
[tony@test-host-10 business_info]$ today=`date +%F` # 获取当天日期
[tony@test-host-10 business_info]$ echo $today
2018-05-24
从包含当天日期的日志记录中进一步筛选出含有Renaming的日志
[tony@test-host-10 business_info]$ grep $today log_fifth | grep Renaming | wc -l
2
[tony@test-host-10 business_info]$ renamed_hdfsFile_result=`grep $today log_fifth | grep Renaming | wc -l`
[tony@test-host-10 business_info]$ echo $renamed_hdfsFile_result
2
经过bc计算,如果true(有2行含有Renaming的记录)则输出1,否则输出0
[tony@test-host-10 business_info]$ echo "$renamed_hdfsFile_result == 2" | bc
1
在linux shell中测试一下脚本
[tony@test-host-10 business_info]$ if [ `echo "$renamed_hdfsFile_result == 2" | bc` -eq 1 ]; then echo "OK~"; else echo "BAD"; fi
OK~
测试成功后,就可以把我们的业务逻辑写入到脚本里了
[tony@test-host-10 business_info]$ cat check_hdfs_dir.sh
#!/bin/bash
today=`date +%F`
renamed_hdfsFile_result=`grep $today log_fifth | grep Renaming | wc -l`
if [ `echo "$renamed_hdfsFile_result == 2" | bc` -eq 1 ]
then
echo "Successfully renamed android_app and ios_app on our HDFS ^_^"
else
echo "Not good!"
fi
##【需求场景】
我想要在整点时间运行某脚本,但是不想使用crontab。比如我想在上午9点开始运行脚本,需要从date时间中获取小时数,再和9比较,如果大于等于9则开始执行脚本。这里用到了bc来比较数值大小。
如果没有安装bc,需要以root用户使用yum install -y bc即可安装,安装后就能直接使用了。示例脚本如下。
#!/bin/bash
function run_daily_job(){
yesterday=`date -d"1 days ago" +%Y-%m-%d`
now_hour=`date | awk -F":" '{print $1}' | awk '{print $5}'`
echo "Present hour number is____${now_hour}"
# 判断当前hour是不是9点或9点以后
if [ `echo "${now_hour} >= 9" | bc` -eq 1 ]; then
echo "Now it's $1"
echo Starting processing data on ${yesterday}
# DO WHAT YOU WANT HERE.
if [ $? -eq 0 ]; then
echo
echo "0=}========> Job finished."
exit 0
else
echo "Something wrong while running job."
exit -1
fi
else
current=$1
echo "Current time is ${current}, not the right time(09:00) to run job. Wait another 30 minutes ..."
sleep 30m
fi
}
while (true); do
current_time=`date +"%Y-%m-%d %H:%M:%S"`
# 此处传参一定要加双引号,否则run_daily_job()中的echo "Current time is ${current} ..." 只会显示年月日,不显示时分秒
run_job "$current_time"
done
Present hour number is____14
Now it’s 2018-05-24 14:29:57
Starting processing data on 2018-05-23
0=}========> Job finished.
##【延伸】加强脚本的健壮性
如果无法获取到now_hour,脚本就失效了。所以,为了保证能在9点钟运行我的脚本,需要在没有获取到now_hour变量值的情况下给一个默认值。读者可以自己在上面脚本里加一行,加在引用now_hour之前的语句即可。
加之前,可以自己验证一下,我这边验证过了,请看——
【若变量hour为空,则用 :- 给一个默认值】
[tony@test-host-10 business_info]$ hour='' #此时hour为空
[tony@test-host-10 business_info]$ echo $hour
[tony@test-host-10 business_info]$ hour=${hour:-9} #hour变量为空,则 :- 给一个默认值9
[tony@test-host-10 business_info]$ echo $hour
9
[tony@test-host-10 business_info]$ hour2=${hour2:-15} #hour2变量为空,则 :- 给一个默认值15
[tony@test-host-10 business_info]$ echo $hour2
15
好了就写到这里,看官们觉得涨知识了,请在文章点个赞 _