在linux上面使用shell脚本统计MYSQL数据情况

本公司涉及到一部分数据推送的业务，然后因为是配置数据传输任务的方式做的，但是有时候数据任务会出现问题，可能不推送等等情况，所以需要有个程序监控是否数据推送完成，简单就是检测规则就是定时跑源数据表和推送目标表的最大ID，然后进行比较。我用Java开发，成本有点大，就直接用Shell脚本简单写一下。

#! /bin/bash
# 将定时跑批任务的结果数据到一张表里
MANAGE_USERNAME=root 
MANAGE_PASSWORD=root
MANAGE_PORT=3306
MANAGE_IPADDRESS=192.168.21.103

# 目标库
PRE_USERNAME=root 
PRE_PASSWORD=root
PRE_PORT=3306
PRE_IPADDRESS=192.168.21.104

# 查询需要跑批的数据表
tables_names=`mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} -e "select table_name from test.table_push_info where table_status=0;" | awk 'NR>1{print $1}'`
for name in $tables_names;
   # 打印看一下效果
	echo $name

拿到表之后就可以用来跑批

max_id=`mysql -u${PRE_USERNAME} -p${PRE_PASSWORD} -h ${PRE_IPADDRESS} -P${PRE_PORT} -e "select COALESCE(id,0) from test.$name order by id desc limit 1;" | tail -1`
# 因为存在如果是没有数据，返回空，这边就判断为空的时候，插入0
    if [ ! $max_id ];then
		mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} test -e "update test.table_push_info set max_id=0,update_time=now() where table_name='$name';"  
	else	
		mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} test -e "update test.table_push_info set max_id=$max_id,update_time=now() where table_name='$name';"  
	fi

上述就是简单完成了

#! /bin/bash
# 将定时跑批任务的结果数据到一张表里
MANAGE_USERNAME=root 
MANAGE_PASSWORD=root
MANAGE_PORT=3306
MANAGE_IPADDRESS=192.168.21.103

# 目标库
PRE_USERNAME=root 
PRE_PASSWORD=root
PRE_PORT=3306
PRE_IPADDRESS=192.168.21.104

# 查询需要跑批的数据表
tables_names=`mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} -e "select table_name from test.table_push_info where table_status=0;" | awk 'NR>1{print $1}'`
for name in $tables_names;
do 
	max_id=`mysql -u${PRE_USERNAME} -p${PRE_PASSWORD} -h ${PRE_IPADDRESS} -P${PRE_PORT} -e "select COALESCE(id,0) from test.$name order by id desc limit 1;" | tail -1`
	# 因为存在如果是没有数据，返回空，这边就判断为空的时候，插入0
    if [ ! $max_id ];then
		mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} test -e "update test.table_push_info set max_id=0,update_time=now() where table_name='$name';"  
	else	
		mysql -u${MANAGE_USERNAME} -p${MANAGE_PASSWORD} -h ${MANAGE_IPADDRESS} -P${MANAGE_PORT} test -e "update test.table_push_info set max_id=$max_id,update_time=now() where table_name='$name';"  
	fi
done

之后将将该程序部署到crontab就好

> chmod +x update_id.sh
> crontab -e
0 * * * * /data/update_id.sh

这样就可以每小时执行一次脚本