shell中上传数据文件到odps

1. 遍历文件

#!/bin/sh

datapath='/var/ftphome/resolve_result/'
odpspath='/usr/local/odps/bin'
while [ "1" = "1" ]
do
    cd ${datapath}
    for i in `ls`
    do
        cd ${odpspath}
        if [ -d "$i" ]
        then
            echo "This is a directory:$i"
        else
            ds=${i:0-18:8};
            filename=$i;
            sh upload.sh $ds $filename
            rm -rf $datapath$filename
        fi
    done
    sleep 5s
done

2. 上传数据文件(upload.sh)

ds=$1
file=$2
sh odpscmd &
odpscmd -e "alter table ods_sn_user_visit_d add if not exists partition(ds=${ds})"
odpscmd -e "tunnel upload  /var/ftphome/resolve_result/${file} ods_sn_user_visit_d/ds=${ds}  -fd 'd\t฿'"


你可能感兴趣的:(shell中上传数据文件到odps)