2019-04-22日 课程总结

三剑客老大之awk

创建测试环境

mkdir -p /server/files/
cat >>/server/files/reg.txt<

三剑客小切磋

[root@oldboyedu59 /oldboy]# grep 'oldb.y' oldboy.txt
I am oldboy teacher!
my blog is http://oldboy.blog.51cto.com
our size is http://blog.oldboyedu.com
my god,i am not oldbey,but OLDBOY!
oldboy oldboy oldboy

[root@oldboyedu59 /oldboy]# sed -n  '/oldb.y/p' oldboy.txt
I am oldboy teacher!
my blog is http://oldboy.blog.51cto.com
our size is http://blog.oldboyedu.com
my god,i am not oldbey,but OLDBOY!
oldboy oldboy oldboy

root@oldboyedu59 /oldboy]# awk '/oldb.y/' oldboy.txt
I am oldboy teacher!
my blog is http://oldboy.blog.51cto.com
our size is http://blog.oldboyedu.com
my god,i am not oldbey,but OLDBOY!
oldboy oldboy oldboy

awk过滤

环境

[root@oldboyedu59 /oldboy]# cat /server/files/reg.txt 
Zhang Dandan    41117397   :250:100:175
Zhang Xiaoyu    390320151  :155:90:201
Meng  Feixue    80042789   :250:60:50
Wu    Waiwai    70271111   :250:80:75
Liu   Bingbing  41117483   :250:100:175
Wang  Xiaoai    3515064655 :50:95:135
Zi    Gege      1986787350 :250:168:200
Li    Youjiu    918391635  :175:75:300
Lao   Nanhai    918391635  :250:100:175

显示第二列以 i 结尾的行

[root@oldboyedu59 /oldboy]# awk '$2~/i$/' /server/files/reg.txt
Wu    Waiwai    70271111   :250:80:75
Wang  Xiaoai    3515064655 :50:95:135
Lao   Nanhai    918391635  :250:100:175

显示第二列以i结尾的行 并显示第一列 和 第三列

[root@oldboyedu59 /oldboy]# awk '$2~/i$/{print $1,$2,$3}' /server/files/reg.txt
Wu Waiwai 70271111
Wang Xiaoai 3515064655
Lao Nanhai 918391635

 第一列是姓氏
 第二列是名字
 第一第二列合起来就是姓名
 第三列是对应的ID号码
 最后三列是三次捐款数量

示例:

显示Xiaoyu的姓氏和ID号码
条件:显示Xiaoyu
{动作}: 显示姓氏和id

[root@oldboyedu59 /oldboy]# awk '\$2~/Xiaoyu/' /server/files/reg.txt
Zhang Xiaoyu    390320151  :155:90:201
[root@oldboyedu59 /oldboy]# 
[root@oldboyedu59 /oldboy]# awk '$2~/Xiaoyu/{print $1,$3}' /server/files/reg.txt
Zhang 390320151

姓氏是Zhang的人，显示他的第二次捐款金额及她的名字
显示所有ID号码最后一位数字是1或5的人的全名
显示所有人的全名，以姓,名的格式显示，如Meng,Feixue
显示Xiaoyu的捐款.每个值时都有以520135

姓氏是Zhang的人，显示他的第二次捐款金额及她的名字

• 条件: 找出包含Zhang的行
• {动作}: 显示捐款的金额和名字

[root@oldboyedu59 /oldboy]# awk '$1~/^Zhang/' /server/files/reg.txt\
Zhang Dandan    41117397   :250:100:175
Zhang Xiaoyu    390320151  :155:90:201
[root@oldboyedu59 /oldboy]# awk '/^Zhang/' /server/files/reg.txt
Zhang Dandan    41117397   :250:100:175
Zhang Xiaoyu    390320151  :155:90:201

'$1~/^Zhang/' 第1列中以Zhang开头的
'/^Zhang/' 这一行中以Zhang开头

最后一列NF 倒数第二行NF-1

[root@oldboyedu59 /oldboy]# awk -F"[ :]+"  '\$1~/^Zhang/{print \$1,\$2,\$5}' /server/files/reg.txt
Zhang Dandan 100
Zhang Xiaoyu 90
[root@oldboyedu59 /oldboy]# awk -F"[ :]+"  '\$1~/^Zhang/{print \$1,\$2,$(NF-1)}' /server/files/reg.txt
Zhang Dandan 100
Zhang Xiaoyu 90

显示所有ID号码最后一位数字是1或5的人的全名

[root@oldboyedu59 /oldboy]# awk '$3~/[15]$/' /server/files/reg.txt
Zhang Xiaoyu    390320151  :155:90:201
Wu    Waiwai    70271111   :250:80:75
Wang  Xiaoai    3515064655 :50:95:135
Li    Youjiu    918391635  :175:75:300
Lao   Nanhai    918391635  :250:100:175

[root@oldboyedu59 /oldboy]# awk '$3~/[15]$/{print $1,$2}' /server/files/reg.txt
Zhang Xiaoyu
Wu Waiwai
Wang Xiaoai
Li Youjiu
Lao Nanhai

显示所有人的全名，以姓,名的格式显示，如Meng,Feixue

[root@oldboyedu59 /oldboy]# awk '{print $1","$2}' /server/files/reg.txt
Zhang,Dandan
Zhang,Xiaoyu
Meng,Feixue
Wu,Waiwai
Liu,Bingbing
Wang,Xiaoai
Zi,Gege
Li,Youjiu
Lao,Nanhai

awk替换

[root@oldboyedu59 /oldboy]# awk '{print \$NF}' /server/files/reg.txt|tr ':' '\$'
\$250\$100\$175
\$155\$90\$201
\$250\$60\$50
\$250\$80\$75
\$250\$100\$175
\$50\$95\$135
\$250\$168\$200
\$175\$75\$300
\$250\$100\$175

sed 's#:#$#g' reg.txt
awk格式：

s==sub
gsub(/:/,"$")
gsub(/找谁/,"替换成什么")
gsub(/找谁/,"替换成什么",某一列)

[root@oldboyedu59 /oldboy]# awk '{gsub(/:/,"$")}' /server/files/reg.txt
[root@oldboyedu59 /oldboy]# awk '{gsub(/:/,"$"); print $0 }' /server/files/reg.txt
Zhang Dandan    41117397   $250$100$175
Zhang Xiaoyu    390320151  $155$90$201
Meng  Feixue    80042789   $250$60$50
Wu    Waiwai    70271111   $250$80$75
Liu   Bingbing  41117483   $250$100$175
Wang  Xiaoai    3515064655 $50$95$135
Zi    Gege      1986787350 $250$168$200
Li    Youjiu    918391635  $175$75$300
Lao   Nanhai    918391635  $250$100$175

[root@oldboyedu59 /oldboy]# awk '{gsub(/:/,"$",$NF); print $4 }' /server/files/reg.txt
$250$100$175
$155$90$201
$250$60$50
$250$80$75
$250$100$175
$50$95$135
$250$168$200
$175$75$300
$250$100$175

创建环境:

head /etc/passwd >/oldboy/passwd.txt
调换/oldboy/passwd.txt文件的第1列和最后一列的位置（不用修改文件内容）

文件内容:

daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
sync:x:5:0:sync:/sbin:/bin/sync
shutdown:x:6:0:shutdown:/sbin:/sbin/shutdown
halt:x:7:0:halt:/sbin:/sbin/halt
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
operator:x:11:0:operator:/root:/sbin/nologin

想要结果:

/bin/bash:x:0:0:root:/root:root
/sbin/nologin:x:1:1:bin:/bin:bin

sed

[root@oldboyedu59 /oldboy]# sed -r 's#(^.*)(:x.*:)(.*\$)#\3#g' passwd.txt
[root@oldboyedu59 /oldboy]# sed -r 's#(^.*)(:x.*:)(.*$)#\3\2\1#g' passwd.txt

awk

[root@oldboyedu59 /oldboy]# awk -F: '{print $7":"$2":"$3":"$4":"$5":"$6":"$1}' passwd.txt
/bin/bash:x:0:0:root:/root:root
/sbin/nologin:x:1:1:bin:/bin:bin
/sbin/nologin:x:2:2:daemon:/sbin:daemon
/sbin/nologin:x:3:4:adm:/var/adm:adm
/sbin/nologin:x:4:7:lp:/var/spool/lpd:lp
/bin/sync:x:5:0:sync:/sbin:sync
/sbin/shutdown:x:6:0:shutdown:/sbin:shutdown
/sbin/halt:x:7:0:halt:/sbin:halt
/sbin/nologin:x:8:12:mail:/var/spool/mail:mail
/sbin/nologin:x:11:0:operator:/root:operator

awk 手里有两杯饮料

awk -F: '{tmp=$1;$1=$NF;$NF=tmp;print $0 }' passwd.txt

awk-调换 /etc/passwd第1列和最后一列的内容-说明
https://www.processon.com/view/link/5aa1df8ae4b0b089b9e60cbd

OFS详解

OFS awk在显示每一列的时候 每一列之间通过什么分割
OFS的内容就是 ,的内容
Output Field Separator 输出分隔符

[root@oldboyedu59 /oldboy]# awk -F: -vOFS=":" '{tmp=$1;$1=$NF;$NF=tmp;print $0 }' passwd.txt
/bin/bash:x:0:0:root:/root:root
/sbin/nologin:x:1:1:bin:/bin:bin
/sbin/nologin:x:2:2:daemon:/sbin:daemon
/sbin/nologin:x:3:4:adm:/var/adm:adm
/sbin/nologin:x:4:7:lp:/var/spool/lpd:lp
/bin/sync:x:5:0:sync:/sbin:sync
/sbin/shutdown:x:6:0:shutdown:/sbin:shutdown
/sbin/halt:x:7:0:halt:/sbin:halt
/sbin/nologin:x:8:12:mail:/var/spool/mail:mail
/sbin/nologin:x:11:0:operator:/root:operator

awk计算

[root@oldboyedu59 /oldboy]# awk 'BEGIN{print 1/3,2*4,2*2*2*2*2,2^10}'
0.333333 8 32 1024

root@oldboyedu59 /oldboy]# cat /oldboy/oldboy.txt
I am oldboy teacher!
I teach linux.

I like badminton ball,billiard ball and chinese chess!
my blog is http://oldboy.blog.51cto.com
our size is http://blog.oldboyedu.com
my qq is 1351441522

not 4900000448
my god,i am not oldbey,but OLDBOY!
oldboy oldboy oldboy

统计文件中空行的数量

• 方法1

[root@oldboyedu59 /oldboy]# wc -l /oldboy/oldboy.txt
11 /oldboy/oldboy.txt

• 方法2

[root@oldboyedu59 /oldboy]# awk '/^$/{i=i+1;print i}' /oldboy/oldboy.txt
1
2

[root@oldboyedu59 /oldboy]# awk '/^$/{i=i+1;print i}' /etc/services 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17

[root@oldboyedu59 /oldboy]# awk '/^$/{i=i+1}END{print i}' /etc/services 
17

统计access.log中 第9列是200的数量

[root@oldboyedu59 /oldboy]# #统计access.log中 第9列是200的数量
[root@oldboyedu59 /oldboy]# 
[root@oldboyedu59 /oldboy]# 
[root@oldboyedu59 /oldboy]# 
[root@oldboyedu59 /oldboy]# awk '$9==200{i++}END{print i}'   access.log 
142666
[root@oldboyedu59 /oldboy]# awk '$9~/^200$/{i++}END{print i}'   access.log 
142666

awk计算执行过程1+...+10

       i  i=i+$1    i 
1       空 i=0+1     1  
2       1  i=1+2     3
3       3  i=3+3     6 
4       6  i=6+4     10

[root@oldboyedu59 /oldboy]# seq 10  |awk '{i=i+$1;print i}'
1
3
6
10
15
21
28
36
45
55