02-shell文本处理三剑客之sed
sed 是流编辑器，但是它不会修改源文件。
sed (流文本编辑器)
用法：sed OPTIONS… [SCRIPT] [INPUTFILE…]
-r: 使用扩展的正则表达式;
-n, –quiet, –silent: 不输出模式空间的内容；
-i：直接编辑原文件；
-e: -e ” -e ” -e ”
‘CMD1;CMD2;CMD3’
-f /path/to/sed_script：从指定的文件读取处理脚本
1 模式空间是什么？
模式空间其实就是内存中一个空间，归sed所有s。ed读取流文本文件的时候，每次读取一行，然后载入内存中，这个内存就成为模式空间。那么我们编辑的内容是属于模式空间中的内容，而不是源文件内容。
[root@hadoop1 shelltest]# sed '1,10d' /etc/fstab
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# sed ‘1,10d’ /etc/fstab 这一句语句的执行逻辑是，sed把内容读到模式空间中，判断是不是1——10的内容，是的话执行删除命令d,所以删除了1——10行内容，11行以后的内容就显示出来了。
sed的工作流程可以总结为：
第一步：提取一行内容到模式空间
第二步：针对模式空间中的内容执行编辑命令。
第三步：将模式空间中的内容进行输出。
第四步：清空模式空间。
2 地址定界：
#（行号）: 指定行
$: 最后一行
/regexp/：任何能够被regexp匹配到的行
\%regexp%: 任何能够被regexp匹配到的行；换用%为边界符号；
/regexp/I
\%regexp%I：匹配时忽略字符大小写；
addr1, addr2: 指定范围内的所有行：
1·,/regexp/
#（行号）,#（行号）
/regexp/,/regexp/
#(行号),+#
first~step：指定起始位置，以及步长 1~3
[root@hadoop1 shelltest]# sed '$d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
[root@hadoop1 shelltest]# 以上功能是删除了最后一行。
[root@hadoop1 shelltest]# sed '/dev/d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# sed '\%/dev/mapper%d' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# sed '1+2d' /etc/fstab
sed: -e expression #1, char 2: unknown command: `+'
[root@hadoop1 shelltest]# sed '1,+2d' /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# sed '1~2d' /etc/fstab
#
# Created by anaconda on Tue Jan 26 09:32:28 2016
# Accessible filesystems, by reference, are maintained under '/dev/disk'
#
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
tmpfs /dev/shm tmpfs defaults 0 0
sysfs /sys sysfs defaults 0 0
[root@hadoop1 shelltest]#基本正则表达式元字符：
字符：. [] [^]
次数：*, \+, \?, \{i,j\}
分组：\(\)
锚定：^, $, \<, \>
或者：|
后向引用：\1, \2, ...
编辑命令：
注意：命令之前可以使用!表示取反操作；
=: 显示行号
# sed -n '\%/dev/mapper%=' /etc/fstab
d: 删除匹配到的行；
p: 打印模式空间中的内容；
通常与-n选项一同使用；
a \text: append，
a \text1\ntext2\ntext3: 附加多行文本
i \text：insert,
i \text1\ntext2: 插入多行文本
c \text: change，把匹配到的行替换为给定的文本
s///: 替换命令
g: 全局替换
i: 不区分大小写
[root@hadoop1 shelltest]# sed -n '\%/dev/%=' /etc/fstab
6
12
13
[root@hadoop1 shelltest]# sed -n '5,7p' /etc/fstab
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
[root@hadoop1 shelltest]# 追加：
[root@hadoop1 shelltest]# sed '1a \xielaoshi' /etc/fstab
xielaoshi
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# 还可以同时执行多条命令：
[root@hadoop1 shelltest]# sed '1d;2p;3=' /etc/fstab
#
#
3
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# 替换：
[root@hadoop1 shelltest]# sed '3c \bbbb' /etc/fstab
#
bbbb
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]#[root@hadoop1 shelltest]# sed 's/dev/xielaoshi/' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /xielaoshi/shm tmpfs defaults 0 0
xielaoshipts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]#全局替换：
[root@hadoop1 shelltest]# sed 's/dev/xielaoshi/g' /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Jan 26 09:32:28 2016
#
# Accessible filesystems, by reference, are maintained under '/xielaoshi/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /xielaoshi/shm tmpfs defaults 0 0
xielaoshipts /xielaoshi/pts xielaoshipts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]#3 练习
练习1：删除/boot/grub/grub.conf文件中所以空白开头的行行首的空白；
[root@hadoop1 shelltest]# cat /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-431.el6.x86_64.img
[root@hadoop1 shelltest]# sed 's/^[[:space:]]\+//g' /boot/grub/grub.conf
# grub.conf generated by anaconda
#
# Note that you do not have to rerun grub after making changes to this file
# NOTICE: You have a /boot partition. This means that
# all kernel and initrd paths are relative to /boot/, eg.
# root (hd0,0)
# kernel /vmlinuz-version ro root=/dev/sda2
# initrd /initrd-[generic-]version.img
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.32-431.el6.x86_64)
root (hd0,0)
kernel /vmlinuz-2.6.32-431.el6.x86_64 ro root=UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 rd_NO_LUKS rd_NO_LVM LANG=en_US.UTF-8 rd_NO_MD SYSFONT=latarcyrheb-sun16 crashkernel=auto KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet
initrd /initramfs-2.6.32-431.el6.x86_64.img
[root@hadoop1 shelltest]# 练习2：删除/etc/fstab文件中所有以#开头，后跟至少一个空白字符的行的行首的#和空白字符；
[root@hadoop1 shelltest]# sed 's/^#[[:space:]]\+//g' /etc/fstab
#
/etc/fstab
Created by anaconda on Tue Jan 26 09:32:28 2016
#
Accessible filesystems, by reference, are maintained under '/dev/disk'
See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=b8a0a61e-c5e1-40e7-9a4f-e9a9096565a9 / ext4 defaults 1 1
UUID=a4623924-f98a-47c9-a9e7-38c06e0fc7f6 /boot ext4 defaults 1 2
UUID=04777095-5e99-46d5-87cf-1bc6665635ee swap swap defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
devpts /dev/pts devpts gid=5,mode=620 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
[root@hadoop1 shelltest]# 练习3：echo一个文件路径给sed命令，取出其基名；取出其目录名；
[root@hadoop1 shelltest]# echo "/etc/sysconfig/" | sed 's@[^/]\{1,\}/\?$@@g'
/etc/
[root@hadoop1 shelltest]# 练习4：替换/etc/inittab文件“id:数字:initdefault:”一行中的那个数字为6；
[root@hadoop1 shelltest]# cat /etc/inittab
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:5:initdefault:
[root@hadoop1 shelltest]# sed 's@id:[0-9]:initdefault:@id:6:initdefault@' /etc/inittab
# inittab is only used by upstart for the default runlevel.
#
# ADDING OTHER CONFIGURATION HERE WILL HAVE NO EFFECT ON YOUR SYSTEM.
#
# System initialization is started by /etc/init/rcS.conf
#
# Individual runlevels are started by /etc/init/rc.conf
#
# Ctrl-Alt-Delete is handled by /etc/init/control-alt-delete.conf
#
# Terminal gettys are handled by /etc/init/tty.conf and /etc/init/serial.conf,
# with configuration in /etc/sysconfig/init.
#
# For information on how to write upstart event handlers, or how
# upstart works, see init(5), init(8), and initctl(8).
#
# Default runlevel. The runlevels used are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
id:6:initdefault
[root@hadoop1 shelltest]# 4 高级编辑命令：
h: 用模式空间中的内容覆盖保持空间中的内容；
H：把模式空间中的内容追求至保持空间；
g: get, 从保持空间取回数据至模式空间，覆盖模式空间中原有的内容；
G：GET, 从保持空间取回数据至模式空间，追加于模式空间中现有的内容后面；
x: 把模式空间和保持空间中的内容进行交换；
n: 读取匹配到的行下一行至模式空间；
N: 追加读匹配到的行下一行至模式空间；
d: 删除模式空间中的内容；
D：删除多行模式空间中首行；
sed ‘1!G;h;$!d’ FILE
[root@hadoop1 shelltest]# sed '1!G;h;$!d' /etc/issue
Kernel \r on an \m
CentOS release 6.5 (Final)
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 显示文件的偶数行：sed -n 'n;p' FILE; sed -n '2~2p' FILE
1
2
3
4
显示文件的奇数行：sed -n 'p;n' FILE; sed -n '1~2;p' FILE
练习：用sed将文件的奇数行与偶数行合并为一行，中间用“&&”隔开；
[root@hadoop1 shelltest]# sed '$!N;s@\n@\&\&@g' /etc/issue
CentOS release 6.5 (Final)&&Kernel \r on an \m
[root@hadoop1 shelltest]# 5 课后练习：说明以下sed命令的意义：
(1) sed ‘ !N; !D’ /etc/issue
[root@hadoop1 shelltest]# sed '$!N;$!D' /etc/issue
Kernel \r on an \m
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]#(2) sed ‘$!d’ /etc/issue
删除非尾行。
(3) sed ‘G’ /etc/issue
[root@hadoop1 shelltest]# sed 'G' /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 因为保持空间里面都是空的，所以追加到模式空间后面全是空行。
(4) sed ‘/^$/d;G’ /etc/issue
[root@hadoop1 shelltest]# sed '/^$/d;G' /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# (5) sed ‘n;d’ /etc/issue
[root@hadoop1 shelltest]# sed 'n;d' /etc/issue
CentOS release 6.5 (Final)
[root@hadoop1 shelltest]# (6) sed -n ‘1!G;h;$p’ /etc/issue
[root@hadoop1 shelltest]# cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
[root@hadoop1 shelltest]# 6 参考资料
模式空间和保持空间的理解：
什么是模式空间和保持空间
O(∩∩)O嗯!O(∩∩)O嗯!O(∩_∩)O嗯!今天就写到这里，休息一下。如果你看到此文，想进一步学习或者和我沟通，加我微信公众号：名字：五十年后 。
