Unix Sed Tutorial: Advanced Sed Substitution Examples
I. Sed Substitution Delimiter
As we discussed in our previous post, we can use the different delimiters such as @ % | ; : in sed substitute command.
Let us first create path.txt file that will be used in all the examples mentioned below.
$ cat path.txt /usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/omni/lbin:/opt/omni/sbin:/root/bin
Example 1 – sed @ delimiter: Substitute /opt/omni/lbin to /opt/tools/bin
When you substitute a path name which has ‘/’, you can use @ as a delimiter instead of ‘/’. In the sed example below, in the last line of the input file, /opt/omni/lbin was changed to /opt/tools/bin.
$ sed 's@/opt/omni/lbin@/opt/tools/bin@g' path.txt /usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/tools/bin:/opt/omni/sbin:/root/bin
Example 2 – sed / delimiter: Substitute /opt/omni/lbin to /opt/tools/bin
When you should use ‘/’ in path name related substitution, you have to escape ‘/’ in the substitution data as shown below. In this sed example, the delimiter ‘/’ was escaped in the REGEXP and REPLACEMENT part.
$ sed 's/\/opt\/omni\/lbin/\/opt\/tools\/bin/g' path.txt /usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin: /opt/tools/bin:/opt/omni/sbin:/root/bin
II. Sed ‘&’ Get Matched String
The precise part of an input line on which the Regular Expression matches is represented by &, which can then be used in the replacement part.
Example 1 – sed & Usage: Substitute /usr/bin/ to /usr/bin/local
$ sed 's@/usr/bin@&/local@g' path.txt /usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin/local:/usr/sas/bin /usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin/local:/opt/omni/bin: /opt/omni/lbin:/opt/omni/sbin:/root/bin
In the above example ‘&’ in the replacement part will replace with /usr/bin which is matched pattern and add it with /local. So in the output all the occurrance of /usr/bin will be replaced with /usr/bin/local
Example 2 – sed & Usage: Match the whole line
& replaces whatever matches with the given REGEXP.
$ sed 's@^.*$@<<<&>>>@g' path.txt <<</usr/kbos/bin:/usr/local/bin:/usr/jbin/:/usr/bin:/usr/sas/bin>>> <<</usr/local/sbin:/sbin:/bin/:/usr/sbin:/usr/bin:/opt/omni/bin:>>> <<</opt/omni/lbin:/opt/omni/sbin:/root/bin>>>
In the above example regexp has “^.*$” which matches the whole line. Replacement part <<<&>>> writes the whole line with <<< and >>> in the beginning and end of the line respectively.
III. Grouping and Back-references in Sed
Grouping can be used in sed like normal regular expression. A group is opened with “\(” and closed with “\)”.Grouping can be used in combination with back-referencing.
Back-reference is the re-use of a part of a Regular Expression selected by grouping. Back-references in sed can be used in both a Regular Expression and in the replacement part of the substitute command.
Example 1: Get only the first path in each line
$ sed 's/\(\/[^:]*\).*/\1/g' path.txt /usr/kbos/bin /usr/local/sbin /opt/omni/lbin
In the above example, \(\/[^:]*\) matches the path available before first : comes. \1 replaces the first matched group.
Example 2: Multigrouping
In the file path.txt change the order of field in the last line of the file.
$ sed '$s@\([^:]*\):\([^:]*\):\([^:]*\)@\3:\2:\1@g' path.txt /usr/kbos/bin:/usr/local/bin:/usr/jbin:/usr/bin:/usr/sas/bin /usr/local/sbin:/sbin:/bin:/usr/sbin:/usr/bin:/opt/omni/bin: /root/bin:/opt/omni/sbin:/opt/omni/lbin
In the above command $ specifies substitution to happen only for the last line.Output shows that the order of the path values in the last line has been reversed.
Example 3: Get the list of usernames in /etc/passwd file
This sed example displays only the first field from the /etc/passwd file.
$sed 's/\([^:]*\).*/\1/' /etc/passwd root bin daemon adm lp sync shutdown
Example 4: Parenthesize first character of each word
This sed example prints the first character of every word in paranthesis.
$ echo "Welcome To The Geek Stuff" | sed 's/\(\b[A-Z]\)/\(\1\)/g' (W)elcome (T)o (T)he (G)eek (S)tuff
Example 5: Commify the simple number.
Let us create file called numbers which has list of numbers. The below sed command example is used to commify the numbers till thousands.
$ cat numbers
1234
12121
3434
123
$sed 's/\(^\|[^0-9.]\)\([0-9]\+\)\([0-9]\{3\}\)/\1\2,\3/g' numbers
1,234
12,121
3,434
123