【读书笔记0103】Beginning linux programming-shell programming
学习体会：本章内容比较多，我这里按照自己的理解重新组织了一下，希望对读过这本书的同学有帮助：
shell programming 是linux学习的必经之路，也是必须学好的，经过一段个人体会是：
shell 编程主要是考察你对shell 的一些结构组织，标识，命令的理解，这些都是日后读scrips所必需掌握的东西。
也许实践才是学习shell的最好方式，所以这一章虽然读完，感觉很多东西只是理解并没有达到能应用自如，这一定需要比较长时间的实践。
这章节，作者依旧娓娓的将shell里面的各种复杂的知识做了归纳，由浅入深，非常适合基础差的同学。
感觉，有经验的如果能仔细看一下这个章节，也对自己的是个系统整理的过程。
What this chapter tell us:
- what the shell is
- basic consideration
-subtleties of syntax:
variables,conditions,program control
lists
functions
commands and commands execution
here document
- debugging
- grep and regular expression
- find
- what the shell is
Beginning of this chapter:+
1 why we should use the shell?
- shell could easy to run at the very complex program which are not time-critical。
- shell use the interpreted ( 解释性) language so not only your can execure the commands and utilitie on the shell but also you can write them.
[ hsy75] what is command and utilities?
e.g.
$ls -al | more
this command use the ls and more utilites.
2 what is shell
is a program that acts as the interface between you and the linux system which support the commands.
- basic consideration
Pipes and redirection ( 也就是用于控制shell的 输入输出)
Redirection:
[ hsy75]个人理解，在default之外的输入输出定义都叫做 redirection( < > >> ):
default:
-0 input
-1 output
-2 error
>> used to append to the file:
e.g.
$ps >> out.txt
>& used to combine two outputs
[ hsy75]2>&1 often used to get the error code to a file
e.g.
$kill -l 1234 >outerr.txt 2>&1
Pipes ( | )
[hsy75] Pipes 就是自动同事处理多个流程到一个你想要的结果
process connected by pipes can run simultaneously and auto rescheduled as data flow between them.
[frankhuang@localhost bin]$ ps |sort|more
11220 pts/1 00:00:00 ps
11221 pts/1 00:00:00 sort
11222 pts/1 00:00:00 more
6800 pts/1 00:00:00 bash
PID TTY TIME CMz
Shell programming = script. creating
1 wildcard 通配符 expansion
shell programming 你必须了解通配符：
*
?
set
^set
{}
and so on:
2 to make a script. executable
[frank@localhost ch02]$ chmod +x first
-subtleties of syntax:
变量variables
[hsy75] 一般shell 变量就是指： a variable by preceding its name with a $
常用的检验变量的方法就是用 echo 把他打印出来
变量定义时候要注意下面几种quote 的不同
" " 括号内均为变量 [hsy75] 好的习惯是用这个东西把变量wrap起来，从而避免shell出错 e.g $"test"
' ' 这表示里面的是字符串 这个也很有用 当你想输出默认字符时候
eg.
y='$'$x // x=100
echo y
$100
这里extra 符号$被打印
\ 反斜杠又叫去功能化标识，用来是上述quote 或者$失效
位置参数变量符：
IFS 用来定义字段分隔符
$1
$2 【hsy75] 这里 1，2.。。positional parameters 是shell里面经常用到的
$* 去除ifs定义的分割符
$@ 不 去除ifs定义的分割符
conditions program control
(if for while until case )
[hsy75] 这里和c很像，可以忽略
注意一下：
[ = test
test include 1string comparision /2arithmetic comparison /3file conditionals.
list( AND OR )
e.g
记住这句就ok了
[ -f file1 ] && command for ture || command for false
functions
shell的函数最重要的就是理解位置参数了(positional parameters),用户端的输入或者是函数参数的传递都是通过这个
eg.
用户通过shell输入：
Frank Huang
那位位置变量默认就是：
$* = Frank Huang
$1 = Frank
$2 = Huang
和位置变量非常有关系的是命令set ,set 经常被用来根据变量来转换为位置变量：
经过set 命令后 data 的值就给了位置变量可以被shell利用了
e.g
set $(date)
echo the month is $2
commands
: = test
. = current shell
echo 这里再一次提到 \的escapte的作用
eval = $( ... ) [ hsy75] extra the content in brace , means give you the value of the value of a vaiable
exec replace the shell
exit n
export just creat an enviroment variable which can be seen by other scripts
expr (expression evaluate) = $(( ... )) [ hsy75]注意这个和前面的 $(...)取变量内容不同 是double parenthsis ,
printf
return
set 很有用的是将一组变量的field 和位置变量联系起来
set the para variables for the shell
eg
echo the data is $(date)
set $(date)
echo the month is $2
shift
trap
unset
two useful commands
find (search the file or directory) [hsy75]diff from grep which find for strings
syntax
find [path] [options] [tests] [actions]
eg.
$find . -mount \(-newer filename -o -name "_*" \) -type f -exec ls -l {}\; -print
[ hsy75]上面这个find 看起来很难，我归纳一下书上的解释下面一下就其实简单
$find . [path=curernt directory] -mount [options=do not search the other files system] \ [quote the braces using a backslash to escapte the meanings in shell] ( [combine the tester using the parentheses] -newer [tests = newer ] filename [tests newer's pattern means the filename is older than the finding file] -o [tests operations means OR] -name [tests= name] "_*" [tests'name pattern means name started with an underscore ] \) [escapte the parentheses] -type [tests type] f [tests type =file] -exec [actions exec] ls -l [the command that we use invode by action] {} [used for -exec or -ok means the full path to the current file] \; [means the parameters on the line of action is end so it is a termiator ] -print [another actions that to print the result]
grep find 的兄弟，经常一起用 find for string ：said by writer it is quite common to have grep as a command passed after a action -exec when using find
grep [options] Pattern [FILES]
[ hsy75] 看起来grep 是非常简单的，但是，当你开始接触 regular expression 的时候，sophisticated would occurs
regular expressiom
^ anchor to the beginning of a line
$ anchor to the end of a line
. any single character
[ ] range of characters
[:blank:] special match patterns,the blank 可以被其他参数代替
-E extended matching
[hsy75] 下面这些是给-E扩展用，所以必须都用 \来escapte shell 的功能
?
*
+
{n}
{n,}
{n,m}
e.g
[frank@localhost ch02]$ grep Th.[[:space:]] words2.txt
The handle toward my hand? Come, let me clutch thee.
The curtain'd sleep; witchcraft celebrates
Thy very stones prate of my whereabout,
[frank@localhost ch02]$ grep Th [匹配Th] . [匹配附加任何一个字母] [[:space:]] [后面紧跟 空格] words2.txt [被查找的文件名]
[frankhuang@localhost ch02]$ grep -E [a-z]\{9\} words2.txt
Proceeding from the heat-oppressed brain?
And such an instrument I was to use.
Thus to mine eyes. Now o'er the one halfworld
The curtain'd sleep; witchcraft celebrates
Pale Hecate's offerings, and wither'd murder,
With Tarquin's ravishing strides, towards his design
Thy very stones prate of my whereabout,
[frankhuang@localhost ch02]$ grep -E [扩展支持开启] [a-z] [range of characters] \ [Escape for shell of {] {9 [有9个字符匹配也就是单词长度为9] \} words2.txt
and commands execution
1 $(command) = '\command'\
we often need to capture the result of a command exexution for use in the shell cript ,the above syntax helps put the output of command into a variable .
the ability to put the result of a command into a script. variable is very powerful.
e.g
set $(who)
writer = $(who)
echo $writer
2 $((arithmetic expansion)) = $(expr arithmetic expansion) 注意这里是双括号
Parameter expansion
3 ${variable} 变量拓展和保护
3.1用来保护shell的扩展时候，只扩展对应的变量
e.g.
for i in 1 2
do
creat_file $test_{i}
creat_file $test_i 报错
done
3.2 参数扩展
${param:default}
类似的还有下面变量扩展参数
# 从头删除匹配字符，如果多个匹配，删除最小匹配长度，并返回
## 从头删除匹配字符，如果多个匹配，删除最长匹配长度，并返回
% 从尾巴删除匹配字符，如果多个匹配，删除最小匹配长度，并返回
%% 从尾巴删除匹配字符，如果多个匹配，删除最长匹配长度，并返回
参数扩展，在redirection output的时候非常有用，
since linux are havily round the idea of filters, the result of one operation must oftern be redirected manually.
e.g.
for image in *.gif
do
cjpeg $image > ${image%%gif}jpg
【hsy75】解释扩展如下
cjpeg [a program to convert from gif to jpg] $image [get the file name from the *.gif] > [redirection] ${ [圈定需要替换的变量] image%% [从尾巴删除匹配字符gif ] gif } jpg
done
比如 draw_0001.gif 将变成 draw_0001.jpg
eg.
#!/bin/sh
unset foo
echo ${foo:-bar}
foo=fud
echo ${foo:-bar}
foo=/usr/bin/X11/startx
echo ${foo#*/}
echo ${foo##*/}
bar=/usr/local/etc/local/networks
echo ${bar%local*}
echo ${bar%%local*}
exit 0
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
here document
其实就是像自动命令的输入，可以和编辑器绑定起来，从而实现通过文件读写来直接运行编辑器里面的scripts
e.g
#!/bin/sh
ed [编译器ed] textfile <<!FunkyStuff! [here document的makers]
3 [ed 编译器命令 move to line 3]
d [ed 编译器命令 删除一行]
.,\ [这里避免shell的作用，从而shell执行的时候可以将$后面的东西看成是非变量，从而保证ed的command 正确执行] $s/is/was/ [替换 is 为was]
w
q
!FunkyStuff!
exit 0
This is line 1
This is line 2
This is line 3 被删除
This is line 4 被替换
将变成
This is line 1
This is line 2
This was line 4
debugging
-n check syntax error only
-v Echoes commands before running
-x after running
set -o xtrace
set +o xtrace
其他的调试选项
-u
-o nounset
-o verbose
我们用上面提到的例子再来说明一下如何调试：
1 没有调试：
[frankhuang@localhost ch02]$ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
2 打开set trace调试
[frankhuang@localhost ch02]$ set -o xtrace
++ echo -ne '\033]0;frankhuang@localhost:~/beginning_linux/beginning/ch02'
[frankhuang@localhost ch02]$ ./param
+ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
++ echo -ne '\033]0;frankhuang@localhost:~/beginning_linux/beginning/ch02'
3关闭set trace 调试
[frankhuang@localhost ch02]$ set +o xtrace
+ set +o xtrace
[frankhuang@localhost ch02]$ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
4
[frankhuang@localhost ch02]$ sh -x ./param
+ unset foo
+ echo bar
bar
+ foo=fud
+ echo fud
fud
+ foo=/usr/bin/X11/startx
+ echo usr/bin/X11/startx
usr/bin/X11/startx
+ echo startx
startx
+ bar=/usr/local/etc/local/networks
+ echo /usr/local/etc/
/usr/local/etc/
+ echo /usr/
/usr/
+ exit 0
5
[frankhuang@localhost ch02]$ sh -v ./param
#!/bin/sh
unset foo
echo ${foo:-bar}
bar
foo=fud
echo ${foo:-bar}
fud
foo=/usr/bin/X11/startx
echo ${foo#*/}
usr/bin/X11/startx
echo ${foo##*/}
startx
bar=/usr/local/etc/local/networks
echo ${bar%local*}
/usr/local/etc/
echo ${bar%%local*}
/usr/
exit 0
going graphical - dialog utility
图形输入工具，这个再编写大型的shell配置文件时候经常用到
e.g
the doc in under development
shell programming 是linux学习的必经之路，也是必须学好的，经过一段个人体会是：
shell 编程主要是考察你对shell 的一些结构组织，标识，命令的理解，这些都是日后读scrips所必需掌握的东西。
也许实践才是学习shell的最好方式，所以这一章虽然读完，感觉很多东西只是理解并没有达到能应用自如，这一定需要比较长时间的实践。
这章节，作者依旧娓娓的将shell里面的各种复杂的知识做了归纳，由浅入深，非常适合基础差的同学。
感觉，有经验的如果能仔细看一下这个章节，也对自己的是个系统整理的过程。
What this chapter tell us:
- what the shell is
- basic consideration
-subtleties of syntax:
variables,conditions,program control
lists
functions
commands and commands execution
here document
- debugging
- grep and regular expression
- find
- what the shell is
Beginning of this chapter:+
1 why we should use the shell?
- shell could easy to run at the very complex program which are not time-critical。
- shell use the interpreted ( 解释性) language so not only your can execure the commands and utilitie on the shell but also you can write them.
[ hsy75] what is command and utilities?
e.g.
$ls -al | more
this command use the ls and more utilites.
2 what is shell
is a program that acts as the interface between you and the linux system which support the commands.
- basic consideration
Pipes and redirection ( 也就是用于控制shell的 输入输出)
Redirection:
[ hsy75]个人理解，在default之外的输入输出定义都叫做 redirection( < > >> ):
default:
-0 input
-1 output
-2 error
>> used to append to the file:
e.g.
$ps >> out.txt
>& used to combine two outputs
[ hsy75]2>&1 often used to get the error code to a file
e.g.
$kill -l 1234 >outerr.txt 2>&1
Pipes ( | )
[hsy75] Pipes 就是自动同事处理多个流程到一个你想要的结果
process connected by pipes can run simultaneously and auto rescheduled as data flow between them.
[frankhuang@localhost bin]$ ps |sort|more
11220 pts/1 00:00:00 ps
11221 pts/1 00:00:00 sort
11222 pts/1 00:00:00 more
6800 pts/1 00:00:00 bash
PID TTY TIME CMz
Shell programming = script. creating
1 wildcard 通配符 expansion
shell programming 你必须了解通配符：
*
?
set
^set
{}
and so on:
2 to make a script. executable
[frank@localhost ch02]$ chmod +x first
-subtleties of syntax:
变量variables
[hsy75] 一般shell 变量就是指： a variable by preceding its name with a $
常用的检验变量的方法就是用 echo 把他打印出来
变量定义时候要注意下面几种quote 的不同
" " 括号内均为变量 [hsy75] 好的习惯是用这个东西把变量wrap起来，从而避免shell出错 e.g $"test"
' ' 这表示里面的是字符串 这个也很有用 当你想输出默认字符时候
eg.
y='$'$x // x=100
echo y
$100
这里extra 符号$被打印
\ 反斜杠又叫去功能化标识，用来是上述quote 或者$失效
位置参数变量符：
IFS 用来定义字段分隔符
$1
$2 【hsy75] 这里 1，2.。。positional parameters 是shell里面经常用到的
$* 去除ifs定义的分割符
$@ 不 去除ifs定义的分割符
conditions program control
(if for while until case )
[hsy75] 这里和c很像，可以忽略
注意一下：
[ = test
test include 1string comparision /2arithmetic comparison /3file conditionals.
list( AND OR )
e.g
记住这句就ok了
[ -f file1 ] && command for ture || command for false
functions
shell的函数最重要的就是理解位置参数了(positional parameters),用户端的输入或者是函数参数的传递都是通过这个
eg.
用户通过shell输入：
Frank Huang
那位位置变量默认就是：
$* = Frank Huang
$1 = Frank
$2 = Huang
和位置变量非常有关系的是命令set ,set 经常被用来根据变量来转换为位置变量：
经过set 命令后 data 的值就给了位置变量可以被shell利用了
e.g
set $(date)
echo the month is $2
commands
: = test
. = current shell
echo 这里再一次提到 \的escapte的作用
eval = $( ... ) [ hsy75] extra the content in brace , means give you the value of the value of a vaiable
exec replace the shell
exit n
export just creat an enviroment variable which can be seen by other scripts
expr (expression evaluate) = $(( ... )) [ hsy75]注意这个和前面的 $(...)取变量内容不同 是double parenthsis ,
printf
return
set 很有用的是将一组变量的field 和位置变量联系起来
set the para variables for the shell
eg
echo the data is $(date)
set $(date)
echo the month is $2
shift
trap
unset
two useful commands
find (search the file or directory) [hsy75]diff from grep which find for strings
syntax
find [path] [options] [tests] [actions]
eg.
$find . -mount \(-newer filename -o -name "_*" \) -type f -exec ls -l {}\; -print
[ hsy75]上面这个find 看起来很难，我归纳一下书上的解释下面一下就其实简单
$find . [path=curernt directory] -mount [options=do not search the other files system] \ [quote the braces using a backslash to escapte the meanings in shell] ( [combine the tester using the parentheses] -newer [tests = newer ] filename [tests newer's pattern means the filename is older than the finding file] -o [tests operations means OR] -name [tests= name] "_*" [tests'name pattern means name started with an underscore ] \) [escapte the parentheses] -type [tests type] f [tests type =file] -exec [actions exec] ls -l [the command that we use invode by action] {} [used for -exec or -ok means the full path to the current file] \; [means the parameters on the line of action is end so it is a termiator ] -print [another actions that to print the result]
grep find 的兄弟，经常一起用 find for string ：said by writer it is quite common to have grep as a command passed after a action -exec when using find
grep [options] Pattern [FILES]
[ hsy75] 看起来grep 是非常简单的，但是，当你开始接触 regular expression 的时候，sophisticated would occurs
regular expressiom
^ anchor to the beginning of a line
$ anchor to the end of a line
. any single character
[ ] range of characters
[:blank:] special match patterns,the blank 可以被其他参数代替
-E extended matching
[hsy75] 下面这些是给-E扩展用，所以必须都用 \来escapte shell 的功能
?
*
+
{n}
{n,}
{n,m}
e.g
[frank@localhost ch02]$ grep Th.[[:space:]] words2.txt
The handle toward my hand? Come, let me clutch thee.
The curtain'd sleep; witchcraft celebrates
Thy very stones prate of my whereabout,
[frank@localhost ch02]$ grep Th [匹配Th] . [匹配附加任何一个字母] [[:space:]] [后面紧跟 空格] words2.txt [被查找的文件名]
[frankhuang@localhost ch02]$ grep -E [a-z]\{9\} words2.txt
Proceeding from the heat-oppressed brain?
And such an instrument I was to use.
Thus to mine eyes. Now o'er the one halfworld
The curtain'd sleep; witchcraft celebrates
Pale Hecate's offerings, and wither'd murder,
With Tarquin's ravishing strides, towards his design
Thy very stones prate of my whereabout,
[frankhuang@localhost ch02]$ grep -E [扩展支持开启] [a-z] [range of characters] \ [Escape for shell of {] {9 [有9个字符匹配也就是单词长度为9] \} words2.txt
and commands execution
1 $(command) = '\command'\
we often need to capture the result of a command exexution for use in the shell cript ,the above syntax helps put the output of command into a variable .
the ability to put the result of a command into a script. variable is very powerful.
e.g
set $(who)
writer = $(who)
echo $writer
2 $((arithmetic expansion)) = $(expr arithmetic expansion) 注意这里是双括号
Parameter expansion
3 ${variable} 变量拓展和保护
3.1用来保护shell的扩展时候，只扩展对应的变量
e.g.
for i in 1 2
do
creat_file $test_{i}
creat_file $test_i 报错
done
3.2 参数扩展
${param:default}
类似的还有下面变量扩展参数
# 从头删除匹配字符，如果多个匹配，删除最小匹配长度，并返回
## 从头删除匹配字符，如果多个匹配，删除最长匹配长度，并返回
% 从尾巴删除匹配字符，如果多个匹配，删除最小匹配长度，并返回
%% 从尾巴删除匹配字符，如果多个匹配，删除最长匹配长度，并返回
参数扩展，在redirection output的时候非常有用，
since linux are havily round the idea of filters, the result of one operation must oftern be redirected manually.
e.g.
for image in *.gif
do
cjpeg $image > ${image%%gif}jpg
【hsy75】解释扩展如下
cjpeg [a program to convert from gif to jpg] $image [get the file name from the *.gif] > [redirection] ${ [圈定需要替换的变量] image%% [从尾巴删除匹配字符gif ] gif } jpg
done
比如 draw_0001.gif 将变成 draw_0001.jpg
eg.
#!/bin/sh
unset foo
echo ${foo:-bar}
foo=fud
echo ${foo:-bar}
foo=/usr/bin/X11/startx
echo ${foo#*/}
echo ${foo##*/}
bar=/usr/local/etc/local/networks
echo ${bar%local*}
echo ${bar%%local*}
exit 0
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
here document
其实就是像自动命令的输入，可以和编辑器绑定起来，从而实现通过文件读写来直接运行编辑器里面的scripts
e.g
#!/bin/sh
ed [编译器ed] textfile <<!FunkyStuff! [here document的makers]
3 [ed 编译器命令 move to line 3]
d [ed 编译器命令 删除一行]
.,\ [这里避免shell的作用，从而shell执行的时候可以将$后面的东西看成是非变量，从而保证ed的command 正确执行] $s/is/was/ [替换 is 为was]
w
q
!FunkyStuff!
exit 0
This is line 1
This is line 2
This is line 3 被删除
This is line 4 被替换
将变成
This is line 1
This is line 2
This was line 4
debugging
-n check syntax error only
-v Echoes commands before running
-x after running
set -o xtrace
set +o xtrace
其他的调试选项
-u
-o nounset
-o verbose
我们用上面提到的例子再来说明一下如何调试：
1 没有调试：
[frankhuang@localhost ch02]$ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
2 打开set trace调试
[frankhuang@localhost ch02]$ set -o xtrace
++ echo -ne '\033]0;frankhuang@localhost:~/beginning_linux/beginning/ch02'
[frankhuang@localhost ch02]$ ./param
+ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
++ echo -ne '\033]0;frankhuang@localhost:~/beginning_linux/beginning/ch02'
3关闭set trace 调试
[frankhuang@localhost ch02]$ set +o xtrace
+ set +o xtrace
[frankhuang@localhost ch02]$ ./param
bar
fud
usr/bin/X11/startx
startx
/usr/local/etc/
/usr/
4
[frankhuang@localhost ch02]$ sh -x ./param
+ unset foo
+ echo bar
bar
+ foo=fud
+ echo fud
fud
+ foo=/usr/bin/X11/startx
+ echo usr/bin/X11/startx
usr/bin/X11/startx
+ echo startx
startx
+ bar=/usr/local/etc/local/networks
+ echo /usr/local/etc/
/usr/local/etc/
+ echo /usr/
/usr/
+ exit 0
5
[frankhuang@localhost ch02]$ sh -v ./param
#!/bin/sh
unset foo
echo ${foo:-bar}
bar
foo=fud
echo ${foo:-bar}
fud
foo=/usr/bin/X11/startx
echo ${foo#*/}
usr/bin/X11/startx
echo ${foo##*/}
startx
bar=/usr/local/etc/local/networks
echo ${bar%local*}
/usr/local/etc/
echo ${bar%%local*}
/usr/
exit 0
going graphical - dialog utility
图形输入工具，这个再编写大型的shell配置文件时候经常用到
e.g
the doc in under development