UNIX Workshop Notes
UNIX Workshop Notes
1 The UNIX Philosophy
- small is beautiful
- make each program do one thing well
- build a prototype as soon as possible
- choose portability over efficiency
- store numerical data in flat files
- use software leverage to your advantage
- use shell scripts to increase leverage and portability
- avoid captive user interfaces
- make every program a filter
The Ten Lesser Tenets
- allow the user to tailor the environment
- make operating system kernels small and lightweight
- use lower case and keep it short
- save trees
- silence is golden
- think parallel
- the sum of the parts if greater than the whole
- look for the ninety percent solution
- worse is better
- think hierarchically
2 /etc/passwd
The /etc/passwd file determines which shell takes effect during your interactive UNIX session.
/etc/passwd also contains account info, which includes UID and GID.
3 init process
Each process can have one or more child process. If you kill the parent of a child process, it automatically becomes the child of init.
init is the last process created at boot time. It always has a process ID (PID) of 1 (i.e. on all Unixes). init is responsible for starting all subsequent processes. Consequently, it is the parent (in other words, mother of all processes!) of all (non-dummy) Unix processes.
4 Address Space
For each new process created, the kernel sets up an address space in memory. This address space consists of the following logical segments:
• text - contains the program's instructions.
• data - contains initialized program variables.
• bss - contains uninitialized program variables.
• stack - a dynamically growable segment, it contains variables allocated locally
and parameters passed to functions in the program.
5 Process
<!--[if !supportLists]-->1) <!--[endif]-->Context of Process & Context Switching
The context of a process is essentially a snapshot of its current runtime environment, including its address space, stack space, etc. At any given time, a process can be in user-mode, kernel-mode, sleeping, waiting on I/O, and so on. The process scheduling subsystem within the kernel uses a time slice of typically 20ms to rotate among currently running processes. Each process is given its share of the CPU for 20ms, then left to sleep until its turn again at the CPU. This process of moving processes in and out of the CPU is called context switching. The kernel makes the operating system appear to be multi-tasking (i.e. running processes concurrently) via the use of efficient context-switching.
<!--[if !supportLists]-->2) <!--[endif]-->Process status:
R - runable, currently on the run queue
S - currently sleeping (waiting on a pipe, completion of I/O, terminal input, memory)
T - currently stopped
Z - defunct or zombie (a process with no swappable image but a structure in memory)
<!--[if !supportLists]-->3) <!--[endif]-->Signal:
SIGHUP 1 Hangup
SIGINT 2 Interrupt
SIGKILL 9 Kill (cannot be caught or ignore)
SIGTERM 15 Terminate (termination signal from SIGKILL)
6 Boot and Shutdown UNIX
(1) Boot System
Boot scripts are normal shell scripts. The boot scripts will execute by “init” process through sh. Boot scripts’ task include set computer name, set timezone, check hard disk with fsck, install file system, configure network interface, start daemon process and network services, start log and disk quota etc.
BSD doesn’t have run level. ATT System V has run level define in /etc/inittab.
(2) Shutdown System
<!--[if !supportLists]-->· <!--[endif]-->Poweroff ( !!! may damage system )
<!--[if !supportLists]-->· <!--[endif]-->Shutdown ( good way )
<!--[if !supportLists]-->· <!--[endif]-->halt ( will change system to single user mode )
<!--[if !supportLists]-->· <!--[endif]-->reboot
<!--[if !supportLists]-->· <!--[endif]-->init or telinit ( change the run level )
7 su V.S. su -
”su <account>” only change to another user’s account. “su – <account>“ will excute
user’s profile while changing.
“su” or “su –“ without any account parameter means you hope to change to
superuser(root).
8 Unix Environment
<!--[if !supportLists]-->1) <!--[endif]-->setup user environment in ksh
<!--[if !supportLists]-->· <!--[endif]-->$HOME/.profile
<!--[if !supportLists]-->· <!--[endif]-->shell variables: varname=value
<!--[if !supportLists]-->· <!--[endif]-->unset variables
<!--[if !supportLists]-->· <!--[endif]-->aliases
<!--[if !supportLists]-->· <!--[endif]-->options
Any variable can become an environment variable with the command: export varnames.
environment file will told sub process other variables, options, aliases that shell wish to told it. Environment file name can be any name, as long as you set the environment variable ENV to the file's name.
Alias is a synonym for a command or command string. You define an alias by entering (or adding to your .profile) a line with the following form: alias new=original
Options let you change the shell's behavior. The basic commands that relate to options are set -o optionnames and set +o optionnames, where optionnames is a list of option names separated by blanks. the - turns the named option on, while the + turns it off.
<!--[if !supportLists]-->2) <!--[endif]-->Built-in Variables
- PS1, PS2, PS3 , PS4
- TERM
- PATH, MANPATH
- HOME
- PWD
- HISTFILE
- EDITOR
- VISUAL
9 Common Control Characters
<!--[if !supportLists]-->1) <!--[endif]-->Erase Character
<!--[if !supportLists]-->· <!--[endif]-->[BACKSPACE]
<!--[if !supportLists]-->· <!--[endif]-->[DELETE], [<state w:st="on"><place w:st="on">DEL</place></state>] key
<!--[if !supportLists]-->· <!--[endif]-->[CTRL-H]
<!--[if !supportLists]-->2) <!--[endif]-->Other Control Characters
<!--[if !supportLists]-->· <!--[endif]-->[CTRL-U] Erases the whole input line; you can start over.
<!--[if !supportLists]-->· <!--[endif]-->[CTRL-S] Pauses output from a program that is writing to the screen.
<!--[if !supportLists]-->· <!--[endif]-->[CTRL-Q] Restarts output after a pause by [CTRL-S].
<!--[if !supportLists]-->· <!--[endif]-->[CTRL-D] Used to signal end-of-input for some programs.
10 File System Inside
<!--[if gte vml 1]><v:shapetype id="_x0000_t75" coordsize="21600,21600" o:spt="75" o:preferrelative="t" path="m@4@5l@4@11@9@11@9@5xe" filled="f" stroked="f"> <v:stroke joinstyle="miter"/> <v:formulas> <v:f eqn="if lineDrawn pixelLineWidth 0"/> <v:f eqn="sum @0 1 0"/> <v:f eqn="sum 0 0 @1"/> <v:f eqn="prod @2 1 2"/> <v:f eqn="prod @3 21600 pixelWidth"/> <v:f eqn="prod @3 21600 pixelHeight"/> <v:f eqn="sum @0 0 1"/> <v:f eqn="prod @6 1 2"/> <v:f eqn="prod @7 21600 pixelWidth"/> <v:f eqn="sum @8 21600 0"/> <v:f eqn="prod @7 21600 pixelHeight"/> <v:f eqn="sum @10 21600 0"/> </v:formulas> <v:path o:extrusionok="f" gradientshapeok="t" o:connecttype="rect"/> <o:lock v:ext="edit" aspectratio="t"/> </v:shapetype><v:shape id="_x0000_i1025" type="#_x0000_t75" style='width:389.25pt; height:144.75pt'> <v:imagedata src="file:///C:/DOCUME~1/bonny/LOCALS~1/Temp/msohtml1/01/clip_image001.emz" o:title=""/> </v:shape><![endif]--><!--[if !vml]--><!--[endif]-->
super block describes the state of the file system: the total size of the partition, the block size, pointers to a list of free blocks, the inode number of the root directory, magic number, etc.
<!--[if gte vml 1]><v:shape id="_x0000_i1026" type="#_x0000_t75" style='width:132.75pt;height:113.25pt'> <v:imagedata src="file:///C:/DOCUME~1/bonny/LOCALS~1/Temp/msohtml1/01/clip_image003.emz" o:title=""/> </v:shape><![endif]--><!--[if !vml]--><!--[endif]-->
11 I/O Redirection
<!--[if gte vml 1]><v:shape id="_x0000_i1027" type="#_x0000_t75" style='width:6in;height:295.5pt'> <v:imagedata src="file:///C:/DOCUME~1/bonny/LOCALS~1/Temp/msohtml1/01/clip_image005.emz" o:title=""/> </v:shape><![endif]--><!--[if !vml]--><!--[endif]-->
xargs [options] [command]
Execute command (with any initial arguments), but read remaining arguments from standard input instead of specifying them directly. xargs passes these arguments in several bundles to command, allowing command to process more arguments than it could normally handle at once. The arguments are typically a long list of filenames (generated by ls or find, for example) that get passed to xargs via a pipe.
Example:
grep for pattern in all files on the home directory /home/foo:
find /home/foo -print | xargs grep pattern > out &
12 Search and Patterns
(1) Searching Commands
<!--[if !supportLists]-->· <!--[endif]-->… find
<!--[if !supportLists]-->· <!--[endif]-->… grep
<!--[if !supportLists]-->· <!--[endif]-->… egrep
<!--[if !supportLists]-->· <!--[endif]-->… fgrep
<!--[if supportFields]><span style='mso-element:field-begin'></span><span style='mso-spacerun:yes'></span>= 1 /* GB3 <span style='mso-element:field-separator'></span><![endif]-->①<!--[if supportFields]><span style='mso-element:field-end'></span><![endif]--> find pathname(s) condition(s)
find descends the directory tree beginning at each pathname and locates files that meet the specified conditions. At least one pathname and one condition must be specified. The most useful conditions include -print (which must be explicitly given to display any output), -name and -type (for general use), -exec and -size (for advanced users), and -mtime and -user (for administrators). Examples:
find $HOME –print
find $HOME –name ‘core’ –exec rm –rf {} /;
<!--[if supportFields]><span style='mso-element:field-begin'></span><span style='mso-spacerun:yes'></span>= 2 /* GB3 <span style='mso-element:field-separator'></span><![endif]-->②<!--[if supportFields]><span style='mso-element:field-end'></span><![endif]--> grep [options] regexp [files]
Search one or more files for lines that match a regular expression regexp. Examples:
List the number of users who use the korn shell:
grep -c /bin/ksh /etc/passwd
List header files that have at least one #include directive:
grep -l '^#include' /usr/include/*
List files that don't contain pattern:
grep -c pattern files | grep :0
(2) Patterns (Regular Expressions )
A regular expression describes a pattern or a particular sequence of characters, although it does not necessarily specify a single exact sequence. Pattern is composed of mixed sequence of characters and metacharacters.
<!--[if gte vml 1]><v:shape id="_x0000_i1028" type="#_x0000_t75" style='width:362.25pt;height:340.5pt'> <v:imagedata src="file:///C:/DOCUME~1/bonny/LOCALS~1/Temp/msohtml1/01/clip_image007.emz" o:title=""/> </v:shape><![endif]--><!--[if !vml]--><!--[endif]-->
13 UNIX Power Tools
(1) Backup & Compression
tar
cpio
dd
compress
uncompress
unzip
gzip/gunzip/zcat
bzip2/bzcat
filename suffix
. .tar
. .Z
. .gz
. .tar.Z
. .tar.gz
. .zip
. .bz; .tar.bz
(2) Text Processing
cut Select columns for display.
join Merge different columns into a database.
paste Merge columns or switch order.
sort Sort or merge files.
tr Translate (redefine) characters.
uniq Find repeated or unique lines in a file.
sed Stream editor, edit one or more files without user interaction
awk Use the pattern-matching program to modify the specified files
(3) Job Control
jobs
fg
bg
ps
kill
(4) Communication and network utilities
ftp
rsh/remsh
$HOME/.rhosts
rup
mailx
$HOME/.forward
ping
netstat
ifconfig