Shell and its relation to computer
-
Shell
Simply put, the shell is a program that takes commands from the keyboard and gives them to the operating system to perform. In the old days, it was the only user interface available on a Unix-like system such as Linux. Nowadays, we have graphical user interfaces (GUIs) in addition to command-line interfaces (CLIs) such as the shell.
On most Linux systems a program called
bash(which stands for Bourne Again SHell, an enhanced version of the original Unix shell program,sh, written by Steve Bourne) acts as the shell program. Besidesbash, there are other shell programs available for Linux systems. These include:ksh,tcshandzsh. Shell and Computer
-
What is a Terminal
It's a program called a terminal emulator. This is a program that opens a window and lets you interact with the shell. There are a bunch of different terminal emulators we can use. Some Linux distributions install several. These might include
gnome-terminal,konsole,xterm,rxvt,kvt,nxterm, andeterm. -
Advantage of
ShellUsing up and down arrow to indicate the
previous commandsUsing
tabIntelliSense to get the complete commandPowerful scripting ability
Powerful
envfunctionality
Frequently used commands
-
manpage - If you are new to a command or you forgot the usage of it then you can use the following command to find detailed instruction
man command-name man --help man -hman page frequently used shortcut -
echo - output string or variable value to the terminal
[linuxprobe@linuxprobe ~]$ echo michaelxuan michaelxuan [linuxprobe@linuxprobe ~]$ echo $SHELL /bin/bash -
date - display or configure the system datetime "date [options] [+ specified format]"
man it if you are willing to check the details[linuxprobe@linuxprobe ~]$ date Wed Jan 6 21:09:08 CST 2021 [linuxprobe@linuxprobe ~]$ date "+%Y-%m-%d -- %H:%M:%S" 2021-01-06 -- 21:09:53 [linuxprobe@linuxprobe ~]$ date "+%j" 006 -
reboot - With its literal meaning, this command will only allow superuser
rootto reboot your machine[linuxprobe@linuxprobe ~]$ reboot -
poweroff - With its literal meaning, this command will only allow superuser
rootto shutdown the computer[linuxprobe@linuxprobe ~]$ poweroff -
wget - Download web page resource in the terminal window
man it if you are willing to check the details[linuxprobe@linuxprobe chap2-frequent-commands]$ wget https://www.linuxprobe.com/docs/LinuxProbe.pdf --2021-01-06 21:16:46-- https://www.linuxprobe.com/docs/LinuxProbe.pdf Resolving www.linuxprobe.com (www.linuxprobe.com)... 180.163.121.217, 180.163.121.218, 180.163.121.219, ... Connecting to www.linuxprobe.com (www.linuxprobe.com)|180.163.121.217|:443... connected. HTTP request sent, awaiting response... 200 OK Length: 17676281 (17M) [application/pdf] Saving to: ‘LinuxProbe.pdf.1’ LinuxProbe.pdf.1 100% [===========================================>] 16.86M 22.8MB/s in 0.7s 2021-01-06 21:16:52 (22.8 MB/s) - ‘LinuxProbe.pdf.1’ saved [17676281/17676281] -
ps - View the status of all processes "ps [args]"
[linuxprobe@linuxprobe chap2-frequent-commands]$ ps aux USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.1 244640 13932 ? Ss 20:09 0:02 /usr/lib/systemd/systemd --switched-root --sys root 2 0.0 0.0 0 0 ? S 20:09 0:00 [kthreadd] root 3 0.0 0.0 0 0 ? I< 20:09 0:00 [rcu_gp] root 4 0.0 0.0 0 0 ? I< 20:09 0:00 [rcu_par_gp] root 6 0.0 0.0 0 0 ? I< 20:09 0:00 [kworker/0:0H-kblockd] root 8 0.0 0.0 0 0 ? I< 20:09 0:00 [mm_percpu_wq] -
top - Watch processes and system workload dynamically
# datetime, elapsed time, users, workload(1min, 5min, 15min average) top - 21:55:23 up 1:45, 1 user, load average: 0.00, 0.00, 0.00 # total tasks, running tasks, sleep tasks, stopped and zombie programs Tasks: 361 total, 1 running, 360 sleeping, 0 stopped, 0 zombie # CPU percentage: user, system, processes in which its priority has been changed, idle resource, etc... %Cpu(s): 0.2 us, 0.6 sy, 0.0 ni, 99.1 id, 0.0 wa, 0.1 hi, 0.0 si, 0.0 st # total ram, free ram, used ram, buffered ram MiB Mem : 7790.6 total, 5558.6 free, 1506.5 used, 725.5 buff/cache # virtual ram in total, free virtual ram, used virtual ram, etc... MiB Swap: 2048.0 total, 2048.0 free, 0.0 used. 5998.1 avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 7843 linuxpr+ 20 0 3788576 188864 90900 S 3.3 2.4 0:43.19 gnome-shell 8675 linuxpr+ 20 0 531592 43156 31768 S 1.3 0.5 0:06.15 gnome- terminal- 971 root 20 0 226452 12280 10784 S 0.3 0.2 0:05.51 vmtoolsd -
pidof -Search for the
PIDof one particular service or program "pidof [args] [service-name]"
PIDis a unique value for which it can be used to distinguish different processes.[linuxprobe@linuxprobe chap2-frequent-commands]$ systemctl status sshd ● sshd.service - OpenSSH server daemon Loaded: loaded (/usr/lib/systemd/system/sshd.service; enabled; vendor preset: enabled) Active: active (running) since Wed 2021-01-06 20:09:56 CST; 16h ago Docs: man:sshd(8) man:sshd_config(5) Main PID: 1132 (sshd) Tasks: 1 (limit: 49666) Memory: 2.5M CGroup: /system.slice/sshd.service └─1132 /usr/sbin/sshd -D [email protected],chacha20- [email protected],aes256-ctr,aes256-cbc,aes128- [email protected],aes1> [linuxprobe@linuxprobe chap2-frequent-commands]$ pidof sshd 1132 -
kill - Terminate a particular service or program by its
PID"kill [args] [pid]"[linuxprobe@linuxprobe chap2-frequent-commands]$ sudo kill `pidof sshd` -
killall -Terminate all corresponding processes which relate to the same service "killall [args] [service-name]"
Complex software often consist of many different service programs which causes a lot of trouble if you have to terminate them one by one[root@linuxprobe ~]# pidof httpd 13581 13580 13579 13578 13577 13576 [root@linuxprobe ~]# killall httpd [root@linuxprobe ~]# pidof httpd [root@linuxprobe ~]#
System status checking
-
networking
- ifconfig -check network configuration and its running status "ifconfig [device] [args]"
[root@linuxprobe etc]# ifconfig ens160: flags=4163 -
uname -Check kernel and system version
[root@linuxprobe etc]# uname -a Linux linuxprobe.com 4.18.0-80.el8.x86_64 #1 SMP Wed Mar 13 12:02:46 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux -
Checking of the current system details
probe@linuxprobe chap2-frequent-commands]$ cat /etc/redhat-release Red Hat Enterprise Linux release 8.0 (Ootpa) -
uptime -Check system workload(including current system time, uptime, terminal number and average workload, etc...), the
lower valuethebetter performance[linuxprobe@linuxprobe chap2-frequent-commands]$ uptime 13:49:15 up 3:31, 1 user, load average: 0.00, 0.00, 0.00 -
free -Check the current ram statistics "free [-h]"
[linuxprobe@linuxprobe chap2-frequent-commands]$ free total used free shared buff/cache available Mem: 7977544 1567160 5661552 17956 748832 6115476 Swap: 2097148 0 2097148 # -h indicates the output is human readable [linuxprobe@linuxprobe chap2-frequent-commands]$ free -h total used free shared buff/cache available Mem: 7.6Gi 1.5Gi 5.4Gi 17Mi 731Mi 5.8Gi Swap: 2.0Gi 0B 2.0Gi -
who -Checking of all the logins "who [args]"
[linuxprobe@linuxprobe chap2-frequent-commands]$ who # username device login time linuxprobe tty2 2021-01-06 20:15 (tty2) -
last -Checking of login record "last [args]"
[linuxprobe@linuxprobe chap2-frequent-commands]$ last linuxpro tty2 tty2 Wed Jan 6 20:15 gone - no logout reboot system boot 4.18.0-80.el8.x8 Wed Jan 6 20:09 still running linuxpro tty2 tty2 Tue Jan 5 21:54 - ...login records are all stored within the local machine in log file format, it is not correct to which just simply believe the output of this command since hackers are smart enough to hijack those files -
history -Display a list of previously used commands "history [-c]"
[linuxprobe@linuxprobe chap2-frequent-commands]$ history 1 history 2 clear 3 dir 4 man history 5 historyModify the /etc/profile HISTSIZE variable for which it controls the length of your command records
history -c- clear all the records
!codeNum- execute the corresponding command again
all the historical commands would be saved in the .bash_history file /home path sosreport -Collection of the system configuration and structure information then output the analytic document
Change Working Dir
pwd - print name of current/working directory
-
cd
cd: cd [-L|[-P [-e]] [-@]] [dir]
Change the shell working directory.cd ~ # to Home cd / # to Root cd - # to Previous Path -
ls - list directory contents
[root@linuxprobe chap2]# ls -ld tes_dir/ drwxr-xr-x. 2 root root 6 Jan 9 21:52 tes_dir
Text file Manipulation
- cat - concatenate files and print on the standard output
man it for more info[root@linuxprobe chap2]# cat wget-log -n 1 2 --2021-01-05 22:19:39-- https://www.linuxprobe.com/wp-content/themes/zanblog2_1_0/ui/font-awesome/fonts/fontawesome-webfont.woff?v=4.0.1 3 Reusing existing connection to www.linuxprobe.com:443. 4 HTTP request sent, awaiting response... 200 OK - more - file perusal filter for crt viewing
view the text content like watching video tape
less - similar funcationaity to more, but it can move backward - head - output the first part of files
tail - output the last part of files
headandtailare similar functionalities but in reversed order,tailis more impressive for which it can refresh the page constantlyif the text content has been editing by someone[root@linuxprobe chap2]# tail -f /var/log/messages Jan 9 21:39:30 linuxprobe NetworkManager[1108]:[1610199570.2630] device (ens160): carrier: link connected Jan 9 21:39:30 linuxprobe kernel: usb 2-2.1: reset full-speed USB device number 4 using uhci_hcd - tr - translate or delete characters
it actually can be used for replacing the text content[root@linuxprobe chap2]# cat tr_test 1. hello world 2. world hello 3. c, cplusplus, java, csharp, python, javascript, etc.. 4. computer science, machine learning, data structure, mathematic algorithm, etc... 5. statistics, data management... [root@linuxprobe chap2]# cat tr_test | tr [a-z] [A-Z] 1. HELLO WORLD 2. WORLD HELLO 3. C, CPLUSPLUS, JAVA, CSHARP, PYTHON, JAVASCRIPT, ETC.. 4. COMPUTER SCIENCE, MACHINE LEARNING, DATA STRUCTURE, MATHEMATIC ALGORITHM, ETC... 5. STATISTICS, DATA MANAGEMENT... - wc - print newline, word, and byte counts for each file
man it for more info[root@linuxprobe chap2]# wc -cmlw tr_test 5 28 205 205 tr_test - stat - display file or file system status
[root@linuxprobe chap2]# stat tr_test File: tr_test Size: 205 Blocks: 8 IO Block: 4096 regular file Device: fd00h/64768d Inode: 780869 Links: 1 Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root) Context: unconfined_u:object_r:admin_home_t:s0 Access: 2021-01-09 22:15:28.145635106 +0800 Modify: 2021-01-09 22:15:09.425636045 +0800 Change: 2021-01-09 22:15:09.425636045 +0800Access - the latest time for which the file has been used or accessed
Modify - the lastest time for which the file content has been edited
Change - the lastest time for which the file property has been edited - cut - remove sections from each line of files
extract data along the cols
[root@linuxprobe chap2]# cat cut_tes
a $ b $ c $ d $ e
c $ c++ $ java $ python $ javasript
[root@linuxprobe chap2]# cut -d$ -f5 cut_tes
e
javasript
- diff diff - compare files line by line
[root@linuxprobe chap2]# cat diff_tes*
a $ b $ c $ d $ e
c $ c++ $ java $ python $ javasript
a $ b $ c $ d $ e
c++ $ c $ javascript $ python $ java
[root@linuxprobe chap2]# diff --brief diff_tes*
Files diff_tes0 and diff_tes1 differ
[root@linuxprobe chap2]# diff -c diff_tes*
*** diff_tes0 2021-01-09 22:43:08.078551832 +0800
--- diff_tes1 2021-01-09 22:44:00.429549206 +0800
***************
*** 1,2 ****
! a $ b $ c $ d $ e
! c $ c++ $ java $ python $ javasript
--- 1,2 ----
! a $ b $ c $ d $ e
! c++ $ c $ javascript $ python $ java
File and Dir manipulation
- touch change file timestamps
it can also be used to create new file
-a change only the access time
-m change only the modification time
-d, --date=STRING # change both access and modification time
parse STRING and use it instead of current time
- mkdir - make directories
-p, --parents
no error if existing, make parent directories as needed
[root@linuxprobe chap2]# mkdir mkdir_test; cd mkdir_test;
[root@linuxprobe mkdir_test]# mkdir -p a/b/c/d/e/f/g
[root@linuxprobe mkdir_test]# ll
total 0
drwxr-xr-x. 3 root root 15 Jan 9 22:49 a
[root@linuxprobe mkdir_test]# tree
.
└── a
└── b
└── c
└── d
└── e
└── f
└── g
7 directories, 0 files
- cp - copy files and directories
if the target is dir, copy the files to the dir;
if the target is file, asking for replacement??
if the target does not exist, execute the copy action
-p retain origianl file property
-d if the target is a "link file", retain the "link" peroperty
-r copy target recursively
-i if target is already exist, asking for replacement
-a equivalent to -pdr
- mv - move (rename) files
- rm - remove files or directories
rm -fR dirname
rm -f /* # !!!WAARNING
- dd - convert and copy a file
if=FILE read from FILE instead of stdin
of=FILE write to FILE instead of stdout
bs=BYTES read and write up to BYTES bytes at a time (default: 512); overrides ibs and obs
count=N copy only N input blocks
# mout the removable or target device
mount /dev/sdb5 /mnt
# start copying the data to the removable or target device
dd if=/dev/sda of=/mnt/backup_sda.img
# copy /dev/zero to target 560_file total counts 560M
[root@linuxprobe ~]# dd if=/dev/zero of=560_file count=1 bs=560M
[root@linuxprobe dd_test]# file /dev/zero
/dev/zero: character special (1/5)
[root@linuxprobe chap2]# cd dd_test/
[root@linuxprobe dd_test]# dd if=/dev/zero of=./dd_test count=1 bs=1M
1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.000598115 s, 1.8 GB/s
[root@linuxprobe dd_test]# tree
.
└── dd_test
0 directories, 1 file
[root@linuxprobe dd_test]# file dd_test
dd_test: data
[root@linuxprobe dd_test]# vim dd_test
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
# create iso file
[root@linuxprobe ~]# dd if=/dev/cdrom of=RHEL-server-7.0-x86_64-LinuxProbe.Com.iso
- file file — determine file type
[root@linuxprobe dd_test]# file /dev/nvme0n1p1
/dev/nvme0n1p1: block special (259/1)
[root@linuxprobe dd_test]# file /dev/zero
/dev/zero: character special (1/5)
compress and uncompress
- tar - an archiving utility
man it for more info
-c create compressed file
-x uncompress the file
-t check the compressed file content
-z gzip compress or uncompress
-j bzip2 comress or uncompress
-J xz compress or uncompress
-v verbose display
-f target file
-p retain the original property
-P --absolute-names
Don't strip leading slashes from file names when creating archives.
[root@linuxprobe tar_test]# ll
total 8
-rw-r--r--. 1 root root 108 Jan 9 23:22 tar_test
-rw-r--r--. 1 root root 108 Jan 9 23:21 tar_test.file
[root@linuxprobe tar_test]# tar -cJvf tar_test.xz tar_test.file
tar_test.file
[root@linuxprobe tar_test]# file tar_test.xz
tar_test.xz: XZ compressed data
[root@linuxprobe tar_test]# tar -xJvf tar_test.xz
tar_test.file
tar -cJvf ????.xz source_file # xz
tar -czvf ????.gz source_file # gzip
tar -cjvf ????.bz2 source_file # bzip2
tar -xJvf ????.xz source_file # xz
tar -xzvf ????.gz source_file # gzip
tar -xjvf ????.bz2 source_file # bzip2
- grep
grep, egrep, fgrep - print lines matching a pattern
-b search binary file as text file
-c count the line of data which has been found
-i ignore case
-n display line number
-v search in reversed order, display only the content other than the search pattern
[root@linuxprobe grep_test]# cat grep_examle.txt
1. python
2. java
3. python framework -- flask
[root@linuxprobe grep_test]# vim grep_examle.txt
[root@linuxprobe grep_test]# grep -iR --color python
grep_examle.txt:1. python
grep_examle.txt:3. python framework -- flask
# find the users who are not allowed to login on the current machine
[root@linuxprobe ~]# grep /sbin/nologin /etc/passwd
bin:x:1:1:bin:/bin:/sbin/nologin
daemon:x:2:2:daemon:/sbin:/sbin/nologin
adm:x:3:4:adm:/var/adm:/sbin/nologin
lp:x:4:7:lp:/var/spool/lpd:/sbin/nologin
mail:x:8:12:mail:/var/spool/mail:/sbin/nologin
- find - search for files in a directory hierarchy
-name target name
-perm target permission(modeexact sameand -modeinclude)
-user File is owned by user uname (numeric user ID allowed).
-group File belongs to group gname (numeric group ID allowed).
-n +nless than or more than n days
-mtime matching modification time
-atime matching access time
-ctime matching file property changed time
-nouser matching files which are owned by no user
-nogroup matching files which are owned by no group
-newer f1 !f2 matching files newer than f1 but not f2
--type b/d/c/p/l/f
matching file type block/dir/character/pipe/link/text file
-size matching specified file size
-prune ignore particular directory
-exec .... {} ; execute some commands after the find command
find xxx -exec yyy {} \;
find xxx the files and put it into {} and then for each file do the yyy commands and remember the \; must be used at the end in order to indicate the execution is ended
[root@linuxprobe find_example]# tree
├── results
│ └── test
└── test1.txt
2 directories, 1 file
[root@linuxprobe find_example]# find -name "test1.txt" -exec cp -a {} ./results/test \;
cp: './results/test/test1.txt' and './results/test/test1.txt' are the same file
[root@linuxprobe find_example]# ls results/test/
test1.txt