Linux通过日志文件统计IP访问次数排序,取前n条

日志内容如下:

1.119.144.106 - - [03/Jan/2019:12:47:50 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:50 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:51 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:51 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.000" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:51 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:51 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.000" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:52 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:53 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.001" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
1.119.144.106 - - [03/Jan/2019:12:47:53 +0800] image.aa.com "GET /image/public/index.php/index/uploads/handle HTTP/1.1  0.000" 500 5 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.110 Safari/537.36" -
123.125.71.82 - - [03/Jan/2019:12:47:57 +0800] image.aa.com "GET /Uploads/150827/55dee28ce9dbf.jpg HTTP/1.1  0.211" 200 509831 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" -
220.181.108.169 - - [03/Jan/2019:12:48:03 +0800] image.aa.com "GET /Uploads/160622/576a046300906.jpg HTTP/1.1  1.567" 200 49965 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" -
115.239.212.137 - - [03/Jan/2019:12:48:15 +0800] image.aa.com "GET /image/public/uploads/170109/587398dc20d79.JPG HTTP/1.1  0.362" 200 985944 "-" "Mozilla/5.0 (Windows NT 5.1; rv:8.0.1) Gecko/20100101 Firefox/8.0.1" -
123.125.71.114 - - [03/Jan/2019:12:48:31 +0800] image.aa.com "GET /Uploads/170505/590c2d515b50a.jpg HTTP/1.1  1.406" 200 25317 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" -
220.243.135.80 - - [03/Jan/2019:12:48:38 +0800] image.aa.com "GET /image/public/uploads/171129/5a1ec9255a89e.jpg HTTP/1.1  0.108" 200 39174 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36" -
220.243.135.143 - - [03/Jan/2019:12:48:38 +0800] image.aa.com "GET /Uploads/170804/5984129b7b4fb.jpg HTTP/1.1  0.172" 200 33351 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36" -
220.243.135.140 - - [03/Jan/2019:12:48:49 +0800] image.aa.com "GET /Uploads/161222/585b9f24c4a99.JPG HTTP/1.1  0.223" 200 381442 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36" -
112.80.254.51 - - [03/Jan/2019:12:48:52 +0800] image.aa.com "GET /Uploads/170806/5985f7ac2fb94.jpg HTTP/1.1  0.179" 200 72244 "-" "Mozilla/5.0 (Windows NT 5.1; rv:8.0.1) Gecko/20100101 Firefox/8.0.1" -
123.125.71.109 - - [03/Jan/2019:12:48:57 +0800] image.aa.com "GET /Uploads/170516/591ac3201f954.jpg HTTP/1.1  3.026" 200 76574 "-" "Mozilla/5.0 (compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)" -
220.243.136.128 - - [03/Jan/2019:12:49:24 +0800] image.aa.com "GET /image/public/uploads/170330/58dc7a95a5a70.jpg HTTP/1.1  0.065" 200 59287 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36" -
220.243.135.128 - - [03/Jan/2019:12:49:24 +0800] image.aa.com "GET /image/public/uploads/170321/58d0c185e5668.jpg HTTP/1.1  0.073" 200 102566 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36" -

第一列是IP地址,现在我想根据IP统计它的数量,就可以写一个shell命令如下:

cat logs/xxx.log | awk '{print $(1)}' | sort | uniq -c | sort -k 1 -n -r|head -10

解释一下上面的命令,

cat logs/xxx.log就是输出要统计的日志。

awk后面跟一个指令,awk '{print $(1)}'就是打印出日志内容的第几列。$1就是第一列

扩展:$(NF)是总列数,那么如果根据倒数第二列统计,就是$(NF-1)。

sort就是对内容进行排序,默认是自然顺序排序。

uniq指令用于排重,而是只适用于相邻两行相同的情况。所以一般结合sort使用。即先sort排序再排重。

uniq -u是只显示唯一的记录行。uniq -c是显示有重复记录的情况。

sort -k 1 -n -r这个指令,参看下面sort指令参数的详细说明

sort选项与参数:
-f  :忽略大小写的差异,例如 A 与 a 视为编码相同;
-b  :忽略最前面的空格符部分;
-M  :以月份的名字来排序,例如 JAN, DEC 等等的排序方法;
-n  :使用『纯数字』进行排序(默认是以文字型态来排序的);
-r  :反向排序;
-u  :就是 uniq ,相同的数据中,仅出现一行代表;
-t  :分隔符,默认是用 [tab] 键来分隔;
-k  :以哪个区间 (field) 来进行排序的意思

所以 sort -k 1 -n -r 指令的意思就是对第一列按照纯数字逆序排序。

这个纯数字是哪里来的呢?是uniq -c来的,原来剩下一列就是IP了,当执行uniq -c指令时,它会统计重复记录的次数并把这次数显示在第一列。所以现在有两列了,第一列是重复次数,第二列是IP。所以这里是按照重复次数排序。

head -10这个不用说了吧,显示前10行。
 

你可能感兴趣的:(服务器,linux)