一夜之间,sogou和百度收录了站点
观察了一下百度的收录,今天放出的是9.4的收录和9.11号的收录
sogou在前几天收录了本站,大约会是收录两天之后在搜索结果页中显示出来,就是会延迟,收录首页之后就会慢慢的来搜其它页面了
google的收录持续的在减少从最初的4000多到现在的1000多,但是还是有在更新页面,因为我改了title
今天晚些时候,具体是正好是零晨,我在查qq上的排名是发现由第2名降到第10页了,这是由于之前我的失误,输出了一个后台的端口页面,之后使用了301重定向,今天弱于将其纠正到正确的url上来了,早上还排在第二的,到晚上就到第十页了,索引显示是9-19号的,看来从抓取到更新大概用了一天的时间,但是之前重定向可是花了好几天的时间
看了一下日志,好像是14更新进行的后台301重定向,看来花了5-6天的时间
这两天sogou的bot来势凶猛,来吧来吧。大概每天爬400多页
这两天500错误控制在70左右,今天是50多,
这个500里也包括一些少数几个乱七8糟的请求比如 /issue/1689 诸如此类,不知道是怎么搞的?
可能和换了wsgi server到gevent有关,但相比以前算是稳定多了,另外还经常报500的/robots.txt找不到, 这个我有点怀疑是网络或是nginx的问题,因为并不是什么时候都报这个错,也可能请求的本身的次数就少,暂记
发现日志中有
61.152.96.118 - - [20/Sep/2010:04:52:45 +0800] "GET / HTTP/1.1" 200 6526 "-" "Oray HttpStream 1.0" "-"
112.65.243.56 - - [20/Sep/2010:04:52:46 +0800] "GET / HTTP/1.1" 200 6526 "-" "Oray HttpStream 1.0" "-"
原来是https://www.oray.com的监控,之前设置的,感觉比监控宝好,监控宝太花不实在
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "GET /pma/scripts/setup.php HTTP/1.1" 500 186 "-" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "GET /phpmyadmin/scripts/setup.php HTTP/1.1" 500 186 "-" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "GET /scripts/setup.php HTTP/1.1" 500 186 "-" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "GET /mysql/scripts/setup.php HTTP/1.1" 500 186 "-" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "POST /pma/scripts/setup.php HTTP/1.1" 500 186 "74.207.245.30" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "POST /scripts/setup.php HTTP/1.1" 500 186 "74.207.245.30" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "GET /phpMyAdmin/scripts/setup.php HTTP/1.1" 500 186 "-" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "POST /phpmyadmin/scripts/setup.php HTTP/1.1" 500 186 "74.207.245.30" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:26 +0800] "POST /mysql/scripts/setup.php HTTP/1.1" 500 186 "74.207.245.30" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"
64.15.159.169 - - [20/Sep/2010:07:46:27 +0800] "POST /phpMyAdmin/scripts/setup.php HTTP/1.1" 500 186 "74.207.245.30" "Mozilla/4.0 (compatible; MSIE 6.0; MSIE 5.5; Windows NT 5.1) Opera 7.01 [en]"