nginx 禁止蜘蛛抓取

if ($http_user_agent ~* (baiduspider|googlebot|soso|bing|sogou|yahoo|sohu-search|yodao|YoudaoBot|robozilla|msnbot|MJ12bot|NHN|Twiceler)) {
            return   403;
        }

你可能感兴趣的:(蜘蛛,禁止,niginx)