How To Ban IP Addresses and Block Bots with .htaccess?
Got a scraper or spambot permanently appearing in your server logs? Do you have some site which all your bandwidth? Maybe you have a need to ban a user with some concrete IP address? Today we will tell you how to do all of that and more with the help of .htaccess!
Bad Bots Identifying
If you've found out a specific user-agent still appearing in your logs, so you have to decide whether you want to find out what it is, or if simply ban it? Here are some ways to understand:
Try a search like Google or something else.
User Agent Database should be checked.
Search again after heading over to Webmaster World or begin another one thread.
After determination that the bot is the thing, that you would like to block, you have to add it to .htaccess file.
Bots Blocking with the .Htaccess
This illustration, and all of the next examples, you can place at the lowest part of your .htaccess file. If there is no file as .htaccess in the site's root directory, you can make a new one on your own. #get rid of the bad bot
RewriteEngine on RewriteCond %{HTTP_USER_AGENT} ^BadBot RewriteRule ^(.*)$ http://go.away/
So, what are the capabilities of this code? This is easy: this lines will tell webserver to look for any bot whose user-agent string begins with "BadBot". When it finds the bot that looks like that, it redirects them to a defunct site with the name "go.away".
This is a wonderful thing to start with, but what to do if you have a need to block much more bots than one?
#get rid of bad bots RewriteEngine on RewriteCond %{HTTP_USER_AGENT} ^BadBot [OR] RewriteCond %{HTTP_USER_AGENT} ^EvilScraper [OR] RewriteCond %{HTTP_USER_AGENT} ^FakeUser RewriteRule ^(.*)$ http://go.away/
The above code demonstrates the same functionality as before, but this time it is blocking 3 various bots. After the first two bot names write the "[OR]" option: this allows the server to understand there's some more in the list.
Bandwidth Leeches Blocking
There may be some forum that's constantly hotlinking your images, and it's taking all of your bandwidth. You could change the image into something extremely shocking, but in some countries that may cause real problems! You can solve this problem via blocking the site, like that:
RewriteEngine on RewriteCond %{HTTP_REFERER} ^http://.*awfulforum\.com [NC] RewriteRule .* - [F]
This code will propose a 403 Forbidden error to everybody who is trying to hotlink your images on awfulforum.com. At last you will get: visitors of the site will find out a broken image, and your bandwidth will be stolen no longer.
This is the code for blocking numerous sites:
RewriteEngine on RewriteCond %{HTTP_REFERER} ^http://.*awfulforum\.com [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*sample\.com [NC,OR] RewriteCond %{HTTP_REFERER} ^http://.*lastsample\.com [NC] RewriteRule .* - [F]
In the case when there is a need to block hotlinking totally, and avoid any hotlinking your files, look through some articles on exploiting .htaccess to block hotlinkers.
How to Ban An IP Address
If you want to ban the access to your site for some person or bot you should use the following way: just ban IP address, which you do not want to see at your website:
order allow,deny deny from 192.168.44.201 deny from 224.39.163.12 deny from 172.16.7.92 allow from all
This sample demonstrates blocking of 3 various IP addresses. In the case when you desire to block a broad range of IP addresses:
order allow,deny deny from 224.39. deny from 10.0.0. allow from all
The previous code will block any IP address beginning with "224.39." or "10.0.0." from entering your site.
At last, the following code blocking any particular ISP from receiving access:
order allow,deny deny from some-evil-isp.com deny from subdomain.another-evil-isp.com allow from all
Summary on exploiting .Htaccess:
As it was shown above, .htaccess is an incredibly strong tool for monitoring who and what on your website. Due to the fact that this is so strong, it's also rather simple for things to go wrong. If you have some typos or errors in your .htaccess file, the server will give out an Error 500 page in place of demonstrating your own site, so be confident to back up .htaccess file before providing any changes.
If you need more information about .htaccess files, we recommend you to check the Definitive Guide to Mod_Rewrite.