DNSBL/Open Proxy-Blocking

History

I've had some problems with abuse via open proxies for a time ago, and when we were banning abusers, they always found a new proxy to use and register new accounts with. Since this forum was a large type we could'nt just ban e-mails etc just like that, because this was leading to a very large amount of other banned users too.

At IRC, in the other hand, we had Open Proxy Monitors, that was banning everything that was blacklisted in some DNSBL-databases. No spammers had a chance to get in there as long they were listed in such database.

This is a plugin that blocks blacklisted hosts from some different DNSBL's. It uses the global_start-hook, a very simple handler for blocking proxies, and a vBphrase called OPM_Deny.


April 2006

The source has been rewritten a bit. The proxychecker is now using a cache that, by default, stores all ip's in a database for 6 hours. It scans some DNSBL's and can be configured to block proxies from bitmasks (defined in the plugin) which makes it a little bit more reliable, because it does'nt block everything it see).

Configuration is made from the plugin (hopefully there will be a nice admin interface in the future). Exceptions (ip's that can pass through this system even if it is a proxy) are also handled differently now.

// CHANGES
//
// 2008-09-20 (2.0.8)
//
// * Changed the routines for how to handle inclusion/exclusions
// * Splitted up plugins for 3.5/3.6 and 3.7
//
// 2007-08-05
//
// * Fixed reported bug, based on resolved hosts ending with 127
// * Changed database-tables to get rid of (hopefully) duplicate keys
// * Added resolver-function
// * Added two new block-methods available at the efnet-rbl
//
// 2006-06-28 (2.0.6/Another fix)
//
// * Proxyinclusions/exclusions didn't work properly
//
// 2006-06-28 (2.0.5/Fix only)
//
// * Fixed a bug in the $block-array that affected some of the blocking results
//
// 2006-06-28 (2.0.4)
//
// * opm.tornevall.org has a new entry for anonymizers, added support for this
// * Default value on "block everything detected" in plugin changed to "no"
//
// 2006-06-26 (2.0.3)
//
// * Created options for admincp (removed plugin-configuration)
// * Fixed a bit-bug for njabl
// * Plugin is now a function (rbl_livecheck) for external lookups
// * Added options for "only block on newuser-registrations"
//
// 2006-06-22 (2.0.3 RC)
//
// * The monitor is now a function
// * Added small compatibility with other plugins (with return)
//
// 2006-05-13
//
// * sorbs zones added (no bitmasking)
// * opm.blitzed.org removed
// * time() changed to TIMENOW
//
// 2006-04-21
// ==========
//
// * proxyinclusions
// quickly add own hosts that should be treated as a proxy
//


How does it work with other vBulletins?

This filter actually works with both 3.5 and 3.6, but for now, they will be separate versions, but for 3.5 and 3.6 you should look here and for 3.7 you should look here.


How to use the compatibility thing

If you have a plugin that you want to use together with the proxy monitor (only returns a value if a an ip-address is registered as a proxy or not) you can call the function rbl_livecheck like this (example):

PHP Code:

global $rblInstalled;

if ($rblInstalled) {
$remoteIsProxy = rbl_livecheck(1, $_SERVER['REMOTE_ADDR']);

//
// .. your code here ..
//

} 


Report bugs if you find them...



Don't forget to install it

[MaTraX]

Could someone remake a bit @eoc_Jason code into produck so it could be installed and uninstalled right from the ACP? Thank you

ps.
I think that this option when will be done and working on 100% should be included as a standard option.

[eoc_Jason]

I actually modifed it even more from what I posted for my forum, only to run the check on specific user groups.

I'll probably futz with it some more today, to make it an actual "product" for my site.

TMM-TT (or anybody) can use the file I uploaded and expand upon it however they want. I don't want to step on anybody's toes with this hack, which is why I'm offering for someone else to manage it. However if nobody else wants to step up to the plate I am more than willing to do it.

EDIT - Here is an update from the previous page. This is released as a "product" so that it adds the database field & phrase and everything automatically.

[thedvs]

You have some typo errors at the bottom in the msg but thats no big deal.

Installed & tested it with my Hide IP platinum application and it blocks the proxies it uses that are blacklisted

I am extremely impressed. Well done eoc Jason you are a star.

I personally think this has to be the hack of the month!

[SmartGnome]

Jason you earn a price

Installed and tested with several proxy proggies and it works fine !!!

Many thanks from me and the mods of my forum.


Ad

PS:

Perhaps it's better to start a new thread or edit the first post (mister moderator) while things are different now then mentioned in the first post IMHO.

[eoc_Jason]

lol, I just noticed there was a type in the OPM_Deny phrase, it's pseudo engrish. Oh well. I'm sure people will customize it to meet their needs anyways.

Like I said, you can have it check other RBL's, and set only the returned error code that you want (if they return multiple), it will stop after the firs the blacklisted found. However I think spamhaus is enough since it is a combination of the 3 big ones out there.

You can additionally test it if you edit your session table directly and set your session OPM = 2. (I had a heck of a time finding an open proxy!).

I've found that about 1 or 2 out of the current users online is marked as an open proxy (I have about 250-400 people online at a time usually). Some are false positives though as I got a few complaints and when I checked them out, they were entries that haven't been tested in weeks or months!

I have several usergroups which people get auto-promoted, so only my guests, and lowest registered usergroup actually get checked, which I might even allow guests too, not sure at this moment in time.

If you only want to check certain usergroups, you can use the following IF statement around the two function calls.

PHP Code:

if (!in_array($vbulletin->userinfo['usergroupid'],array(1,2,3,etc...)))
{
    // The two session OPM IF statements would go here.
} 


[skydancer]

Nice work! I had kludged something together earlier, but a plugin is a lot better. I've installed it and will test it out

I'm not sure I agree with the comments on caching. This plugin uses gethostbyname() to do a DNS request to the blocklist servers. This DNS request will be issued to the (colo provider's) DNS cache and that server will already cache the answer for you. This server will keep it in the cache as long as the TTL of the record in the blocklist, so you are caching it as long as the blocklist specifies it, which is often hours or days, and you have the best chance of having an up to date result, because your DNS takes care of caching and retrieving.

Caching would also make the code more complex and prone to errors, also it costs extra time for a feature that DNS already provides. A network DNS lookup is also often faster than a database table lookup and does not stress the disk.

When you check one blocklist, the network load is comparable to doing HostnameLookups On in Apache, which is minimal for all but the largest sites. If needed, I would advise checking a composite blocklist that mirrors many other lists, so you only spend one DNS packet in and out.

[cnutter]

Originally Posted by eoc_Jason

I actually modifed it even more from what I posted for my forum, only to run the check on specific user groups.

I'll probably futz with it some more today, to make it an actual "product" for my site.

TMM-TT (or anybody) can use the file I uploaded and expand upon it however they want. I don't want to step on anybody's toes with this hack, which is why I'm offering for someone else to manage it. However if nobody else wants to step up to the plate I am more than willing to do it.

EDIT - Here is an update from the previous page. This is released as a "product" so that it adds the database field & phrase and everything automatically.

eoc_Jason

Doesnt look like anyone wants to step up... Anyway, great product and its already working and has stopped a few of my problem childern on my site. Though my only conplaint was it blocked one of my regulars who isnt even on a open proxy BUT who's ISP has lots of blacklisted IPs because of spam/proxy abuses. He is using a static ip so he is having to submit it to spamhaus and see if they will remove it. Plus he is contacting his ISP about the issue...

As for any future updates you or anyone might make on this product it would be nice to have a exculed IP list via your admincp. Basicly, a nice little field were I could add ips that should be exempt from the blacklist access block even if they are blacklisted via spamhaus or some other list you might be using via this product. But then again one user having to deal with this is all that big of deal....

Anyway, thanx again for a great product.

[ImportPassion]

Originally Posted by eoc_Jason

EDIT - Here is an update from the previous page. This is released as a "product" so that it adds the database field & phrase and everything automatically.

AWESOME!! I have this one idiot that keeps coming back saying crap like "u can't get rid of me" blah blah. I hope this does, at least it is a step in the right direction.

[eoc_Jason]

Skydancer - I agree with your statements about how DNS "should" cache the info, however, not knowing what all RBLs set their TTL to, or also any mis-configurations, having one little bit of info stored locally makes sense (to me).

The current RBL lookup in the code is spamhaus' list, which is actually a combined open proxy list from 3 sources. So you get a 3-fer-1 in the query.

Lastly, the session info for vB is a HEAP table (meaning it's only stored in memory, not on the HD). vB does a "SELECT *" from the table when doing it's inital session stuff, so there is no added overhead in that case.

Like I said in a previous post, the worst-case scenario that I can think of is if the RBL goes does or does not respond in a timely fashion. I did not put in any timeout / exception handing.

Originally Posted by skydancer

Nice work! I had kludged something together earlier, but a plugin is a lot better. I've installed it and will test it out

I'm not sure I agree with the comments on caching. This plugin uses gethostbyname() to do a DNS request to the blocklist servers. This DNS request will be issued to the (colo provider's) DNS cache and that server will already cache the answer for you. This server will keep it in the cache as long as the TTL of the record in the blocklist, so you are caching it as long as the blocklist specifies it, which is often hours or days, and you have the best chance of having an up to date result, because your DNS takes care of caching and retrieving.

Caching would also make the code more complex and prone to errors, also it costs extra time for a feature that DNS already provides. A network DNS lookup is also often faster than a database table lookup and does not stress the disk.

When you check one blocklist, the network load is comparable to doing HostnameLookups On in Apache, which is minimal for all but the largest sites. If needed, I would advise checking a composite blocklist that mirrors many other lists, so you only spend one DNS packet in and out.

cnutter - Yes, I had the same issue with a couple of my members getting dumped with bad dynamic IPs. However, all the sites that spamhaus uses has extremely simple (one-click removal) links, which will remove the IP and by the next update (hourly) should no longer be blocked. An IP exclusion would not work for most people as they have dynamic IPs. Instead you might want to consider doing it by usergroups, as I posted a snip of code above about bypassing the check for certain groups.

[TMM-TT]

Regarding the "Stepping on someones toes"; it's ok with me if you step on my toes, because I needed help with this plugin from the start myself. I knew that you guys in here could help me with this one, to make it better. So any ideas are still welcome. My RBL-DNS opm.tornevall.org is free to use as long as it exists, and will be continuously updated with new proxies, and old proxies will be marked as "not working" when they are not reachable anymore.

The values for working and not working are:

PHP Code:

$bit_working = 2;
$bit_blitzed = 4;
$bit_timeout = 8;
$bit_error = 16; 


.. and the blitzed-one is used if the proxies are matched with the blitzed database (Currently around 4200 out of 28300 registered proxies has been matched to their database).

By the way, thanks for all help.