Integration with vBulletin - vB Bad Behavior

/**
* vB Bad Behavior is free software; you can redistribute it and/or modify it under
* the terms of the GNU Lesser General Public License as published by the Free
* Software Foundation; either version 3 of the License, or (at your option) any
* later version.
*
* This program is distributed in the hope that it will be useful, but WITHOUT ANY
* WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
* PARTICULAR PURPOSE. See the GNU Lesser General Public License for more details.
*/

What is vB Bad Behavior?
This is an integration of the Bad Behavior software with vBulletin.

What is Bad Behavior?
Bad Behavior is a PHP-based solution for blocking link spam and the robots which deliver it. Bad Behavior complements other link spam solutions by acting as a gatekeeper, preventing spammers from ever delivering their junk, and in many cases, from ever reading your site in the first place. This keeps your site's load down, makes your site logs cleaner, and can help prevent denial of service conditions caused by spammers.

Visit http://bad-behavior.ioerror.us/ for more.

Features
For more information on the features of Bad Behavior (and subsequently this mod) please go to Bad Behavior's site:

http://bad-behavior.ioerror.us/documentation/benefits/

For features related to the mod itself, please take a look at the screenshots.

This mod should work with the entire 3.x series (well, beginning with 3.5), but it's only been tested on 3.8.x. I'm not sure if this works on vB 4.x yet, as I've not tested it - but if you try it out, let me know!

Installation
1. Extract the contents of the zip file.
2. Upload the contents of the `upload` folder to your forum root.
3. Enter your AdminCP and go to Plugins & Products > Manage Products > [Add/Import Product]
4. Import the product using the `product-vb_badbehavior.xml` file.
5. Configure the mod in AdminCP -> vBulletin Options -> vBulletin Options -> vB Bad Behavior Options

Upgrading

vB Bad Behavior
In many cases, all you'll need to do to upgrade is follow the installation instructions above.

The only difference, will be you'll need to allow the files to overwrite. Also, when re-importing the product file, you'll need to set "Allow Overwrite" to "Yes".

Bad Behavior
Bad Behavior's files are at `/includes/bad-behavior/`. If you wish to update manually go to:

http://bad-behavior.ioerror.us/download/

And download the latest development version. Extract the zip, and upload the contents of `bad-behavior` to `/includes/bad-behavior/` allowing the files to overwrite.

Versions
The current version of Bad Behavior this mod is using is: v2.2.14
The current version of Bad Behavior (development) is: v2.2.14

Changelog
Version 1.0.13, 04/23/2013

• Bad Behavior upgraded to 2.2.14

Version 1.0.12, 12/21/2012 -- Released: 02/05/2013

• Bad Behavior upgraded to 2.2.13
• Added some more ranges to whitelist.ini

Version 1.0.10, 09/09/2012

• Bad Behavior upgraded to 2.2.10

Version 1.0.9, 06/17/2012

• Bad Behavior upgraded to 2.2.7

Version 1.0.8, 06/12/2012

• Bad Behavior upgraded to 2.2.6
• New Setting: EU Cookie

Version 1.0.7, 05/04/2012

• Bad Behavior upgraded to 2.2.3
• Cron/Scheduled Task for automatic log pruning added.

Version 1.0.6, 01/04/2012

• Bad Behavior upgraded to 2.1.15

Version 1.0.5, 05/26/2011

• Added option for bypassing users/members.
• If the visitor is a user, and is in usergroup 5, 6, or 7 (admin/mod/super mod) - Bad Behavior is bypassed.
• Modified bad-behavior core to check for Google Web Preview
  • file edited: /includes/bad-behavior/core.inc.php
• Added a link beside the IP address in the log for WhoIs.

Version 1.0.4, 04/28/2011

• Bad Behavior upgraded to 2.1.13 (fixes search engine block issues)
• Added Paypal/Paypal IPN IP address to the whitelist.
• Added payment gateway file names to the whitelist.

Version 1.0.3, 04/21/2011

• Fix #1: Pruning log doesn't work.
• Fix #3: POST more than two days after GET (added support for BB's javascript)
• Fix #5: Cannot modify header information error (suppressed error in BB's function)
• Implemented #6: Filter per key (new admincp option to list keys not to be shown in log)
• Implemented #9: Show link to member profile (if userid is found in headers, link to profile)

Version 1.0.2, 04/10/2011

• Updated /includes/functions_vb_badbehavior.php to:
  • disable Reverse Proxy if Reverse Proxy Addresses are empty
  • distinguish SQL queries using "SET", for example: SET @@session.wait_timeout = 90 - which is used by BB
  • set "offsite_forms" to false by default, as it's not really needed in vB IMHO, and it can cause problems with certain setups
  • cleaned up the bb2_read_settings() function and fixed a typo in one of the vbulletin options calls
• Updated /includes/whitelist.ini to include the following GOOGLE ranges:
  • 74.125.0.0/16
  • 216.239.32.0/19
  • 209.85.128.0/17
  • 66.102.0.0/20
• Updated /admincp/vb_badbehavior.php
  • Log pruning was pruning all logs, despite what was entered for number of days

Version 1.0.1, 04/06/2011

• Bad Behavior upgraded to 2.1.12
• Changed files:
  • /includes/bad-behavior/core.inc.php
  • /includes/bad-behavior/searchengine.inc.php
• "Verbose" admin option now set to "No" by default.

Version 1.0.0, 04/05/2011

• Initial release.

Screenshots
Screenshots can now be seen at: http://www.secondversion.com/images/vb/vb_badbehavior/

I was running out of room for attachments here on vB.org


Development

https://github.com/ericsizemore/vb_b...ree/master/vb3


Only those who "Mark As Installed" will receive support for this modification.

[error10]

Originally Posted by Alfa1

These are donation / subscription payments that are blocked.
The url for this one relates to my payment module and contains variables relating to payment information.

PayPal has a long history of sending their IPN notifications without a User-Agent. There's nothing I've been able to do to convince them to send a User-Agent except to advise affected people to complain to PayPal. In the meantime you can whitelist their IP addresses.

Originally Posted by Lee G

A bit more playing around and it looks like google gets blocked when reverse proxy is enabled

If this happens, make sure that X-Forwarded-For is actually the header that your load balancer or accelerator is setting when it forwards HTTP requests to your server. If it uses a different header, be sure to change it. You may also need to list the IP address(es) for your load balancer.

If you aren't using a reverse proxy or load balancer, then you should not enable this option.

Originally Posted by Eric

Regarding: POST more than two days after GET

Looks like this is happening if it's been 48hrs + between the screener cookie and a form submission:

Eric, the cookie needs to be refreshed on every page load, especially for logged-in users.

If caching is in use, then the cached pages need to be expired at least every 48 hours.

Originally Posted by Eric

Alfa1, I do apologize, but I have no idea right now what the issue is with the Accept header. There are a few possibilities, however, such as if the user is using a proxy/VPN... or if they are running the browser in "private" mode - there is also some PC software that could cause the problem. I'm going to talk with Michael (error10) about this, and see if he has any ideas.

Most of the time, these are actual spambots.

The rest of the time, it's somebody who installed Norton or something and whatever they're using is stripping out random headers, and the user doesn't really know what's going on. Or someone who thinks they know what they're doing who is a bit extreme with their "privacy". Often these require somebody to actually talk with the user and figure out what's actually going on.

Like Eric, I'm glad my code has been helpful in reducing the spam and DoS problems for your forums. I'm nearing the 2.2 core release and as soon as I have that out, I can get back to work on some core stuff that's been waiting a long time. I'll be posting an updated roadmap for 3.0 soon.

[error10]

Eric, I've pushed out 2.1.13 which should resolve the Google and Yahoo search engine issues.

[Simon Lloyd]

Error10, how do we use your latest release?
Also, this may sound a bit daft to you but i'm sure there are many other users of your efforts that want to ask, could you explain (because i don't have a clue) what each part of this is and what it does, i, like most forum owners are paranoid at denying real users or visitors, so it would be a great help (or perhaps release it as an article on your site)

Here's a POST one

POST http://www.thecodecage.com/forumz/me...l/register.php HTTP/1.1
Accept-Encoding: gzip, deflate
Accept-Language: ru-RU
Connection: Close
Host: www.thecodecage.com
User-Agent: Mozilla/4.0 (MSIE 6.0; Windows NT 5.1)

Here's a GET one

GET /forumz/showthread.php?t=162877 HTTP/1.1
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Charset: ISO-8859-2,utf-8;q=0.7,*;q=0.7
Accept-Encoding: gzip, deflate
Accept-Language: pl,en-us;q=0.7,en;q=0.3
Connection: keep-alive
Cookie: tcclastvisit=1302940743; tcclastactivity=0; tccuserlgv=1; __utma=118899148.698522646.1302940750.1302940750.1 302940750.1; __utmz=118899148.1302940750.1.1.utmcsr=google|utmc cn=(organic)|utmcmd=organic|utmctr=kod%20%20vba%20 has%C5%82o; __utmv=118899148.usergroup-1-Unregistered%20%2F%20Not%20Logged%20In; tccsessionhash=39baa5cc5c25fad88452daba12603a3f; vbet_sessionUsed=1
Host: www.thecodecage.com
Keep-Alive: 115
Referer: http://www.google.pl/search?q=kalend...&start=20&sa=N
User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
X-Rewrite-Url: /forumz/showthread.php?t=162877

If we know exactly what we are looking at or for it would help, and of course you will have more constructive questions

[Alfa1]

Originally Posted by Eric

Alfa1, I do apologize, but I have no idea right now what the issue is with the Accept header. There are a few possibilities, however, such as if the user is using a proxy/VPN... or if they are running the browser in "private" mode - there is also some PC software that could cause the problem. I'm going to talk with Michael (error10) about this, and see if he has any ideas.

Until there is a solution for 'Required header 'Accept' missing', is there a way to not block users for this reason? Its blocking about 50 valid users every 24 hours. I have no doubt that its caused by registered members with security software. I do not want to block these real users. Talking to all these users or whitelisting all their IPs is not possible.

Originally Posted by error10

Most of the time, these are actual spambots.

In the logs of my limited testing these have been 100% real users.

Originally Posted by error10

PayPal has a long history of sending their IPN notifications without a User-Agent. There's nothing I've been able to do to convince them to send a User-Agent except to advise affected people to complain to PayPal. In the meantime you can whitelist their IP addresses.

I would be highly surprised if anyone would be able to convince paypal about anything. I have whitelisted the script.

New feature request:
Alert staff if registered member performs SQL injection or other attacks

One thing that I find missing in this addon is a way to feed bad bot data to the blacklist. Please consider to add such functionality. Either as part of this addon or as Projecthoneypot integration. Added to tracker: Feed data to blacklist

[Simon Lloyd]

Originally Posted by Alfa1

I would be highly surprised if anyone would be able to convince paypal about anything. I have whitelisted the script.

Could you give details of this whitelisting as i'd like to do the same

[Alfa1]

In /includes/whitelist.ini find: example.php
Replace with the script that you want to whitelist.

[Simon Lloyd]

, i know how to whitelist i'd like to know what you did to whitelist paypal?

[Alfa1]

Originally Posted by Simon Lloyd

, i know how to whitelist i'd like to know what you did to whitelist paypal?

I whitelisted the php file that handles my subscriptions.

[carsafety]

Originally Posted by Alfa1

I whitelisted the php file that handles my subscriptions.

I've been watching this mod, planning to install it soon. Do all of these things like search engine spiders, Paypal, adsense and other legitimate scripts come whitelisted out of the box?

If not, is there a list of them somewhere with simple instructions on how to add them?

[error10]

Originally Posted by Simon Lloyd

Error10, how do we use your latest release?
Also, this may sound a bit daft to you but i'm sure there are many other users of your efforts that want to ask, could you explain (because i don't have a clue) what each part of this is and what it does, i, like most forum owners are paranoid at denying real users or visitors, so it would be a great help (or perhaps release it as an article on your site)

Here's a POST oneHere's a GET oneIf we know exactly what we are looking at or for it would help, and of course you will have more constructive questions

Simon, for vBulletin you don't necessarily use it directly; but wait for Eric to package it up and post the update. Bad Behavior consists of two parts, the core code which does the work of deciding whether something is bad or not, and a platform connector which lets it talk to vBulletin (or WordPress or MediaWiki or Drupal or whatever). I maintain the core, and Eric maintains the vBulletin connector, packaging the two together into a single downloadable mod. If Eric ever got run over by a bus, it would be possible to take the core and add it in yourself, but let's hope nobody ever gets run over by a bus.

As for the two entries you posted, the Log entry gives an indication of what the issue was, and of course with POST requests you can inspect the entity. The first one is a pretty blatant registration spam. I'm not sure what the issue is with the second one. Perhaps it was on Project Honey Pot? It doesn't look like you provided the log entry for them, so I can't really be certain.

Originally Posted by Alfa1

Until there is a solution for 'Required header 'Accept' missing', is there a way to not block users for this reason? Its blocking about 50 valid users every 24 hours. I have no doubt that its caused by registered members with security software. I do not want to block these real users. Talking to all these users or whitelisting all their IPs is not possible.

In the logs of my limited testing these have been 100% real users.

I would be highly surprised if anyone would be able to convince paypal about anything. I have whitelisted the script.

New feature request:
Alert staff if registered member performs SQL injection or other attacks

One thing that I find missing in this addon is a way to feed bad bot data to the blacklist. Please consider to add such functionality. Either as part of this addon or as Projecthoneypot integration. Added to tracker: Feed data to blacklist

I don't want to block real users either, if I can avoid it. But see directly above for Simon's posting of a registration spam, where the spammer has omitted the Accept: header. And obviously not everything is foreseeable. Legitimate users caught by this already get a message stating that it's likely caused by their browser privacy software and some basic instructions on reconfiguring the software. These could always be improved if I knew the specific software causing the problem. I could also move this test to strict mode, though since it actually does block a lot of spam, I fear it would make Bad Behavior almost useless. So this is a hard problem.

A way to send in data, both on bad bots and on legitimate users inappropriately blocked, is on my roadmap already. As for notifying the admin of particular events, I think that will be on Eric.