Administrative and Maintenance Tools - vbStopForumSpam - known spammer lookup for new registrations

vbStopForumSpam

This provides access to a RBL type system for forum admins, listing known spam IP / email / usernames. The RBL database is provided by www.stopforumspam.com. You do NOT need an API key from the website in order to access the database. only to submit data if you should wish to do so.

At the point of user registration, the mod checks if the IP number / provided username / email addresses appear on a block list and can block the registration.

Whilst this isnt the most perfect way to stop all forum spam, its another step that spammers have to overcome.

VB4 here https://vborg.vbsupport.ru/showthrea...hreadid=230921
Its the same code, it works in 3.54 to 4.0


What it does

It checks with a remote database of known forum spammers. Their IP number, email address and forum username are tested and based on your configuration, you can reject / log / accept user registrations based on what you get back.

This version doesnt have
- whitelisting or the ability to submit users to the database but it will within the next week.
- automatic user deletion / post / PM purging. There are good tools out there already, this does something else.

Instructions are included in the installation.txt file - PLEASE read it first and dont forget to actually upload the files in the upload folder, otherwise it WILL kill your registration progress and you wont see the log file options in admincp. You do not need to download the product-vbstopforumspam-3.54.xml file unless you are using a vBulletin version older than 3.6.0

Changes to vB
- 3 new database tables
- 2 database table alternations
- No new templates.
- 2 Hook (register_addmember_process & register_addmember_complete)

Ive tested it but had feedback that it works with versions as old as 3.6.2... Support should go back to older versions, as long as they have hook support for register_addmember_process / register_addmember_complete

Known to work - tested by me
- vBulletin 3.6.8 on Apache 2.2 / PHP 5.1.2 on Linux using cUrl
- vBulletin 3.7 Gold on Apache 2.0 / PHP 4.4.3 on Windows without cUrl (template changes wont work on 3.7 - thats in the next version with auto template changes)

For code to submit spammers to the database, check this post for code changes
https://vborg.vbsupport.ru/showpost....&postcount=288

Reported in the thread to work
- 3.6.1, 3.6.2, 3.6.9, 3.6.10, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.74, 3.80, 3.81, 3.82, 3.83, 3.8.4, 4.0beta3

If you have 3.54, then you can use the product-vbstopforumspam-3.54.xml file attached instead of the one in the ZIP file, which will allow older vBulletin versions to access this mods' features. I personally havent tested this version, its a user contribution, thanks to Darrell Mobley, that changes the way the XML works when imported into older versions.

Installers should remember to refresh their ACP navigation window when they first install it so they can see the new log file menu item.

REQUIRES MySQL 4.1.1+

Future versions
- Automatic integration into vBulletin to add users to the stopForumSpam.com database from a form
- Whitelisting of username / IP / email addresses
- AJAX integration to allow for lookups from within the users profile
- Decreased remote query count from three per user to one per user.

Versions / Changes

0.1 Initial Release

0.2 pedigrees special brew birthday release.
- Small security update. If you have 0.1 installed, download 0.2 and replace your existing functions_vbsfs.php with the one in the archive. It just tests to see if its running inside the VB framework before anything else. This is what happens when you code at 2am after drinking wine

0.3
- stopped it processing valid registrations twice
- moved all non-function code into the plugin. Not a big one as 0.2 basically did that
- fixed a typo in the log pruner that stopped it working (404)
- removed unused fields from the database for people with mysql that doesnt support varchar > 255 (ie mySQL4). If you have 0.2 installed and dont need to prune your logs just yet, you dont really need to install this version but can instead wait for 1.0 unless of a massive security update.

0.4
- logs registrations that arent/wouldnt be blocked
- fixed XML errors when username has a space it in
- tightened up the cache so that it doesnt test a username against an email name to give a bypass result (for when a username is an email address that isnt banned where the email address is)
- fixed some basic logic errors in the PHP

0.6
- Should work on PHP 4.4 now - rewrote the XML with PHP4 in mind (tested on Apache2.0/PHP 4.4.3)
- Fixed a caching system where data wasnt being updated correctly which could cause a remote query when one wasnt needed
- Possible false negative situation when a spammer was blocked due to SFS.com being down who then visited again when it was up but within the cache expiry time
- Remote query failure when the result page isnt XML should work a bit better now. It does a very basic test for valid XML results.
- Fixed log purging (again) and it should actually work properly now.
- No longer requires PHP5
- The log viewer now links to a user profile when registration is allowed.

v0.61 - Removed a template change that was invalid vBulletin code. The package you download will still say its 0.60 however

NB : When upgrading from any version to 0.6, you must remove and then add the plugin due to changes in one of the database tables

You need to have an API key from www.stopforumspam.com in order to submit data, its free and easy to get... You DONT need an API key in order to use this mod however, only to submit spammer data.

Issues are
- The usergroup permissions / view details etc DONT work. I jumped the gun and put the permissions controls in there before I put the code in. Please delete the includes/xml/bitfield_vbstopforumspam.xml file and rebuild your postbit

Installation
- Follow the instructions in the zip file, that includes upload the correct folders
- ONLY download the 3.54 xml file if youre using a vbulletin version prior to 3.6.0. use this file to install the mod instead of the xml file in the zip file.

Please click Installed

[pedigree]

Badger hmm thats strange, it might happen I guess if the user refreshed the page and then has data resubmitted. What Ill do is add code to check if the user/email is in use and them stops processing, allow control to pass back to vbulletin for rejection... No point doing all that processing if the user exists.

Thanks for that, it made me thinks

And this is an open request...

Post here what you would like to see in the ways of stats, charts, graphs etc...

so far Ive got

Bar graphs
- Registrations per hour blocked / allowed by policy / allowed with old data / allowed clean / whitelisted / blacklisted

Pie charts
- Blocked domains by domain name
- Blocked domains by country
- Blocked registrations by domains
- Blocked registrations by IP in 16/24 bit subnets (hard on the CPU)
- Cache hits vs misses vs time

[BadgerDog]

Hi pedigree ...

I've been using this mod for the last 6 months without any problems, but since it handles the registrattion page differently thatn the default vbulletin code, could that be why it repeated 6 times? If he kept making mistakes and it's a java code based registration, it might be a factor?

https://vborg.vbsupport.ru/showthread.php?t=144869

Regards,
Badger

[Wired1]

Originally Posted by Alfa1

BTW: does this mod extend the time needed for registration? Does it conflict with ISBOT?

Nope. When they hit submit, it compares their IP / email address / user name against a list, and if they're on the list, no entry permitted. I believe they just come back to the registration page so they can change their user name / email address.


pedigree: in 0.7.0, can we now sort the log by all columns, and/or do a search on it? As a secondary suggestion (and I can see this somewhat being outside the scope of your mod, so if you agree, no biggie), maybe compare all people who were banned in the past month, and compare it to successful registrants in the vBStopForumSpam logs? This way, an admin can have a quick list to manually report to the black list (e.g. hitting a button to do it)? This would allow the admin to only report those who have banned because of spamming, and not violating other rules on the forum.

Come to think of it, if someone was PERM banned via an infraction (granted not many use infractions in this way I think), maybe this mod could somehow detect it and auto-report it?


Badger, the repeats are normal. It's the user screwing up, that's it. I see it all the time in my logs, and my register form is standard. They're just mis-typing or something.

Now, if the error was added to the log, that would be nice, although I'm not sure how much it would help to prevent spam.

[skippybosco]

Took a bit of time and tallied up stats for all of June for one of my sites

Out of the 7180 registrations for the 30 day period:

~.03%/250 were caught by SFS Email check
~25%/1860 were caught by SFS UserID check
~45%/3260 were caught by SFS IP check

~25%/1810 made it past SFS Check

note: of the 1810 that made it past SFS, 780 of those were caught by RBL Checking and rejected, 12 slipped through completely but were manually discovered, no false positives that I am aware of.

And this is an open request...

Post here what you would like to see in the ways of stats, charts, graphs etc...

How is the data being aggregated? Have you indexed the log table for reporting or are you normalizing the reporting data elsewhere?

I'd be interested in some long term % based trending (week, month, year) as line graphs

Things like:

% of blocked registrations over time
Of blocked registrations, % email, % ip, % username on a single graph over time

Thanks again and Great Job!

[pedigree]

At the moment, the only data that I have going to a stats table is cache hit/miss numbers, records on an hourly total. Ive done that with a roundrobin type system, almost. It purges records over 1 year old (just as RRDtool does) and logs on hourly totals, inserting new records and updating existing ones (for those with mysql knowledge, its an insert on duplicate key update with an index on the primary date field)

I thought about using RRDtool but couldnt find a pure php implementation of it as I couldnt rely on hosts having binary execution rights etc.

All of the graphs are going to have weekly/monthly and yearly trending for each - something that I hope wont kill the server too much, hence why Im using hourly totals rather than just inserting a new row per sample.

The stats tables will be seperate from the log data so if you purge your logs, the stats will remain untouched.

There is also a diagnostics logs, off by default, where it will attempt to log each step of the registration process with records purged after 7 days. As this table contains a TEXT field, I wanted to keep the size down. This is mainly for debug review if something starts going wrong with the code.

skippy - I was thinking about RBL checking but there is already a package doing so that I left it at that Would you like to see something like it in this, with graphing? I was thinking about adding SURBL support in a later version.

Wired1 - You will be able to sort on all the fields and Ill add a search as well. It will be a match on any field, you wont be able to specify which field you want to search on as I want to get this released before my son is born. Ill be working during my paternity leave when I really shouldnt, so I wont have too much spare time. As for the ban/report, its a bit out of the scope but Ill add it to the list for future releases.

[skippybosco]

pedigree, a single mod to manage all of my spam validation would be a holy grail for me...

that being said, having two right now is not the end of the world and many things on your list seem much higher priority (especially that whole son thing :-p)

[Alfa1]

It seems to work very well on vb 3.7.1 PL2

So well, that I am being flooding by emails from this hack. How do I turn them off?

[BadgerDog]

Ok, I caught a spammer who wasn't blocked by "stop forum spam" site.

How do I now get that information, IP address, email address and user name to their site?

Thanks...

Regards,
Badger

[skippybosco]

So well, that I am being flooding by emails from this hack. How do I turn them off?

@Alpha1: What emails are you receiving? I don't remember there even being an option for this mod to email?

Ok, I caught a spammer who wasn't blocked by "stop forum spam" site.

How do I now get that information, IP address, email address and user name to their site?

@BadgerDog: You can add new entries at: http://www.stopforumspam.com/add

[BadgerDog]

Originally Posted by skippybosco

@BadgerDog: You can add new entries at: http://www.stopforumspam.com/add

Thanks... did that ... :up:

My first "kill"...

Regards,
Badger