Administrative and Maintenance Tools - vbStopForumSpam - known spammer lookup for new registrations

vbStopForumSpam

This provides access to a RBL type system for forum admins, listing known spam IP / email / usernames. The RBL database is provided by www.stopforumspam.com. You do NOT need an API key from the website in order to access the database. only to submit data if you should wish to do so.

At the point of user registration, the mod checks if the IP number / provided username / email addresses appear on a block list and can block the registration.

Whilst this isnt the most perfect way to stop all forum spam, its another step that spammers have to overcome.

VB4 here https://vborg.vbsupport.ru/showthrea...hreadid=230921
Its the same code, it works in 3.54 to 4.0


What it does

It checks with a remote database of known forum spammers. Their IP number, email address and forum username are tested and based on your configuration, you can reject / log / accept user registrations based on what you get back.

This version doesnt have
- whitelisting or the ability to submit users to the database but it will within the next week.
- automatic user deletion / post / PM purging. There are good tools out there already, this does something else.

Instructions are included in the installation.txt file - PLEASE read it first and dont forget to actually upload the files in the upload folder, otherwise it WILL kill your registration progress and you wont see the log file options in admincp. You do not need to download the product-vbstopforumspam-3.54.xml file unless you are using a vBulletin version older than 3.6.0

Changes to vB
- 3 new database tables
- 2 database table alternations
- No new templates.
- 2 Hook (register_addmember_process & register_addmember_complete)

Ive tested it but had feedback that it works with versions as old as 3.6.2... Support should go back to older versions, as long as they have hook support for register_addmember_process / register_addmember_complete

Known to work - tested by me
- vBulletin 3.6.8 on Apache 2.2 / PHP 5.1.2 on Linux using cUrl
- vBulletin 3.7 Gold on Apache 2.0 / PHP 4.4.3 on Windows without cUrl (template changes wont work on 3.7 - thats in the next version with auto template changes)

For code to submit spammers to the database, check this post for code changes
https://vborg.vbsupport.ru/showpost....&postcount=288

Reported in the thread to work
- 3.6.1, 3.6.2, 3.6.9, 3.6.10, 3.7.0, 3.7.1, 3.7.2, 3.7.3, 3.74, 3.80, 3.81, 3.82, 3.83, 3.8.4, 4.0beta3

If you have 3.54, then you can use the product-vbstopforumspam-3.54.xml file attached instead of the one in the ZIP file, which will allow older vBulletin versions to access this mods' features. I personally havent tested this version, its a user contribution, thanks to Darrell Mobley, that changes the way the XML works when imported into older versions.

Installers should remember to refresh their ACP navigation window when they first install it so they can see the new log file menu item.

REQUIRES MySQL 4.1.1+

Future versions
- Automatic integration into vBulletin to add users to the stopForumSpam.com database from a form
- Whitelisting of username / IP / email addresses
- AJAX integration to allow for lookups from within the users profile
- Decreased remote query count from three per user to one per user.

Versions / Changes

0.1 Initial Release

0.2 pedigrees special brew birthday release.
- Small security update. If you have 0.1 installed, download 0.2 and replace your existing functions_vbsfs.php with the one in the archive. It just tests to see if its running inside the VB framework before anything else. This is what happens when you code at 2am after drinking wine

0.3
- stopped it processing valid registrations twice
- moved all non-function code into the plugin. Not a big one as 0.2 basically did that
- fixed a typo in the log pruner that stopped it working (404)
- removed unused fields from the database for people with mysql that doesnt support varchar > 255 (ie mySQL4). If you have 0.2 installed and dont need to prune your logs just yet, you dont really need to install this version but can instead wait for 1.0 unless of a massive security update.

0.4
- logs registrations that arent/wouldnt be blocked
- fixed XML errors when username has a space it in
- tightened up the cache so that it doesnt test a username against an email name to give a bypass result (for when a username is an email address that isnt banned where the email address is)
- fixed some basic logic errors in the PHP

0.6
- Should work on PHP 4.4 now - rewrote the XML with PHP4 in mind (tested on Apache2.0/PHP 4.4.3)
- Fixed a caching system where data wasnt being updated correctly which could cause a remote query when one wasnt needed
- Possible false negative situation when a spammer was blocked due to SFS.com being down who then visited again when it was up but within the cache expiry time
- Remote query failure when the result page isnt XML should work a bit better now. It does a very basic test for valid XML results.
- Fixed log purging (again) and it should actually work properly now.
- No longer requires PHP5
- The log viewer now links to a user profile when registration is allowed.

v0.61 - Removed a template change that was invalid vBulletin code. The package you download will still say its 0.60 however

NB : When upgrading from any version to 0.6, you must remove and then add the plugin due to changes in one of the database tables

You need to have an API key from www.stopforumspam.com in order to submit data, its free and easy to get... You DONT need an API key in order to use this mod however, only to submit spammer data.

Issues are
- The usergroup permissions / view details etc DONT work. I jumped the gun and put the permissions controls in there before I put the code in. Please delete the includes/xml/bitfield_vbstopforumspam.xml file and rebuild your postbit

Installation
- Follow the instructions in the zip file, that includes upload the correct folders
- ONLY download the 3.54 xml file if youre using a vbulletin version prior to 3.6.0. use this file to install the mod instead of the xml file in the zip file.

Please click Installed

[pedigree]

Originally Posted by allenelson

installed / tested. awesome mod, exactly what i was looking for when searching for spam. kudos to you

Thanks Now click Installed

[Thomas P]

@pedigree: I just checked, the last NN spammers weren't clever enough to use a proxy. Mayber they are even in countries where they cannot/must not use proxies...

[pedigree]

Originally Posted by Thomas P

@pedigree: I just checked, the last NN spammers weren't clever enough to use a proxy. Mayber they are even in countries where they cannot/must not use proxies...

Thank you Great Firewall of China.... Adding a check box to "Block anonymous proxies" isnt a huge code change with all of the code rewrites Im doing. Ill add it, you never know, it might get a spammer

[skippybosco]

My weekly praise to pedigree and the amazing work on this mod.

Just took a look at the last few days activity:

Out of 6740 attempted registrations:

• 1850 of them were valid users
• 100 were blocked due to email
• 3200 were blocked due to IP
• 1590 were blocked due to username

Thank you Great Firewall of China.... Adding a check box to "Block anonymous proxies" isnt a huge code change with all of the code rewrites Im doing. Ill add it, you never know, it might get a spammer

For what it is worth, I use a real time analytic tool GetClicky. One thing that I do when determining if a user was improperly rejected is to check the IP address and see if there is activity on my site. 100% of the time so far they do not show up, most likely indicating that they are blocking javascript from executing. I'm not sure if this is helpful information as a secondary check or not as it seems that it may catch users with outdated browsers as well, but so far is a good canary for me.

[Twin_Turbo]

PHP Code:

$bbuserinfo[ipaddress] 


That gives me my own IP address instead of the users (in userinfo template), what's the correct var for the members ip addy?

[Wired1]

I realize a lot of these suggestions are dependent on a better API / submission system from StopForumSpam.com

Suggestion: Once a submission function is built in, perhaps add an additional layer of analyzing to it? Example:

User1 / Email1 / IP1 was found to be a spammer via IP today. 3 hours later, they attempted to register again. User1 / Email1 / IP2.

So, if someone that's attempting to join passes, a last check would compare their user name and/or email and/or IP to previously blocked registrations. This way, they could be shut down from registering under slightly different credentials. An option in the adminCP could be added so the Admin can say how many days back in the log file to check (if not the whole thing).

Suggestion: If a submission is blocked, grab the rest of the offending info from the StopForumSpam site, and compare against the suggestion. If it doesn't all match, submit the submission's info so the StopForumSpam site is more complete.

Suggestion: Also, perhaps a function that would compare only allowed registrations to the StopForumSpam site. After all, some spammers make a login, and then don't spam for days. If another forum has caught them and flagged them, now you can be aware of this "sleeper" member and ban them. Notice could come via PM or New Thread post (akin to the Multiple Login AE mod here).

Not sure what would be best: checking on a CRON (or something similar), or only checking via a manual button. Perhaps an additional table column, so that if an allowed registration was checked 3 times after the account was created, it won't be checked again (so as to limit bandwidth and resources, both on the forum and StopForumSpam's site). Also, an option for the admin to manually OK an account, so it's bypassed in this check (or automatic, e.g. 20 posts in the forum, or in a certain group, or whatever).

Suggestion: A search for the log would be nice as well

Suggestion: Once the manual / auto submission tool (and possibly some of the others I've suggested) are in place, color code (or whatever) the log? Example: Spammers I've submitted in red, (easily customizable by admin via FFFFFF), or mark by symbols (searchable of course), ones thoroughly checked in green, etc.

[skippybosco]

The obvious risk is that users that were originally flagged incorrectly that you manually approved would get caught. If this kind of checking happened it would need to only affect users of a certain user group(s).

My vote would be to tie the "re-check" to an activity that would warrant concern (posting, PMing, etc)..

Scenario being a "Registered" group and a "Promoted Registered" group. Users in registered group are checked against StopForum before being allowed to post. Failures can either be quarantined or prevented.

Set up a promotion schedule (time, # posts, whatever) to move users to Registered so your trusted users don't pay performance penalties.

[Wired1]

You're talking about the comparison down the road? It doesn't have to auto-ban them or anything, just notify the admin of this new information. They can decide depending on the user's posting habits, the frequency that they showed up on the list (that's already part of the API), a simple google check, etc.

[pedigree]

Originally Posted by Wired1

I realize a lot of these suggestions are dependent on a better API / submission system from StopForumSpam.com

Suggestion: Once a submission function is built in, perhaps add an additional layer of analyzing to it? Example:

User1 / Email1 / IP1 was found to be a spammer via IP today. 3 hours later, they attempted to register again. User1 / Email1 / IP2.

So, if someone that's attempting to join passes, a last check would compare their user name and/or email and/or IP to previously blocked registrations. This way, they could be shut down from registering under slightly different credentials. An option in the adminCP could be added so the Admin can say how many days back in the log file to check (if not the whole thing).

Suggestion: If a submission is blocked, grab the rest of the offending info from the StopForumSpam site, and compare against the suggestion. If it doesn't all match, submit the submission's info so the StopForumSpam site is more complete.

StopforumSpam.com doesnt have the functionality to pull all the other details for spammers based on just one field. We cant get a list of IPs that SpammerX has logged in from. I can however, put all the details for a failed registration into the database. To be useful over a 24 hour period, I might think about changing the local cache from 90 minutes to 24 hours.

Suggestion: Also, perhaps a function that would compare only allowed registrations to the StopForumSpam site. After all, some spammers make a login, and then don't spam for days. If another forum has caught them and flagged them, now you can be aware of this "sleeper" member and ban them. Notice could come via PM or New Thread post (akin to the Multiple Login AE mod here).

When I get all the template code working, there will be options for mods to examine users, refresh SFS.com data and act on that. In order to run proactive scanning against all the users against all SFS.com data, would require more work both on my time (I have a baby due in 6 weeks) and in cron execution time. Unless you have the Ajax cron mod installed, running a job like that would kill so poor users session. Maybe in v1+ Ill be able to add an automated pull from SFS.com of their IP lists. Anything more would be a lot more code. I want to get the next version out, working before baby is born because after that, well, I wont have much more time until I go back to work.

Not sure what would be best: checking on a CRON (or something similar), or only checking via a manual button. Perhaps an additional table column, so that if an allowed registration was checked 3 times after the account was created, it won't be checked again (so as to limit bandwidth and resources, both on the forum and StopForumSpam's site). Also, an option for the admin to manually OK an account, so it's bypassed in this check (or automatic, e.g. 20 posts in the forum, or in a certain group, or whatever).

Suggestion: A search for the log would be nice as well

Ill add a search to the logs after I get all the new code working

Suggestion: Once the manual / auto submission tool (and possibly some of the others I've suggested) are in place, color code (or whatever) the log? Example: Spammers I've submitted in red, (easily customizable by admin via FFFFFF), or mark by symbols (searchable of course), ones thoroughly checked in green, etc.

Auto submit can be sorted and logs updated to reflect that, its not a big, as long as you have cURL installed. Without cURL, automated submission will be extremely difficult, something that can be done but something that Im not really wanting to code for.

main goals for the next couple of weeks, is to get the core rewrite sorted, the 2.6 and 3.7 template changes sorted so that they rewrite templates on the fly, allowing mods to view all the data about the user, do whois/google searches, updating SFS.com data and submitting them manually, better loggind support and statistics reporting. Auto submitting user data to SFS.com might sneak in there but any companion, posting, PM etc will have to wait until all that is stable.

I spend 4-5 hours a day on a train to/from work so my time is a bit limited but Im trying to get it all sorted and out asap.

[pedigree]

Originally Posted by Twin_Turbo

PHP Code:

$bbuserinfo[ipaddress] 


That gives me my own IP address instead of the users (in userinfo template), what's the correct var for the members ip addy?

Yeah, thats why I posted an update to v0.61 saying that you should remove the template changes. In the next version, all the templates changes will be automatically made. Ill have a couple of people who said they will help me in the testing, which will ensure I dont make some an awful screwup again

I think it might be $userinfo[host] but Im not on my machine, that could be wrong.