What I might be able to do is leave blacklisted cache hits, in the cache for 24 or 48 hours, and "no result" hits for 30 minutes. That would help that a bit.

I don't mean to be argumentative, pedigree, but what you seem to be saying is my estimates of central database load and the percentage of queries being blocked by the local cache are wrong somehow. You're the expert. There's no denying that. But I don't see where I went wrong. :confused: With an average of 120 blocked bot registration attempts a day (1,200 bot registrations blocked in 10 days) on my site, that's an average of about 5 bot registration attempts per hour or 2.5 every 30 minutes. If those numbers are right, and based on my log file, they certainly seem to be, how can that 30 minute local cache be blocking very many bots? :erm:

I don't get it. What have I overlooked? Is the problem that bot registration attempts tend to be concentrated (and thus are much higher) at certain times of the day? Is THAT where my logic went wrong?

Thanks!

i just want to say thank you pedigree
its awsome ,i got it running some moths and not ONE spammer came past them dogs
big big thank you ,its a true time saver

You're right, denman. This product is an absolutely great tool.

Searching back through logs = extending cache times but without the headache of parsing another table when I have the mechanism in place already.

Were working an the figures on about 6000 unique IPs generating 600,000 api calls per day. Most arent cached like my mod does. Im only aware of two that are, mine and First defense against spam for wordpress.

At the moment, Im concentrating on finishing the almost total recode of the website, api etc.

Okay. I get it. I think you're saying there's no need to add a log scan when you already have the cache scan working and can easily increase the cache time. Makes sense. Also makes sense to only keep not found items in the cache for 30 minutes or so but keep the found (blacklisted) items in the cache longer. Again, the goal is to reduce the central server load. Why waste its time and energy checking on a bot it rejected an hour or two ago trying to join the same site?

Gotcha. Over time, perhaps you should consider dynamically adjusting the size / duration of the local cache size based on time or the volume of requests processed by that site. That way smaller sites with lower volumes might be able to cache 24 or 48 hours of requests and drastically cut their reliance on the central server while larger / busier sites would not be penailzed because they receive more requests but might rely more heavily on the central server instead. For instance, you might cap the size of local cache at say 300 / 600 entries or 24 / 48 hours whichever came first. It's a thought.

600,000 calls per day is 25,000 calls per hour or 417 calls per minute. With that volume of requests, it's no wonder you're having DNS server outages and server/DB overload problems. I'll bet you cut that load in half by increasing the local server cache time to 12, 24 or 48 hours.

I'll bet Floris and other busy site operators like him would approve of this approach and small site guys (like me) will get better performance and less service outages too.

Good solution! Hope my thinking about this helped. :)

Exact same problem here on v3.8.4 patch level 2.

Have tried browsing to the file from my local machine as well as uploading direct from server.

Why not try the 3.8x version? https://vborg.vbsupport.ru/showthread.php?t=222536

Exact same problem I'm afraid. Sorry, no idea how I ended up in this thread, thanks for the heads-up.

The 3.6 and 3.8 versions are exactly the same and the only difference between them and the 4.0 version is that the 4.0 version has a slightly different mod ID as the forums here dont allow ID duplicates.

Ive never been able to reproduce the invalid file on import as I always import

product-vbstopforumspam.xml

And I have it running myself on 3.8.4 PL2

Is there a problem with this plugin?? Did the Spammers find a way around??? I have been getting many registrations the last few days from spammers, and most of them are listed on the Stop Forum Spam site, but are still being allowed to register????

John

There was some downtime on the sfs site which meant that if you had the mod set to allow registrations if the ip/name/email check timed out they'd get through.

Had a couple of nasties myself

Where is the vBSFS cache stored by the way, in the vB SQL db or a separate file somewhere?

vbstopforumspam_remotecache table in mysq

Ive got a small bug fix due for release shortly, that touches on the caching

the problem continues, dropped you PM with the IP of the server.

Try downloading the patches from this site again, Then obviously, you'll want to unzip them and then upload them to your server again. Certainly sounds to me like something got glitched somewhere.

I'm only running vb 3.8.1 on the site where I'm using this addon; but I'm not seeing any issues of the type you describe at all.

I'd say roll back and start over from the very beginning (including the download and the extract from the ZIP file) and see if that doesn't solve it. Also please note this comment from the install instructions:

If you cant see the logs in ACP/ Statistics and Logs, then you didnt upload the contents of the "upload" folder.... Its there for a reason, upload its contents to the root of your forum. This isnt an issue, its just something that 99% of the "it doesnt work" issues arise from

I doubt that it applies in your case; but it's certainly worth checking.

Good luck. Hope this helps!

Floris is all fixed so just ascender to go.

Let me try to explain it from how I've looked over the code.

The main need for the local cache is because of where the hook location is in the registration code. Pretend a person (or bot) messes up their registration and the system throws an error (say they messed up the CAPTCHA, or forgot to fill out a required field). The hook to check SFS occurs after every "submit" is hit on the registration page, but before vB error checking and final user saving is done. Thus if a bot is pounding the registration with failed attempts it would in turn be pounding SFS with the same query over and over again. The local cache alleviates this problem since the Username/Email/IP should be the same through several attempts to get a successful registration.

I think it would be okay to cache the definite hits for the long-term, but usually after failing in that one short time-span, you won't see the same user/email/ip combo again. (especially if you already banned the user, which would also prevent the same email address from being used). And if it's a dynamic IP, eventually a legitimate user *could* get blocked because you cached results for several weeks (or are not checking how recent the last bot was seen).

But anyhow, I think having a short local cache is more than sufficient. I've seen bots pound the registration with failed attempts, but still the local cache is not allways effective since they sometimes will try different usernames/emails/IPs when the previous attempt is not successful. Fortunately for me there's tell-tale signs of being a bot (i.e. 10 registration attempts in one second) which gets them submitted to SFS. Also I have other checking that I do before querying SFS to tell if they are a bot, that helps Pedigree out in reducing load on his server.

Thanks, eco_Jason. This explanation was very helpful even though it doesn't quite fit the bot behaviors I've observed. One thing I have realized over the past 3 years is that if one works out a strategy that is highly effective in defeating these borg-beasts they soon begin to deliberately study you to figure out how you're defeating them and try to devise a strategy to work around your protection technique. For example, the appeal of one of my sites is primarily U.S.-Regional So 2 years ago I decided I could block 99.999% of bots just by blocking and/or removing all registrations from anywhere in the world outside the 7 major U.S. and Canadian time zones. This was virtually 100% effective for months. In fact it was so effective I eventually set the underlying mysql query up as a cron-ed script and ran it every 15 minutes 24 hours a day 7 days a week. Presto. NO more bots! It worked perfectly for months.

Then one day I noticed I had a new registered user whose name was "FigureOutTimeMachine" (I'm not kidding you. So help me God, that WAS it's name!). Then within a few days my "time zone defense shield" suddenly didn't work anymore because the bot authors had rewritten their code and were now deliberately lying to my server about what time zone they came from.

Frankly, I suspect we may be seeing different attack patterns and strategies because the bot-authors are using their tools to study how our defenses work in deliberate targeted efforts to figure out how to defeat them. If THAT doesn't increase your paranoia level, it sure should! It says our enemies are studying us and getting smarter about how to defeat our best defenses every day... :eek:

Thanks again for the explanation, Jason. I'm not afraid to say that even after 42 years in the IT field, I'm still learning. Anyone who doesn't believe we're fighting the skirmishes of the world's first cyber war should come spend a few months on the front lines with us. They'd learn they're wrong real fast.

Just in the past few days I've had people contact me that are trying to register and they are not spammers.

I'm going to have to disable this and find something out it seems.

@TMH63: Have you verified that their IP/Email/Username are in fact part of the SFS database?

As I've seen the "I have to disable this mod" statement a few times in the last few weeks, it is probably worth reiterating that you do not have to block users that are caught by SFS. Instead have those users put into a more restrictive usergroup which either has their posts validated via Akismet or has it's posts manually validated by your site moderators. You can then set an automatic promotion strategy (based on time, # posts, etc) to move them to a less restrictive usergroup.

This gives you the best of both worlds... even in the rare case there is a false positive, your users can still register and post, while at the same time having peace of mind that content is being vetted before it is being presented to your users.

I've been experiencing the same problem the past few days as well. When looking at the vbStopForumSpam logs, it will frequently show an "Allowed registration" message yet the user account is non-existent. Sometimes the users contact me and complains, or sometimes the users try and try again, and I'll see the same "allowed registration" message multiple times (with different timestamps) until eventually the account is actually created.

The dead giveaway that there is a problem finalizing registrations is with the spam bots and their method of changing their email address and/or IP address. When I look at the log I'll see the same spam username getting the "allowed registration" message numerous times, all with different timestamps, different email addresses, and different IP addresses. So this would confirm that the user is attempting to register, being allowed to register, but then something goes wrong.

On a plus side, the mod is still working in the sense that I still see users being rejected based off of the stopforumspam queries.

EDIT: Question. If a user fills out the registration form, and enters the captcha wrong, doesn't enter a password, or makes some other error on the registration page, does this script still query stopforumspam? If so, my original assumption that this script is malfunctioning may be wrong, it's very likely then that some of my users are failing to fill out the registration form correctly which would explain the numerous "allowed registration" messages as they try to fill out the form.

Suggestion:

At present, it seems the name search is not on exact match - at least, if I enter a name as a manual search it's not (e.g., dan will be flagged as matches to daniel, dan123, rodan, etc.).

It should either be eliminated as a potential criterion or made so that only exact matches will work to ban a registration, as it is for email addresses and IP addresses.

This script doesn't use a "search" it checks for a yes or no.

For example, dan would show up as a yes via this page:
http://www.stopforumspam.com/api?username=dan

Does this mod work for vB4?

yes.

You misunderstand me.

The point is how "dan" is checked - and against what. Would you get a "yes" if the member was dan because it matched with daniel, dan2010,123dan, etc.?

Try a manual search: http://www.stopforumspam.com/search?q=dan

My hypothetical member "dan" isn't any of those 500 listed but seems to be considered a match. It should not return a "yes" unless it finds a "dan" in the spam list, not a "jordan" or anything else.

There are 2 downloads available in the mod post.

1 XML file and the Zip folder

The Zip folder includes an XML file too but with another name. Which XML is the newest version?

That's because the search is likely setup to search through the database. The API is likely setup for a precision response.

Example:
http://www.stopforumspam.com/api?username=jor <--- shows as NOT a spammer
http://www.stopforumspam.com/api?username=jordan <--- shows as a spammer

Tommyc, is there any possibility your site is running both this tool and either the ISBOT or the Stop the Registration Bots tools?

When I first installed vBSFS and ISBOT, I was seeing the same symptoms you describe on my site. The vBSFS log showed "registration allowed" but the user never showed up as a completed registration. After carefully checking a few of those users, I realized most of the ones that made it past vBSFS were being blocked by ISBOT because the user registered too fast. I was able to prove this because ISBOT always sent me a "Bot Registration was blocked" explanatory email when it blocked a registration and in each case where a "registration allowed" message was in the vBSFS log but the user never appeared in the vb_user table, I got an email from ISBOT explaining why the user was blocked. Furthermore when I looked at those "registration allowed" users based on the info vBSFS had recorded (username, email and IP address), there was always something about the user that would have caused me to block or remove them manually anyway.

For example, the same username, had been used in previous registrations that were blocked by vBSFS because either the email or IP was already in the vBSFS database even though the username was NOT. However, THIS TIME the bot managed to pick a combo of username, email and IP vBSFS didn't yet have on file and had thus managed to sneak past the vBSFS database tests. Yet after maneuvering past vBSFS the registration was blocked by ISBOT for the reason I mentioned earlier.

After a few days of that, I uninstalled ISBOT and installed "Stop The Registration Bots" instead. I did so because that tool has even more tricks in its bot-blocking reportoire (like hidden/required form fields with names that can easily be changed) that are designed to improve the tool's ability to thwart registration bots.

One thing I did notice, after changing from ISBOT to "Stop The Registration Bots" was I stopped getting the "Bot Registration was blocked" emails that were always sent by ISBOT. I haven't figured out why those emails are not being sent by "Stop The Registration Bots". It may be a bug in the addon that causes this, or the email may be getting blocked by my spam filters or maybe there's an email address issue. All I know is I'm not getting those emails the way I did with ISBOT. Yet, in every case where I've seen a "registration allowed" message in the vBSFS log but the user never appeared in the vb_user table; if I examine the vBSFS log's info carefully, I find either the username, email address (gawab.com or mail.ru, etc.) or IP address tells me the registrant is a bot I'd have manually deleted anyway. In short, though I didn't get the expected email I'm convinced "Stop The Registration Bots" based on the rules it uses.

If my hunch is right, the same thing may be happening to you. When vBSFS fails to detect the registrant as a bot and lets it pass, "Stop The Registration Bots" blocks it based on its design criteria.

That would explain why so many are seeing "Registration allowed" entries in the vBSFS log but can't find the users in vb_users.

Are you running "VBStopForumSpam" and "Stop The Registration Bots" too?

Thanks.

Works like a charm on vb 4.0.2 :D

Any help?

i use the one in the zip

How can I rebuild the postbit like the author mentioned in his topic post?

Also noticed another issue. I created a test account and after registering it showed this message:

Why did it classify me as spammer?

I only run vBSFS, coupled with reCaptcha. I see "registration allowed" for several vBSFS log entries, some of which finally make it through to a complete registration, others don't. These entries are usually within a minute apart.

My theory is that when someone fails the reCaptcha, and is returned to the registration screen displaying the reCaptcha image, the vBSFS routine runs again.... causing multiple log entries for the same person.

Thank you. I didn't get a notification for your reply for some reason but that is reassuring.

The file called "3.54" is for version 3.5x as stated in the mod description

Any help? :(

To be clear, and sorry to offend any sensitive developers and their loyal supporters, our experience with this mod was not "the rare case [of] a false positive'. We found many cases of false positives. We only found a "rare care" in that people would not just get angry and never return, because no legitimate users likes to be called a spammer. The "rare cases
are the ones who will take the time to contact board admins and mods after repeated humiliation and rejection. It is very offensive to users to be rejected as a spammer because they wanted a user name called "dantheman" (just an example) and a spammer somewhere in the world has the same name.

By using CAPTCHA and a few simple bot checking programs, we never had a problem with spam prior to installing this mod, we simply installed it to test it (open minded and very eager for it's success). We have since disabled it because of the false positives; and instead of this rule-based system, we installed a very efficient Bayesian Anti Spam classifier we are very happy with. Our statistical system (AI) works much better than a rule-based mod, especially when the rule base is generated by "community" where it is easy to submit anyone to the DB and time consuming to get out of jail.

Finally, we are not trying to "sell anything" and don't want you do download any of our software (we have none), nor do we want to make a name for ourselves, nor do we have any conflict of interest at all. We simply warn our fellow forum friends that community-driven rule-based systems, all of them, not only this one, are well known for false positives, more than well trained statistical based systems.

What the developers are doing is noble and their heart is in the right place. Folks will benefit and many will simply think the system works well as it blocks both spammers and legitimate potential users .... because most blocked legit users will simply leave your forum, silently, and go elsewhere. This is human nature.

In closing, false positives are not rare in rule-based systems like this (community contributed). What is "rare" is for rejected users to take the time to contact your board and explain what happened. Rejected users generally just get angry and go elsewhere. That is a consequence of running a system prone to false positives.

We disabled the mod and did not do it because we were angry, upset or unhappy with the zeal and passion of the supporters here. We disabled it because it does not work for us because too many false positives. Plus, we don't like having registration checks dependent on a third party server up or down, especially when it adds little value to our forums.