Hi Kevin, Thanks for answering quickly. What a pleasant surprise.

Now what I mean with when the spambots have learned to deal with that is that after a while I am pretty certain spambots will become aware of how they are prevented from registering and start delaying.

From what I was able to see there is no where the time the bot took. I have set it currently to post to forum, which is great as it publishes the spamming email addresses, and exposes them to email harvesters LOL. I thought I saw something about a log file but I was unable to find where that goes. It certainly is not in the settings of the spambot blocker. And ideally the log file would have all the information so that I can run an analysis after the fact.

Oh, I see. When you said "spambot" I thought you meant this mod. Anyway, as I said I don't monitor what this blocks myself, but I haven't heard anyone say anything that makes me think any spambots are smart enough to adapt to this. We discussed it earlier in the thread and I think the consensus is that there's a small percentage of vb sites using this scheme for protection, so it wouldn't be worth programming a spambot to deal with it. I think if they fail to successfully register, for whatever the reason, they just move on.

There is no log file, although I've been talking about it for a long time. At this point I don't think there ever will be one, but who knows, I might get motivated some day to release an update with a few new features.

First of all the bots themselves aren't intelligent. They are part of a program, programmed by humans.

Using timers for well over three years and studying bot behavior carefully, I have seen no evidence adjustments are made for timers.

Think about it - first you as a XRumer programmer have to know there is a timer. Then you have to know there is a minimum and a maximum setting. Then you have to guess the setting, site to site.

It won't happen, because they are hitting 1000s of sites all at the same time, and individual results don't draw any attention - it's like fishing with a big seining net. Automation, bulk and speed are all that matters.

It's easy to think you're personally being targeted and studied, but you're not. You're just one in the many 1000s.

I am certain for now this is precisely what happens, until there are enough sites where they are starting to investigate why they no longer can register as they 'used to'. The whole spam, anti spam is after all a whack a mole game. It was that for many years and will be for many years to come. So basically a fair assumption is that over time spambots will learn that it is much more effective to wait for 'x' seconds and then send the registration screen back. That would be a simple change in the spambot and over time they will do that. Its just a question when, not if.
I would offer my help in getting a log file into the mod if nothing else because I'd love to have it and I am fairly certain some other folks would be able to get some insight as well. e.g. Right now I believe its about 70% yahoo.com addresses responsible for the spam registration attempts, but I'd love to be able to have real data, not just "gut" data.

It is not a "simple change."

Using timers for well over three years and studying bot behavior carefully, I have seen no evidence adjustments are made for timers.

Think about it - first you as a XRumer programmer have to know there is a timer. Then you have to know there is a minimum and a maximum setting. Then you have to guess the setting, site to site. Impossible. And not even doable, since you are dealing with 10s of 1000s of sites you are hitting. There's also nothing in XRumer that tracks failed registration attempts anyway.

It won't happen, because they are hitting 1000s of sites all at the same time, and individual results don't draw any attention - it's like fishing with a big seining net. Automation, bulk and speed are all that matters.

It's easy to think you're personally being targeted and studied, but you're not. You're just one in the many 1000s.

Conventional "wisdom" says the spammers are all powerful, all-talented, smarter than us gods of some sort, who will always figure a way around any hindrance we dummies out here might devise. And it just ain't so.

I agree with you. I think it's a given that if this were a built in feature of vbulletin, it would have been defeated long ago. It would actually be fairly trivial for a programmer who knew what they were doing.


Yeah, I had a version of this that I reworked and never released that had all the data collected, it just needed the actual logging code. I'm not sure if I still have that around. But the major obstacle for me is that I haven't looked at this code in a long time. Well, it's not much of an obstacle but it's been enough to keep me from thinking about it.

He would also have to know the settings max and min, for every site with this mod he might encounter. And since he can't possibly, he's more or less urinating in the wind. Might get some improvement, might not. It would be a guessing game.

This is all mostly XRumer doing the automated registration and posting, and currently it has nothing in it at all that allows the user to set delays. They can program that into it I am sure, but without somehow being able to detect the time settings they are going to be hit and miss at best, and miss more than hit.

Agreed that if this was a native feature of vBulletin it would get alot more attention than it does - but then we get to the details, where the devils always are.

I have been programming for SO many years before I quit being actively involved in code I can tell you from my own knowledge and experience, THAT change is not only simple, its trivial. It may not be implemented yet because there are not enough site out there so that it became a nuisance, but a programmer who could not implement: "Wait at least x number of seconds and after that wait between y and z seconds to make it look its a human in 10 minutes flat should stop programming. He or she is not worth the money they are being paid for.

Now OTOH, making the spam bots multi-threaded or multi-process or both, That is not a trivial undertaking if it is not designed into it from the get go. I have no insight into spambots and if they are multi-threaded or multi process designed, but if they are, the change to wait is so trivial that it is laughable.

Which brings be back to gathering information: Kevin, would you be willing to let me take a stab at the code? And just to be clear: I do NOT want to take over or step on anybodies toes. I just offer some time to implement a feature *I* would like to see and then hand the software back to its owner and inventor.

Why dont you pm him the new changes to see what he thinks

Sounds trivial until you realize the x and y settings are different for every site.....

Because he specifically asks not to be PM'ed and I honor this wish.

He allows for exceptions - we PM all the time.:D

Why do we have to discuss this? Seriously, If *I* would be a spambot programmer I would make my input to look like a very slow user, say fill the form between 1.5 and 2 minutes. Done. You can never find out from the time any more if its a spambot or a (slow) user.

You would fail the max time check in this Mod, and be rejected.

Next.

Ant then you would not make as much money as someone else competing against you, cause they would get wayyy many more registrations than you.

You are cutting off users shorter than 2 minutes? Your users must be way more intelligent and faster than the average user out there. Anyway, I am done discussing this particular issue.

Apparently you did not read the multithreaded comment.

Who in the heck takes longer than two minutes to register to a site, it this a senior citizen home? LMAO

That is the point, you have no way of knowing what the min/max settings are, in order to consistently program around them.

Do like I did, and actually BUY your copy of XRumer. Perchance to see what it is capable of, and what it is not capable of. Find out how it really works.The way this Mod works, if they fail the max time check they have to start over.

See? They are not cut off.

That max time check can be anything, and it is designed to stop the script kiddie stuff you are proposing.

You say you can better the coding of this, but when I said send the changes to the coder it sounds like all talk no action. If you can do it, then do it don't back down

I'd be willing to bet that there's a window of time that's common to almost everyone using this or similar mod. A suggested window has even been posted here. It's got to be about the same for everyone because it has to be comfortable for a user. You *could* set the min to 2 minutes and use the timer to keep people from submitting too fast, but that would be very annoying.


Yeah, go ahead. I'll apologize ahead of time because this was kind of hacked together. It was originally put together in about 2 hours and was very simple, then I made a half-assed attempt to rework it. But hopefully it's not too bad that you can't follow it.

Never can tell what the settings are. Making it luck at best, to program around this. Hit or miss.

And where you hit? They change the settings.

It's not as simple or trivial as has been said.

Max, we've discussed this before and as I said then, we'll just have to disagree. I don't want this thread to go on all night going back and forth when no new info is being added.

But I do give you full credit for recognizing the value of this approach, and reviving Calorie's old mod. This has hundreds of installs, and the other mod (that shall not be named) had hundreds, and I'm guessing a lot of people either didn't hit install or implemeted their own, so I'd say there must be at least a couple thousand installs, and it's all because you paid someone to update it.

Well not to belabor the point Kevin, but minutes is the setting for max time, in your mod. Default is 1 minute.

I have it set at 2 minutes. I imagine most will be at one, but would change if they start getting registrations through.

So, a XRumer programmer would have to figure out what minimum time, and what max time, to plan for, for 10K sites.

Hit and miss.And as you might know, there is a all new approach, coming. Works so well I am able to allow guest posting on my board, and still get no spam through.

Well, I just meant 2 minutes, like 120 seconds. I'd guess that 45 seconds would be a good target. Or you could start at 30 and if it fails, increase it next attempt. You could ban the ip, but as we know finding another ip address for the next attempt isn't a problem.

In your mod we can't set seconds, for max time. Must be minutes.:D

Minimum time is where we can enter seconds.

Plus I never got to thank you for picking up the fumbled ball so to speak, and taking the time to code this yourself with all original code. It's one of the best products on vBorg.

Thanks!

Right, but I was talking about the minimum time. As an extreme example of the point I was making, maybe your registration form takes an average of 45 seconds for humans to fill out. Well, to defeat hypothetical spambots that know how to delay, you could say I'll make my minimum 120 seconds and humans will have to wait for the timer to run out. But that would be annoying.

Anyway, let's just enjoy the fact that bots don't seem to be getting through.

I'll take this offline, PM sent.

I have logged many, many bot registration attempts, they go anywhere from, 0.623 seconds to 100001 seconds, but most of them hit under 10 seconds.

And of course, the email reports your mod generates do tell us the times used by the bots, and why they failed register. And I spent a whole year charting them. Speed is their whole business.

I never used the "wait for time to expire" feature though. It tips off the minimum time setting.:D

I have added a little bit of functionality to spambot blocker. Mainly it is to write all the attempts of spambots trying to register into a log, well, actually a DB table, so that those attempts can be data-mined afterwards. E.g. that way I can see with certainty, what I already suspected: more than 80% of all spammers are using @yahoo.com email addresses.

I am attaching a few screenshots of the results of the data captured for two weeks now. If anybody is interested in the changes, post here and I will create a list of changes.

A live version of those pages can be found at: http://www.rx3forums.com/forums/pages.php?pageid=4

Added another feature to this mod: Now it does add the spammers IP address to the list of banned IPs. That way the first time they try to register is also their last time. Since that list could potentially get rather long I am currently discussing to change this and put the list of blocked IPs into the DB into its own table. That way access would not slow down dramatically over time but stay rather rather constant and would only be one (rather simple) SQL statement.

Thoughts?

This is a brilliant idea IMO. Would there be a way to choose in the admin panel if you wanted to have the IP's added to the banned IP list in Settings > Options > User Banning Options or via a separate DB table?

apleschu has been kind enough to share his additions, but I haven't had a chance to release them yet. But I will.

Regarding the banned ip lookup, I don't really know what to recommend when it comes to that kind of thing. But I remember a few years ago we tried this thing called zbblock which automatically blocked IPs, and the way it works is to ban one it appends it to a file (separated from the preceding one by a comma). Then to check an ip it does file_get_contents, then uses the php function substr_count. Anyway, just another idea. Edit: ...but now that I think about it, that was probably done because zbblock doesn't use a database at all. Although it does have the advantage of being able to do it's work before vbulletin does it's initialization.

Right now I have the blocked IPs in the default blocked IP list. *I* am a fan of using standard facilities unless there is a need not to, so for now I see no reason not to keep the list of IPs in the default spot.

In addition I have set it up in a way so that the list is also sorted, so that it can be searched faster by a binary search rather than the linear search that is currently implemented. Either someone else is going to implement the binary of at some point in the future when I get to have too much time on my hands again I will change/add that

kh99: I am going to IM you what you need. Specifically because I made this already IPv6 safe ;) So there are a few changes to the package I sent you, that was only IPv4

OK, sounds good. I guess I thought you wanted to change it because you thought keeping it in the setting might end up causing problems when it gets big, but I don't have any specific reason to think that myself.

honestly, I don't know yet. for now I let it coast and see where that goes. if I find the energy and time to implement a binary search then that would still be LOTS faster than any DB search. Although the memory use would be unpredictable, especially if that would be used by forums that have 10's of thousands of banned IP addresses. And with the voracity the spammers are trying to register at my tiny forum it can't take too long before I have that.

I guess I will cross that bridge when I get to it. Right now this works and it works nicely. Once a spammer is identified their IP is blocked for good and they will never again use processor cycles or even given a second chance to see if they got smarter.

You guys are the pros when it comes to figuring out which methods work best with the least overhead. If using the vBulletin "built-in" blocked IP section works good enough, then it should be just fine for my small forum as well. I generally like mods / plugins that don't make changes to databases anyway.

Just watching the SpamBot Stopper log the IP's of every attempt, i'm sure the IP lists will be in the thousands in no time. They're almost always IP's from China anyway.

EDIT:
I forgot to ask. Does the latest version here in this thread need to be downloaded and installed to get gain this functionality?

hmmmm.... this Missing or invalid check value was mention but never answered.

I still get it ever so often...