Ultimate Spambot Killer

[y2ksw]

This product is a natural evolution of many single solutions. It is an attempt to integrate in vBulletin the features other applications have, without requiring a lot of code, and without obviously giving the code to the spammers.

With this software, your board connects to one of the most powerful email checking engines available. The engine itself is maintained remotely and can currently be accessed at the production server. After debugging and wide-scale test, it will be moved to vbulletin.it.

Please read first the included help before installing!


BETA RELEASE

1.0.4: FIX misplaced bracket
1.0.5 More reliable connections and a few new options, updated help. This version should be the last one before release.

Online Help (http://uspam.pagerobot.com/uspam.html)

Please continue discussing here (https://vborg.vbsupport.ru/showthread.php?t=195437) (stable release).

[popowich]

What exactly is it connecting to? Something along the lines of SpamAssassin?

Why would someone make themselves dependent on an external resource?

What happens to a new post of your service is not available?

I'm not bashing you, just trying to ask some educated questions. :)

If you'd rather PM details to me that is fine. Thanks!

-Raymond

[Hornstar]

Can you elaborate a bit more. You say it is the ultimate spambot killer, but in what way? in bots spamming or in bots registering?

what does it do with the spam, etc. how do I know it is not removing legit posts, etc. we need more detail.

[River_rush]

What exactly is it connecting to? Something along the lines of SpamAssassin?

Why would someone make themselves dependent on an external resource?

What happens to a new post of your service is not available?

I'm not bashing you, just trying to ask some educated questions. :)

If you'd rather PM details to me that is fine. Thanks!

-Raymond
according to the xml file I think its http://www.pagerobot.com/

[y2ksw]

What exactly is it connecting to? Something along the lines of SpamAssassin?

Why would someone make themselves dependent on an external resource?

What happens to a new post of your service is not available?

I'm not bashing you, just trying to ask some educated questions. :)

If you'd rather PM details to me that is fine. Thanks!

-Raymond
It's connecting to Spamhaus, and if not available or overloaded, to alternate services. It has also a local database where known spammers are cached for some time.

If somebody was so badly spammed as some of my servers (hundreds of spammers and tens of thousands of posts), and any service would help me to get rid of them, I would choose it. I have made my research and found that nothing exists which compares to SpamAssasin, and so I coded the connection to those databases. Also I added some of the features I had already, such as the local spammer database and formal email checking. All together they do a really good job.

New posts are not affected, only new users.

Please feel free to ask, I will reply to all questions, all but the exact coding ;)

[y2ksw]

Can you elaborate a bit more. You say it is the ultimate spambot killer, but in what way? in bots spamming or in bots registering?

what does it do with the spam, etc. how do I know it is not removing legit posts, etc. we need more detail.
It does not allow to register spam bots and notoriously known spammers from certain IP's.

If a user registers, it's email (and optionally IP) is compared to the local database (mine) first and if a match was found, rejected.

Flow:

An incoming email address is formally checked and rejected if invalid.
If no IP match was found, it connects to the world-wide spammer databases in a certain order, and if there is a match, the IP is stored locally and the user email rejected.
If the email address apears not to exist (MX lookup & initiation of email transmit, quit before sending), the user email is stored to the local database and will be rejected.
If none of the above applies, the user may register. At this point it is sure the email account is not coming from a known spammer and the address is really valid.You may test this service by issuing:

http://uspam.pagerobot.com/index.php?email=info@domain.tld&ip=64.201.124.1

for a known spammer, and anything else for valid or invalid email addresses. If no IP is specified, the IP test against the world-wide databases is not performed.

If the emal is invalid, a blank page or 0 is returned. If the service is manually down, a negative value is shown. If the email is valid, a 1 is shown.

[y2ksw]

The final mail check is currently down - I have to fix a timeout issue on many requests. Thus, emails return 1 if the spamhaus test passed.

[Delphiprogrammi]

www.pagerobot.com (http://www.pagerobot.com) i take it he has a brandfree licence if not ....... :p

[y2ksw]

www.pagerobot.com (http://www.pagerobot.com) i take it he has a brandfree licence if not ....... :p
In fact I have a branding free license ;)

[y2ksw]

The final mail check is currently down - I have to fix a timeout issue on many requests. Thus, emails return 1 if the spamhaus test passed.
The service is up again. Also Gray-Listed mailboxes are accepted now. If they don't exist, the registration will bounce anyway.

[cedivad]

Really good job. Thank you man!

[g00gl3r]

Reserved.

[pedigree]

this is what www.stopforumspam.com has been doing for several months, accumulating a database of 80,000 known forum spammers and provides code (and a vbulletin mod) for access/submitting to it, allowing spambots to be denied registration.

[y2ksw]

this is what www.stopforumspam.com (http://www.stopforumspam.com) has been doing for several months, accumulating a database of 80,000 known forum spammers and provides code (and a vbulletin mod) for access/submitting to it, allowing spambots to be denied registration.
Nice :) I haven't heared about it though, else I probably wouldn't have started this adventure.

Edit:

I've looked into your project and it seems to be pretty much the same we have, besides the website. We probably never will have more than a single webpage and concentrate on backend coding only. I'll soon look into it a little better. Sorry if this sounds like stockfish, but I'm falling into pieces :) Goodnight!

[Alfa1]

this is what www.stopforumspam.com (http://www.stopforumspam.com) has been doing for several months, accumulating a database of 80,000 known forum spammers and provides code (and a vbulletin mod) for access/submitting to it, allowing spambots to be denied registration.
Yes, but this also seems to check the validity of email addresses, which is a big plus. Maybe this can be combined with vbstopforumspam?

An incoming email address is formally checked and rejected if invalid.
If no IP match was found, it connects to the world-wide spammer databases in a certain order, and if there is a match, the IP is stored locally and the user email rejected.
If the email address apears not to exist (MX lookup & initiation of email transmit, quit before sending), the user email is stored to the local database and will be rejected.
If none of the above applies, the user may register. At this point it is sure the email account is not coming from a known spammer and the address is really valid.

[pedigree]

Yes, but this also seems to check the validity of email addresses, which is a big plus. Maybe this can be combined with vbstopforumspam?

Sure it could but the problem with a single source of active SMTP scanning is that mail servers could just block connections or worse, tarpit them by reducing the TCP window size to 0, therefore just killing any scanning machine thats busy and invalidating any results, possibly timing out causing unhandled errors... if I ran a mail server and had it hit by a non-stop mail scanning site, it would get the tarpit treatment.

Take gmail for example, it waits to the end of the conversation before rejecting email addresses. This could pass email testing but be an invalid email address. Other sites will just accept everything and bin non-valid recipents later. Lets not forget (and I havent looked at the source) you can have a mail server running without a MX record. stopforumspam.com does tests to govern if an email is a valid format but it doesnt connect to a remote server to test if we got a 220/451/452 result code.

A single point for email scanning isnt a great method, as much as I like making it hard for spammers. By doing active scanning, your results are skewed by the remote end which you have no control over. vBulletin already has a method for controlling email validity, in that you must confirm a registration. For those sites without email confirmation, well, they deserved to get spammed really. While this doesnt stop a member registering to that point and hoping that their details will still be visible somewhere, email tests could be done on your server in a more distributed manner. There is no reason why those results couldnt then be submitted to a central site for further testing.

Im not trying to rag on anyones effort, god knows I had enough myself when I released my mod but Im just playing devils advocate :)

[y2ksw]

Sure it could but the problem with a single source of active SMTP scanning is that mail servers could just block connections or worse, tarpit them by reducing the TCP window size to 0, therefore just killing any scanning machine thats busy and invalidating any results, possibly timing out causing unhandled errors... if I ran a mail server and had it hit by a non-stop mail scanning site, it would get the tarpit treatment.
Right. However we have to start at some point, and if we had the fear to move against spammers, we shouldn't move at all. I'm aware of the risks, but at the end we are not sending any email, and just try to find out if the mailbox exists, with a simple mail conversation which may come any time from any server. It doesn't mean we are spammers, and those who protect do know. The point of one source is easy to understand - it is a start project and every server is an expense, I'm sure you know. And on the other hand it never will be only one source, because once everybody starts to share, spammers are defeated from many places. And in fact this is the only way.

Take gmail for example, it waits to the end of the conversation before rejecting email addresses. This could pass email testing but be an invalid email address. Other sites will just accept everything and bin non-valid recipents later. Lets not forget (and I havent looked at the source) you can have a mail server running without a MX record. stopforumspam.com does tests to govern if an email is a valid format but it doesnt connect to a remote server to test if we got a 220/451/452 result code.
GMail seems to handle email as any other server. The few cases an email results valid but effectively isn't will be handled by the mail server. But if you ever had a forum with thousands of users and frequent email bounces just because they don't care even to register, you will also understand that a basic connection test isn't the worst thing. In fact this is what many people ask for: more reliable email addresses.

A single point for email scanning isnt a great method, as much as I like making it hard for spammers. By doing active scanning, your results are skewed by the remote end which you have no control over. vBulletin already has a method for controlling email validity, in that you must confirm a registration. For those sites without email confirmation, well, they deserved to get spammed really. While this doesnt stop a member registering to that point and hoping that their details will still be visible somewhere, email tests could be done on your server in a more distributed manner. There is no reason why those results couldnt then be submitted to a central site for further testing.
We have to build that. As far as I know there is no public database where to look at if an email address is valid and existing. It is rather the contrary, since otherwise the spammers could just download that database and send spam to 1.000.000.000 world-wide users instead of only 1.000.000 at a single time.

Im not trying to rag on anyones effort, god knows I had enough myself when I released my mod but Im just playing devils advocate :)
We can share experiences, databases, code. But it must be us to start the fight, waiting for other people to join and actively help will take a long time. Since your site is already up for quite some time (a year or so), you will know how much it takes, unless you already had a lot of helpers. ;)

[y2ksw]

We have added an online Help (yet under development), which explains what this service does. You may access these pages from here (http://uspam.pagerobot.com/uspam.html).

[y2ksw]

Yes, but this also seems to check the validity of email addresses, which is a big plus. Maybe this can be combined with vbstopforumspam?
I believe it can be combined easily. As far as have seen by now, the collected data does not differ a lot from what we store, and additional features can be shared once the service has been tested by many forums.

I'm not sure if one central SMTP mailbox test is practicable though. Best would be a cluster of providers offering this particular service and client applications will choose randomly or sequentially from that list.