vb.org Archive - vbSpamBuster v0.2

Page 1 of 2

Show 40 post(s) from this thread on one page

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 3.0 Beta Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=34)

- - vbSpamBuster v0.2 (https://vborg.vbsupport.ru/showthread.php?t=76756)

UK Jimbo

02-16-2005 10:00 PM

vbSpamBuster v0.2

Following the release of vB SpamBuster v0.1 earlier in the week I've added some of the improvements based on the discussion about it.

This version is a complete rewrite over v0.1 and adds the major features:

Moves spam into the moderation queue rather than giving the user a "permission denied error"
Has a highly configurable scoring system

As you can see the ruleset is fairly small at the moment I still think that it will catch a lot of spam posts. Any help/feedback from people running the system and either getting "false positives" or want to add any better rules to the default list please let me know.

Installation instructions

Download spambuster-libs.php and spambuster-rules.php.

You may wish to edit the define values at the top of the spambuster-libs.php file. Later on you might want to edit the rules in spambuster-rules.php too.

Upload spambuster-libs.php and spambuster-rules.php into your includes directory.

In both newthread.php and newreply.php find:

PHP Code:


		
			
require_once('./includes/functions_bigthree.php');

after it add:

PHP Code:


		
			
require_once('./includes/spambuster-libs.php');

In newthread.php find:

PHP Code:


		
			
verify_forum_password($foruminfo['forumid'], $foruminfo['password']);

after it add:

PHP Code:


		
			
// do a spambuster test

sb_test($_POST['subject'],$_POST['message'].$_POST['WYSIWYG_HTML']);

Now in newreply.php find:

PHP Code:


		
			
$threadid = intval($_REQUEST['threadid']);

after it add:

PHP Code:


		
			
// do a spambuster test

sb_test($_POST['title'],$_POST['message'].$_POST['WYSIWYG_HTML']);

As ever any feedback would be gratefully received.

Polo	02-17-2005 08:01 PM

Thanks for sharing :) UK jimbo

oldengine

02-17-2005 08:32 PM

Man I'd love to add a whole list of expressions to this kind of code!

==============================
visit our website

visit our web site

promotional

marketing

campaign

thousands

New web site

Urgent! Please Read!

work at home

earn a living

earn quickly

join our company

Fire Your boss

Azhrialilu

02-18-2005 08:11 AM

Just a couple then, huh? LOL

Great idea *clicks install*

oldengine

02-18-2005 12:32 PM

I should explain that I come to vBulletin after running eight years of WebBBS on my server. That script has a "naughty words" function that allows you to have a file of words or phrases that then kicks those posts into the moderated bin as they come in.

This has worked excellently for Nigerian scam posts and the typical overnight spam crap that gets posted from the far east. In fact, it worked so well that I created a duplicate function for the blind email system. This stopped Nigerian scam responses to my classified ads cold!

This is the first time that a hack to vBulletin has even come close to solving these problems. UK Jimbo, I support your work on this hack and I have subscribed to this thread. I realize that this was initially set up to block URL links, but if you have an example as to how my phrase word file could be coded into $spam_exprs, I'd surely appreciate it. Otherwise the number of $spam_exprs in the code could get lengthy.

EDIT: I have tried this hack in my testvb and it works fine. Now for the refinements of kicking those posts into the moderated mode.

nso	02-18-2005 03:57 PM

It would be really great if you could implement the definition of the spam_exprs into the admin control panel. Also a logfunction would be nice to have.

There is very little spam on my board, but I still very much support any effort in this direction. Great work

UK Jimbo

02-18-2005 04:43 PM

Thanks for the feedback everyone.

As time permits I'm going to be looking at:

adding the detection to newreply.php
dropping bad posts into a moderation queue
logging/reporting trapped posts
configuration via the admincp (with possible word list import/export)

nso	02-18-2005 05:37 PM

Quote:

Originally Posted by UK Jimbo

[*]adding the detection to newreply.php[/list]

Actually, to me it seems like adding your newthread.php code in the corresponding place in newreply.php would do the trick

fly	02-18-2005 07:46 PM

With all due respect, wouldn't it just be easier to not let new users post URLs?

UK Jimbo

02-19-2005 12:15 AM

Quote:

Originally Posted by flypaper

With all due respect, wouldn't it just be easier to not let new users post URLs?

Yup - but you can't do that with vBulletin out of the box can you?

Just drop the regular expression http:// into the config for this hack and hey presto you'd be able to do that.

oldengine

02-19-2005 02:09 AM

Quote:

Originally Posted by UK Jimbo

As time permits I'm going to be looking at:

configuration via the admincp (with possible word list import/export)

Make sure that the list isn't just words. As you can see by my list of spam lines above, the rejection power is made by the combination of words into spam phrases.

My actual list is hilarious and lengthy, all taken from actual posts and emails.

UK Jimbo

02-19-2005 09:22 AM

Quote:

Originally Posted by oldengine

For "words" read "regular expressions". Is in the example code above it'll be using perl regular expressions which are very powerful when it comes to pattern matching.

Snow	02-19-2005 10:23 AM

Quote:

Originally Posted by UK Jimbo

Yup - but you can't do that with vBulletin out of the box can you?

Just drop the regular expression http:// into the config for this hack and hey presto you'd be able to do that.

Does this hack work on thread titles as well or is it just for spam in the post itself? The reason I ask is that I had installed (a while back) a different hack that prevented new users from posting urls in posts and signatures but as a result, spammers started posting the urls in the thread titles and their profiles. It was very common to see "visit my homepage from my profile to find out more".

UK Jimbo

02-19-2005 10:29 AM

Quote:

Originally Posted by Snow

Good point.

It'll be easy to include post/thread titles in the list of fields checked so I'll add that in. Good point about profiles. I'll look to see if this is possible but might make things a bit more tricky.

Mink_

02-19-2005 05:02 PM

To get that working in the ACP, I suggest you just make a new database table and call the values into an array with mysql_fetch_array().

Also, I suggest you use the similar_text() function and allow the user to configure what percent 2 words have to be for the replacement to happen. That can get rid of peoblems with people figuring out how to get around the hack by saying things like "Vist my wbsite."

UK Jimbo

02-19-2005 05:11 PM

I've made some progress with this today. It's now dropping threads into the moderation queue.

In the next few days I'll put another release up on here with all the changes I've made and will be reading the regular expressions out of a separate text file. After that's sorted the next release will concentrate on admincp integration.

Quote:

Originally Posted by Mink_

I suggest you use the similar_text() function and allow the user to configure what percent 2 words have to be for the replacement to happen.

I've never spotted that function before! I think I may support both regular expression and similar_text phrase lists. Watch this space.

UK Jimbo

02-20-2005 03:49 PM

Following some further thought and after having used spamassassin a fair bit in the past I think that I'm going to run with a similar rule based system.

A config file has a list of rules within it. Each rule has a score which is added to an overall score for the post should that rule match the post.

At the end of testing a post for "spaminess" the overall score of the post is compared to a configurable "threshold" score. If the score for a post is over the threshold then it's deemed to be spam.

This setup, coupled with perl regular expression and similar_text support should make for a fairly simple but highly configurable system.

After there's a stable release of this code I'd hope that a number of us could worth together to produce a decent standard list of patterns that people can use out of the box. I'm going to need some help with that stage as the forums I admin have fairly low amounts of spam compared to some other sites.

As an aside SpamBuster nipped 8 spam posts in the butt this morning on visordown.com :)

UK Jimbo

02-21-2005 11:52 AM

Moves spam into the moderation queue rather than giving the user a "permission denied error"
Has a highly configurable scoring system

PHP Code:


		
			
require_once('./includes/functions_bigthree.php');

after it add:

PHP Code:


		
			
require_once('./includes/spambuster-libs.php');

In newthread.php find:

PHP Code:


		
			
verify_forum_password($foruminfo['forumid'], $foruminfo['password']);

after it add:

PHP Code:


		
			
// do a spambuster test

sb_test($_POST['subject'],$_POST['message'].$_POST['WYSIWYG_HTML']);

Now in newreply.php find:

PHP Code:


		
			
$threadid = intval($_REQUEST['threadid']);

after it add:

PHP Code:


		
			
// do a spambuster test

sb_test($_POST['title'],$_POST['message'].$_POST['WYSIWYG_HTML']);

As ever any feedback would be gratefully received.

UK Jimbo

02-21-2005 12:07 PM

v0.02 released at https://vborg.vbsupport.ru/showthread.php?t=76756

Polo	02-21-2005 04:09 PM

I dont think you can Make a new thread when having a new version of a hack... I think you should use the same thread... i'm not sure, but I think I read that somewhere... anyways, let me go look around...

UK Jimbo

02-21-2005 04:14 PM

<a href="https://vborg.vbsupport.ru/announcement.php?f=38&announcementid=18" target="_blank">https://vborg.vbsupport.ru/annou...ouncementid=18</a> is from the main release boards (and doesn't mention versions).

I couldn't find anything for the Beta forums either.

Erwin

02-21-2005 08:30 PM

Quote:

Originally Posted by UK Jimbo

https://vborg.vbsupport.ru/announcem...ouncementid=18 is from the main release boards (and doesn't mention versions).

I couldn't find anything for the Beta forums either.

I added it to the rules now. But most hackers know that this is frowned upon. Why create a new thread for a new version? Some hacks have 20 versions - do they get 20 hacks? :)

UK Jimbo

02-21-2005 08:31 PM

Quote:

Originally Posted by Erwin

I added it to the rules now. But most hackers know that this is frowned upon. Why create a new thread for a new version? Some hacks have 20 versions - do they get 20 hacks? :)

Fair play - I'd not used the forums enough to realise this was frowned upon.

Please feel free to merge the two threads in this forum.

oldengine

02-23-2005 11:42 AM

Hopefully, the new thread and subsequent re-merging didn't throw cold water on this hack. It is (to me) one of the more significant hacks on this site. Unfortunately, it is quite hard to find once removed from the list of "Latest Hacks" on the home page.

UK Jimbo

02-23-2005 12:11 PM

Hi oldengine - thanks again for the support.

Michael Morris and I will be working together to merge SpamKiller and SpamBuster together. We've had one brainstorming session over MSN Messenger but will be having more in the coming days and hopefully releasing something soon.

https://vborg.vbsupport.ru/showthread.php?t=76800

as an aside no spam has been sucessfully posted on visordown.com in the last four days :D

Odoin

03-02-2005 11:00 AM

Hi Guys...

Would this work with the NNTP mod? I get tons of spam from the NNTP groups.

Thank you..

oldengine

03-04-2005 11:30 PM

UK Jimbo: I fear that the inclusion of an external site download of material has overburdened this idea.

What started out initially would work within my server and be updated by myself only as needed. The spam quantity that I get does not justify an automated update. You saw part of my word list in an earlier post. That same word file could include the heart of a domain name to block a URL as well.

Is the hack at the begining of this thread complete and sufficient for a self-maintained anti spam system based on a word/phrase list kept in a file?

White Knight

03-07-2005 03:41 PM

Just to give you a heads up. I installed it and its working great. Thanks for the Hack!

oldengine

05-18-2005 01:42 AM

Clicks install, finally. (Just had someone post some crap that I'd rather not have see the light of day.)

I guess I've figured it all out.

EDIT:

# contins the word free anywhere
regexp:a_free:5:any:"/free/i"

So, to add "work at home" as a phrase:

regexp:a_free:5:any:"/work at home/i"

Correct? (of course we could make that one a 10.) :D

007	05-20-2005 06:58 AM

It would be cool if it would autoban users when a certain number of their posts in a row are flagged as spam. :)

oldengine

09-25-2005 03:08 AM

I'm starting to panic! This and a lot of other good mods are either going to go down the drain or I'm going to stay on vB 3.0.9 and forget about 3.5.

Now I see why there are still users on vB 2. :disappointed:

UK Jimbo

09-25-2005 08:50 AM

As soon there is a gold release of 3.5 and we've migrated the forum I use to it then I'll port this mod over to 3.5 (to save my sanity patching in this and a load of other mods every time the forum software gets upgraded).

:)

oldengine

09-27-2005 12:23 AM

Glad to hear this. Spambuster is a valued asset to vBulletin!

motorhaven

10-01-2005 03:20 AM

There's a major hole in this hack...

Create a new thread or post without a spam link or keyword. Post appears (as it should). Now edit the post and put in the spam link/keyword. Post is live, instead of in the moderation queue.

motorhaven

10-01-2005 04:09 AM

Okay, here's the fix:

In editpost.php look for:

Code:

require_once('./includes/functions_editor.php');

Below that add:

Code:

require_once('./includes/spambuster-libs.php');

In editpost.php look for:

Code:

        build_post_index($postid , $foruminfo);



                $url = "showthread.php?$session[sessionurl]p=$postid#post$postid";

                eval(print_standard_redirect('redirect_editthanks'));

Above that add:

Code:

        sb_test( $edit['title'], $edit['message'] ); 

        if ( $foruminfo['moderatenewpost'] == 1 )

        {

                    $thepost = $DB_site->query_first("SELECT * FROM " . TABLE_PREFIX . "post WHERE postid = $postid");



            // First post in thread?

            if ( $thepost['parentid'] == 0 )

            {

                $hidetype = "thread";

                            $DB_site->query("UPDATE " . TABLE_PREFIX . "thread SET visible=0 WHERE threadid = $threadinfo[threadid]");

            }

            else

            {

                $hidetype = "post";

            }

                    $DB_site->query("UPDATE " . TABLE_PREFIX . "post SET visible=0 WHERE postid = $postid");



                    $DB_site->query("DELETE FROM " . TABLE_PREFIX . "moderation WHERE postid = $postid");



                        $DB_site->query("INSERT INTO " . TABLE_PREFIX . "moderation (threadid, postid, type )

                                        VALUES ($threadinfo[threadid], $postid, '$hidetype')" );

        }

Could be done more efficiently but I threw it together in 15 minutes and it works.

UK Jimbo

10-04-2005 12:03 PM

Thanks :)

I'll make sure that it's not an issue when I upgrade the code to work with vBulletin 3.5.

UK Jimbo

10-17-2005 12:00 AM

I've not forgotten about v3.5ing this (and possible enhancing it a bit).

Watch this space...

oldengine

10-18-2005 02:02 AM

Definitely been watching. It's my gateway to 3.5. :)

oldengine

11-06-2005 01:14 AM

<bump>

nwelsh

12-02-2005 07:21 PM

you know what, this hack is really good for the contact us form!

All times are GMT. The time now is 04:43 PM.

Page 1 of 2

Show 40 post(s) from this thread on one page

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.01392 seconds Memory Usage 1,859KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (4)bbcode_code_printable (12)bbcode_php_printable (9)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)pagenav (1)pagenav_curpage (1)pagenav_pagelink (1)post_thanks_navbar_search (1)printthread (40)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start pagenav_page pagenav_complete bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: