Go Back   vb.org Archive > vBulletin Modifications > Archive > vB.org Archives > vBulletin 3.7 > vBulletin 3.7 Add-ons
FAQ Community Calendar Today's Posts Search

Reply
 
Thread Tools
Keyword weight based spam detector Details »»
Keyword weight based spam detector
Version: 0.1.0, by HuangA HuangA is offline
Developer Last Online: Jun 2013 Show Printable Version Email this Page

Category: Mini Mods - Version: 3.7.0 Beta 5 Rating:
Released: 07-07-2008 Last Update: 07-08-2008 Installs: 40
Uses Plugins
Translations Is in Beta Stage  
No support by the author.

I coded this one because I constantly had to moderate and / delete them lengthy lame cell phone ads on vBulletin.com's forums and my own forums. You know, buy iphone, ipod touch, noika blah blah blah sony ericsson blah blah blah etc. etc. etc. email us, we're legitimate business in a country you've never heard of, blah blah blah spam.

While Akismet does work on filtering them out, some times they still leak through.

I know there's two other keyword based tools that automatically adds things to moderation queue (One from SirAdrian and one from tweakmonkey), but it doesn't work too well for me, because I run an iPhone / iPod Touch site and I can't have those keywords on auto spam for simply appearing. So, here's what I did for mine...

What does this product do?
  • Adds 1 vBulletin Options setting group, with 4 settings
  • Allows you to define a list of keywords with associated score
  • Allows you to set a threshold for automatic moderation
  • Allows you to set a threshold for automatic rejection
  • Allows you to set a post count limit for posts to be scanned
  • Adds 1 plugin which gets ran at newpost_process
  • Adds 1 plugin which gets ran at editpost_update_process

How does it work?
1) You configure your keyword list, and score weight. For example, I use this list:
Code:
Nokia|0.5
iPhone|0.5
iPod Touch|0.5
Order|0.5
HTC|0.5
Samsung|0.5
Sony Ericsson|0.5
hotmail|0.5
$|0.5
usd|0.5
url|0.3
email|0.5
The list basically means each time the plugin sees "Noika", it will get a score of 0.5, $, 0.5, etc. etc. A tally of all the score is totaled, and
2) You configure your moderation score, for example, I use 50.
3) You configure your rejection score, for example, I use 100.
4) You configure your exemption post count, for example, I use 5.

When a new post is being created (this could be a thread, or a reply, doesn't matter, they both trigger newpost_process hook), the plugin will count how many times each keyword appears, and total the score. If it is higher than or equal to the moderation score, it will tuck the post into moderation queue. If it is higher than or equal to the rejection score, a standard vBulletin error message is shown to the user.

How much overhead does this add? Realistically, not much... depending on amount of keywords used, I'd say most likely under 0.05 seconds of your CPU time for each post. If you are really that worried, you can set your exemption post count to something lower, and so lesser posts are scanned. Default is 5 right now.

This have been tested on 3.7.0 Beta 5, and 3.7.2. I see no reason why it would not work on 3.6.x series, too.

Change log
0.0.0 => 0.1.0
  • Changed error message to use vBulletin error message screen instead of die()
  • Added option for omitting after certain post count (default 5)
  • Added default values to options
  • Fixed options not appearing after product import (I forgot to export them for 0.0.0)
  • Added scanning for editing post (AJAX doesn't seem to give error... I'll work on that for 0.1.1 later)

Show Your Support

  • This modification may not be copied, reproduced or published elsewhere without author's permission.

Comments
  #22  
Old 09-08-2009, 09:22 PM
TimberFloorAu's Avatar
TimberFloorAu TimberFloorAu is offline
 
Join Date: May 2008
Location: Brisbane
Posts: 2,264
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Installed on 3.8.4

JUst so I can get my head round all this, if we set a weight of 1.0 for say the word viagra, and that is used 48 times within a new post of a member who has less than 5 posts, and our threshold score is say 50... then their post gets POSTED ?
Reply With Quote
  #23  
Old 09-09-2009, 06:00 AM
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Posts: 102
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Assuming moderation threshold score set at 50, reject threshold set to 100 and it scans only people with < 5 posts...

User 1 with post count of 50:
Quote:
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
Post appears.

User 2 with post count of 2:
Quote:
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra
Post appears.

User 3 with post count of 4:
Quote:
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
Post goes to moderation queue.

User 4 with post count of 1:
Quote:
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
viagra viagra viagra viagra viagra viagra viagra viagra viagra viagra
Cialis. Prescription. ... oh, and watch replicas!
Post doesn't even get posted.


Of course, you should have more than one word, and they'd run into the limit a lot sooner. You can also have things like $, http://, url=, etc. on your list. You can also adjust the threshold limits. You can also adjust the keyword weight (IE: viagra = 50 points right away).

Long story short, you control the magic.

Nokia cell phones anyone?
Reply With Quote
  #24  
Old 09-09-2009, 07:32 AM
TimberFloorAu's Avatar
TimberFloorAu TimberFloorAu is offline
 
Join Date: May 2008
Location: Brisbane
Posts: 2,264
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Cheers Andy ! Good one.
Reply With Quote
  #25  
Old 09-15-2009, 10:17 AM
HansiB HansiB is offline
 
Join Date: May 2003
Posts: 5
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Where do i put/find the keyword and threshold settings?
Reply With Quote
  #26  
Old 09-16-2009, 09:17 AM
HuangA's Avatar
HuangA HuangA is offline
 
Join Date: Jun 2004
Posts: 102
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

You can enter your keywords and weights in Admin CP > vBulletin Options > Keyword Weight Anti-spam.
You will need to find and craft your own set of filter to cater the spams that you are getting though. The default set really only targets one set of commonly recurring ad for certain cell phone seller...
Reply With Quote
  #27  
Old 05-07-2010, 08:09 AM
washingtonboise washingtonboise is offline
 
Join Date: Mar 2009
Posts: 86
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Very impressed with this mod, found that negative values also work.
e.g. if you have a forum about selling cialis and don't want people posting about viagra, you can give 'cialis' a negative value so that if someone happens to use both in the same post (maybe they're comparing the differences and that makes it NOW an on-topic post), chances are the system will allow the conversation unless they say viagra multiple times.
Reply With Quote
  #28  
Old 05-09-2010, 07:03 PM
InsaneNutter InsaneNutter is offline
 
Join Date: Apr 2007
Posts: 27
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Works perfectly to stop bots spamming warez / trojans too.

For example hotfile.com is now worth 1 point on our forum, so when a bot spams links to warez hosted on hotfile.com the post is instantly rejected

Surprised this mod has not gained more attention, I can see this been a great asset to our forum!

Keep up the great work.
Reply With Quote
  #29  
Old 06-07-2010, 07:21 AM
maxc0der maxc0der is offline
 
Join Date: Mar 2007
Posts: 3
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Impressive... Just wanted to say thanks.

It hasn't been updated for a while but I hope we can see a 4.0 version.
Reply With Quote
  #30  
Old 06-22-2010, 05:05 PM
EtaiWix EtaiWix is offline
 
Join Date: Jun 2010
Posts: 23
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

I'm so sorry for the noob question, but how do I install it?

I downloaded the XML file but I'm at a loss for what to do...

Thanks for the help!
Reply With Quote
  #31  
Old 06-22-2010, 05:52 PM
EtaiWix EtaiWix is offline
 
Join Date: Jun 2010
Posts: 23
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Nevermind, I found it... THANKS!!!!

This is GREAT!!

Now if only someone who knows some coding can help me with my other problem...
Reply With Quote
Reply


Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 08:03 AM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.04418 seconds
  • Memory Usage 2,315KB
  • Queries Executed 25 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)bbcode_code
  • (4)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)modsystem_post
  • (1)navbar
  • (6)navbar_link
  • (120)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (3)pagenav_pagelink
  • (11)post_thanks_box
  • (11)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (11)post_thanks_postbit_info
  • (10)postbit
  • (11)postbit_onlinestatus
  • (11)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • pagenav_page
  • pagenav_complete
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete