vb.org Archive
Page 54 of 84 « First 4445253 54 555664 Last »
Show 40 post(s) from this thread on one page

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 4.x Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=245)
-   -   Miscellaneous Hacks - Ban Spiders by User Agent (https://vborg.vbsupport.ru/showthread.php?t=268208)

qpurser 12-10-2013 03:29 PM
I added "Majestics MJ12bot Spider" to the list some time ago but for some reason he keeps coming back and I can see he is viewing threads.
I added them as "majestics" and "Majestics MJ12bot Spider"

When they are on the list are they not banned and redirected automatically?

Avros 12-10-2013 06:31 PM
Not all Bots will adhere to or obey the text file. If anything it will provide them with more information than you want them to have.

Only a minority of spiders adhere to those rules.

Max Taxable 12-10-2013 06:55 PM
Quote:
Originally Posted by Avros (Post 2467411)
Not all Bots will adhere to or obey the text file. If anything it will provide them with more information than you want them to have.

Only a minority of spiders adhere to those rules.
They can ignore "robots.txt" but they can't ignore this. It's not "robots.txt" it has teeth.

Here's my current blacklist:
SemrushBot
SeznamBot
http_requester
BIXOCRAWLER
KomodiaBot
QACC
Plukkie
botje
Opera/9.80
'Mozilla
CompSpyBot
ScreenerBot
Chrome/15
python-requests
ZumBot
Ruby
Add Catalog
Genieo
socialbm_bot
Ezooms
omgilibot
Go http package
JikeSpider
Python-urllib
Iron
sputnik
Xenu
Wotbox
200PleaseBot
360Spider
Indy Library
Sogou
SEOstats
baiduspider
beta.statsit.com
statsit
SiteIntel
Yandex
GomezAgent
Nesotebot
DCPbot
AOL Advertising R&D
DataCha0s
aiHitBot
Apache-HttpClient
Zend_Http_Client
ReverseGet
XXX bot Content
vBSEO
spbot
OffByOne
thyroidbuzz
AcoonBot
coccoc
xpymep
proxyproxy2884
AppEngine
start.exe
Semiocast HTTP client
Firefox/3.6.23
Firefox/3.6.3
TurnitinBot
curl
SwpLc/1.6
GrepNetstat.com
news bot
AskTbPTV
checks
panopta
App3le
PhantomJS
AlwaysOnline
SISTRIX
proximic
CRAWL-E/0.6.4
WebMoney
HTMLParser
oBot
UnisterBot
ERACrawler
MSIE 2
MSIE 3
MSIE 4
MSIE 5
MSIE 6
crawler4j
NCSA_Mosaic
Rippers
80legs
Firefox/3.5.6
YaBrowser
majestic
EasouSpider
User-Agent
FunWebProducts
I am not seeing anything from "majestic" in online.php. Or any of these for that matter.

Simon Lloyd 12-10-2013 08:52 PM
What you need to do is check their UA when they are online, the chances are that what you've entered in the list is NOT in their UA that's why they still get through :*

Stevenwi 12-15-2013 07:13 AM
Installed :)

sv1cec 12-31-2013 11:10 AM
Marked as installed, but it messes up my vB 4.2.2 Suite after a while. So I removed it.

ozzy47 12-31-2013 11:12 AM
Messes it up how, what you put is not going to help anyone debug your issues.

sv1cec 12-31-2013 11:21 AM
Correct, apologies, but I wanted to remove it quickly, since users were complaining.

Well, after I install it, and I start getting emails from the hack, the normal layout of the page is messed up, and the site is shown in a very strange way, you could say that no formatting is done in the page, only characters shown.

I tried it twice and I had two friends email me saying that the layout was messed up. I checked and sure enough it was. I'll reinstall this and see if I can grab a shot of before and after.

ozzy47 12-31-2013 11:29 AM
Yeah that is strange, I have not seen issues like that. Perhaps if it does it again, instead of un installing, just disable it, incase someone needs to debug it on your site.

sv1cec 12-31-2013 11:39 AM
Reinstalled it and waiting.

One strange thing is that some of the emails take way too long to be sent. I just received an email which was sent about 70 minutes ago. Is that normal?


All times are GMT. The time now is 09:12 PM.
Page 54 of 84 « First 4445253 54 555664 Last »
Show 40 post(s) from this thread on one page

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02047 seconds
  • Memory Usage 1,742KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 
Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 
Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete