Thread: Administrative and Maintenance Tools - Nexia's Web Crawlers Bundle
View Single Post
  #2  
Old 05-11-2009, 12:55 PM
nexialys
Guest
 
Posts: n/a
Default

a note... this engine use the Spiders List from Dream... with his permission.
http://spiderlist.codeforgers.com/

someone said to me that vbSEO Googlemap was doing something similar... hum... yeah, similar, tracking 3 web crawlers -- you have to generate new entries by hand if you want to track each crawler...

this engine is compatible with Paul M's Guest tracking... it's not doing exactly the same thing, but you know, you choose... this engine is not tracking guests activity, just crawlers page hits. (adding a small query per page, but useful when you REALLY need to know what web crawlers are doing on your site) -- really good for site owners showing potential for paid Ads!

Also, there will be a newer version with more settings soon... permissions and blocking, etc

...

as a success story, i must say i had a great visitor this week, that i would never been able to track without this engine... the "Majestics MJ12bot" crawler make 20 times the hits on my site that Google was able to do in months... i checked their site, and it was obvious they were trying to leech the site, not crawl it...
Reply With Quote
 
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01219 seconds
  • Memory Usage 1,756KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD_SHOWPOST
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)post_thanks_box
  • (1)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (1)post_thanks_postbit_info
  • (1)postbit
  • (1)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • reputationlevel
  • showthread
Included Files:
  • ./showpost.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_postinfo_query
  • fetch_postinfo
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showpost_start
  • bbcode_fetch_tags
  • bbcode_create
  • postbit_factory
  • showpost_post
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • showpost_complete