vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.5 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=113)
-   -   Spider Watcher (https://vborg.vbsupport.ru/showthread.php?t=104582)

Zia 06-19-2006 09:58 AM

Quote:

Originally Posted by DementedMindz
the format that is used on vbulletin.com wont work with it... even the one zia post wont either... i posted a few pages back a working one... the format they have one vbulletin.com is wrong and its not even like that stock so i have no clue why they update the list there cause its still a wrong format

DementedMindz..
its really a strange matter...it should not be wrong..its published by Stadler on vb.com.

And the latest one is working for me. the new spider is listed/update/entried there..they are showing over on my spiders.php page..

do u mind to drop few lines on vb.com ,in the thread regarding the error u found ?

I think :
let me xplain :
a site got a spider Google-feedfetcher - 100 times.

Google-Feedfetcher : wastnt listed on the old spiderlist.xml , so it was appeared as un-known spider's block.

after replace the new spiderlist.xml : if again google-feedfecther spider again crawl the site
visit no#101(100 times on old list.xml) , it will be appeard in rss-spider block.

The old data base will not be modified...
its my point of view.

Edit: plz see this hacks description mikelbeck refer the same thread of vb.com. So i dont think mikel use any other format.
Quote:

To make sure that you can decode the maximum amount of spiders, you should grab the latest spiderlist.xml and replace the spiders_vbulletin.xml file in your forumhome/includes/xml/ directory with the one from this thread: http://www.vbulletin.com/forum/showthread.php?t=76662


Thnx.

DementedMindz 06-19-2006 11:23 AM

Quote:

Originally Posted by Zia
DementedMindz..
its really a strange matter...it should not be wrong..its published by Stadler on vb.com.

And the latest one is working for me. the new spider is listed/update/entried there..they are showing over on my spiders.php page..

take a look at stock vbulletin and take a look at stadlers its wrong... i have fixed it on mine but i dont use this no more as of the high number of queries on that page... alot of members here were saying it wasnt working for them either and then once i posted mine with how the format was with the stock vbulletin file it worked perfect for them... i havent looked at the file now but it was way off last time...

here is the top of stadlers

Code:

  <?xml version="1.0" encoding="ISO-8859-1" ?>
  <!DOCTYPE spiderlist (View Source for full doctype...)>
- <!--  $Id$
  -->


here is how it should be just like it is on the stock vbulletin file


Code:

<?xml version="1.0" encoding="ISO-8859-1"?>

<searchspiders>


mikelbeck 06-20-2006 12:04 AM

Ok, people, I've had some time to get back to this. Sorry it's taken so long....

I've got Beta 10 running here: http://www.heavymetalscene.com/spiders.php

Notice how much faster it is. I've pulled out the "rolling up" code and placed it in a file that's run once an hour via a cron job. So the data isn't 100% up to date, but it uses much (much) less queries than the previous version.

What else do you want to see in this release?

DementedMindz 06-20-2006 12:08 AM

how many queries are on that page? also will it auto update once a new spider is added?

mikelbeck 06-20-2006 12:21 AM

Quote:

Originally Posted by DementedMindz
how many queries are on that page?

8! :classic:

No matter how much data is in the database, it's still 8 queries.

http://www.heavymetalscene.com/spiders.php (not a whole lot of spider data): Page generated in 0.19605 seconds with 8 queries
http://www.forumowners.com/spiders.php (quite a bit of spider data): Page generated in 0.51553 seconds with 8 queries
http://www.listedwrong.com/forums/spiders.php (very little spider data): Page generated in 0.14583 seconds with 8 queries

Quote:

also will it auto update once a new spider is added?
When a new spider is added to the xml file you mean? I'll see if I can make it so it'll evaluate the spider xml file and update the "unknown" spiders if there's new data.

DementedMindz 06-20-2006 04:06 AM

yes that is what i ment sorry... yeah that would be great and glad to see you brought them down...

Zia 06-20-2006 08:21 AM

its nice mikelbeck to knowing abt the new version..

please give thought about not lost old spider visitng data after install new version..
:) Less query is really delicious things

[high]* Zia is waiting for the new release[/high]

Stadler 06-21-2006 10:02 AM

Can you please explain what you changed in my file?

btw: you shouldn't relay on the root-tag. Freddie implemented the xml-file before I've finished the DTD. However: online.php of vBulletin works with both root-tags, since it simply ignores it.

Regards,
Christian Stadler

lsgworldl 06-22-2006 10:05 AM

excellent

4yBak 06-23-2006 10:29 AM

why do you save all data without parsing in field PAGE?
Example:
Quote:

faq, index, showthread, member, forumdisplay, calendar, memberlist, register, faq, showthread, showthread, showthread, showthread, showthread, index, showthread, showthread, showthread, showthread, showthread, index, member, showthread, login, member, sho
So, as you see, in this field has many repetitions like showthread, index, member, etc.

I think, in DB must be just one mention about each page.

PS: you can do that in this statistics was include information when spiders visited forums archived pages? Thanks.


All times are GMT. The time now is 11:04 AM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01436 seconds
  • Memory Usage 1,751KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (6)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete