vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)
-   -   vbArchive - Search Engine Indexer for vBulletin (https://vborg.vbsupport.ru/showthread.php?t=47667)

Floris 02-08-2003 02:42 PM

Quote:

Originally posted by TECK
Well, you posted screenshots, so I presumed is done.
Then you should wait before you post anything... :p

And I don't like to sit down. :banana:

Then we just let you wait another day maybe :D

TECK 02-08-2003 02:51 PM

[high]* TECK starts the revolution.... :)[/high]

Floris 02-08-2003 11:47 PM

inph - you got the code done! Stop playing wc3 and start posting :) now EYE even get inpatient

wooolF[RM] 02-09-2003 12:38 AM

]
Quote:

Originally posted by xiphoid
Stop playing wc3 and start posting :)
LOL ;) :cheeky: *sorry for spam* :classic:

inphinity 02-09-2003 01:05 AM

oi wc3 is important :p

useragent checking

Works both standalone and as a very nice compliment to TECK's vbarchive hack.

What does it do?
Allows you to match the useragent for Guests in Who's Online and display custom names/urls for recognised useragents such as Google, Teoma, Inktomi etc

You can also use it for matching the useragent anywhere on vb, ie for Currently Active Users on forumhome. expect a jazzed up online.php with icons next to names for which browser people are using sometime.

Why?
Got bored of looking up IPs then digging around the session table trying to find out which guests were really web robots also nosey to see who was reading the archives.

Install
Instructions in the file, should work with 2.2.x
Install time, 3-5mins. level, medium.

List of Detected Web Robots (thanks to TECK for listing the main ones)
Last updated: 08/02/03 10pm GMT
googlebot www.google.com Google
gulliver www.northernlight.com Northern Light
ia_archiver www.archive.org The Internet Archive
internetseer www.internetseer.com Internet Seer
linkalarm linkalarm.com Link Alarm
mercator www.research.compaq.com/SRC/mercator Mercator
openbot www.openfind.com.tw Openbot
pingalink www.pingalink.com PingALink Monitor
psbot www.picsearch.com/bot.html PicSearch
scooter www.altavista.com AltaVista
slurp www.inktomi.com/slurp.html Inktomi
turnitinbot www.turnitin.com/robot/crawlerinfo.html Turnitin
slysearch www.turnitin.com/robot/crawlerinfo.html Turnitin
zeus http://www.waltbren.com/products/zeu...rnet_robot.htm Zeus Internet Marketing
zyborg www.wisenutbot.com WiseNut
teoma www.teoma.com Teoma/Ask Jeeves

-- these last 3 are generic and will display the useragent on who's online with a link to robotstxt.org where you can look up the useragent for obsecure and new bots.

spider Web Spider
spyder Web Spyder
crawl Web Crawler
robot Web Robot

Screenshots?
Who's online:
https://vborg.vbsupport.ru/attachmen...&postid=351495
https://vborg.vbsupport.ru/attachmen...&postid=351832
https://vborg.vbsupport.ru/attachmen...&postid=351533

Currently Active Users
https://vborg.vbsupport.ru/attachmen...&postid=351831

enjoy,
inph

thanks to floris for screenshots and testing

inphinity 02-09-2003 01:15 AM

just a quick note if you're using TECK's guest_crawler

you should add the trailing dot to the ip addresses

PHP Code:

    'Google' => '216.239.46[b][COLOR=yellow][SIZE=3].[/SIZE][/COLOR][/b]',
'Inktomi' => '66.196.72[b][COLOR=yellow][SIZE=3].[/SIZE][/COLOR][/b]' 

so that you dont match (ie an octet at the beginning):
*216.239.46*
*66.196.72*

with the trailing dot you will only match:
216.239.46.*
66.196.72.*

:)

also a minor point for the vbarchive installer

the templates added are set to templatesetid=-1
which is fine but in vB's upgrade scripts, lines like:

Code:

$DB_site->query("DELETE FROM template WHERE templatesetid=-1 AND title<>'options'");
tend to obliterate peoples templates :)

i would recommend adding the templates twice once with -1 and then again with the style id's so they appear as custom templates (with default content)

limey 02-09-2003 05:48 AM

hey is 1000 hits by googlebot in 2 days good?

edited the number from 609 -1000

TECK 02-09-2003 07:52 AM

Quote:

Originally posted by inphinity
just a quick note if you're using TECK's guest_crawler

you should add the trailing dot to the ip addresses

PHP Code:

    'Google' => '216.239.46[b][COLOR=yellow][SIZE=3].[/SIZE][/COLOR][/b]',
'Inktomi' => '66.196.72[b][COLOR=yellow][SIZE=3].[/SIZE][/COLOR][/b]' 


Very good point, I edited the file. :)
Also, I'm going to add your mod in the first post, with credit of course. Great job. :)
About the templates, is really easy, simply run again the installer, no need to recustomize the templates because they are automatically saved, if they are edited (not original), so your work is not lost...

EDIT: Link added. Check no. 5 in Forum Optimizations section. ;)

Floris 02-09-2003 10:52 AM

Glad you liked the hack teck

wooolF[RM] 02-09-2003 01:42 PM

]@ inphinity > big thanx for adding that feature to show web robots on home page! and also thanx for releasing this addon! Very nice :)

PS: I think you should also release it as a hack so people will know it exist and it will be possible to add this hack to the fine collection of vb.org hacks :)

Cheers!

wooolF[RM] 02-09-2003 08:39 PM

]:: 51 members, 46 guests and 32 web robots (Google) on the boards

Nice to see it on the main forum page! Thanx again for the great addon! :D

Mike Gaidin 02-09-2003 08:45 PM

Quote:

Originally posted by inphinity
oi wc3 is important :p

useragent checking

Works both standalone and as a very nice compliment to TECK's vbarchive hack.


In the instructions for the modification of functions.php it just has a piece of code, but no instructions as to where to put it. Where does it go?

wooolF[RM] 02-09-2003 08:57 PM

]Find
PHP Code:

?> 

Add ABOVE this code the code mentioned in the attached file

limey 02-09-2003 09:13 PM

Looks like those first googlebots were scouts and they sent the deepcrawlers over the past few days. Here they are in action.

wooolF[RM] 02-09-2003 09:16 PM

]They are crawling my forum right now :D
I have about 120 users online + 32 Google bots :D

codewebs 02-09-2003 10:39 PM

isnt this a bot? inktomi2-wat.server.ntl.com

if so how can i add it to inphinity 's hack?

inphinity 02-10-2003 08:19 AM

Quote:

Originally posted by codewebs
isnt this a bot? inktomi2-wat.server.ntl.com

if so how can i add it to inphinity 's hack?

that isnt a web robot its:

Inktomi's Traffic Server network cache (transparent cache aka web proxy)

basically sits between ntls end users and the net and saves them bandwidth by caching files and requires no end user configuration

an old new article: http://www.internetnews.com/xSP/article.php/44591

--
on a side note
updated instructions for location of code in functions.php (thanks J-OST and wooolF)

## At the bottom of functions.php
## Just BEFORE ?>

Floris 02-10-2003 12:38 PM

Most users ever online was 81 on 10-02-2003 at 03:41.

whoo hoo :)
No members online last night but a few, and just a few normal guests. 75+ search engines crawlers

scary :alien:

hypedave 02-10-2003 01:31 PM

Hey Teck,
I finally got this puppy installed, the archive is working great, but now when I go to my vbhome page, I get the following error

Fatal error: Cannot redeclare archive_nopermission() (previously declared in /home/ochroma/public_html/alpha1/global.php:228) in /home/ochroma/public_html/alpha1/forum/admin/functions.php on line 2497

What have I done wrong at 3am in the morning?

line 2497 looks like this

PHP Code:

function archive_nopermission() 


TECK 02-10-2003 02:04 PM

You need to install the vbHome addon instead (search engine indexer), that does this.
http://www.teckwizards.com/forum/sho...=&threadid=617

mini2 02-10-2003 04:05 PM

Just installed. Works well, just got to wait for the little spiders to come and crawl all over my site.....

www.mini2.com/forum/archive/

Thanks very much.

Floris 02-10-2003 04:27 PM

By the way, is the latest archive hack xhtml compliant, or can we expect an update for 1.4 (maybe with the other tweaks too) ? So it feels more natrual together with vbhome4

TECK 02-10-2003 05:57 PM

I just found a site to submit for FREE to several engines. It doesn't hurt if we try it...
http://www.ineedhits.com/add-it/free/

TECK 02-10-2003 06:53 PM

Quote:

Originally posted by xiphoid
By the way, is the latest archive hack xhtml compliant, or can we expect an update for 1.4 (maybe with the other tweaks too) ? So it feels more natrual together with vbhome4
No is not, Floris. It uses the old vBulletin code.
For those who want a true XHTML compliant solution, you will have to install vbHome (lite) and use it's indexer add-on or to edit your current templates to make it also XHTML compliant.
I will not release a new version.

saint_seiya 02-10-2003 08:48 PM

Triple post! *applause*

Hwulex 02-10-2003 10:45 PM

w00t w00t! :D

Google came a nosing round today. Up to 28 on at one point, unfortunately I missed it, but was reliable informed by members of the event :D

TECK 02-12-2003 06:57 PM

Quote:

Originally posted by saint_seiya
Triple post! *applause*
Deleted all extra posts... for some reason VB was not responding... :)
Hwulex: So far the highest number of crawlers I saw on my site was 38. Some of other users got hit up to 47...

KeithMcL 02-13-2003 05:39 PM

I just installed this hack and it's working great (www.webdevforums.com/archive/). About to go submit my link to all SE's mentioned in your first post.

I was going to install the other SE friendly hack, but after reading the problems other members had I decided on trying this one and it was dead easy to install. Good Job :)

BTW, I also changed the metatags keywords and description to my own ones.

Now lets hope that I get indexed by all the SE's I submit too.

WoodiE 02-14-2003 04:52 PM

TECK,

Yey ANOTHER great hack from you! I had it installed within only a few minutes and it work great ( http:www.RCNitroTalk.com/forum/archive )

Thanks!


-Michael

TECK 02-15-2003 10:47 PM

You are supposed to have it in http://www.mydomain.com/forum/ your archive.
You are not posting related to the vbHome add-on, right? Because it would be the wrong thread.

TECK 02-15-2003 11:31 PM

What is the URL of your forums? Is the root?

TECK 02-15-2003 11:44 PM

It will not work, unless you hack all files.
Your archive files must be in the /forum folder, as instructed in the readme file (I quote from):
Quote:

[ROOT FOLDER] (no files to upload here)
---[FORUM] - upload here the following files:
- archive.txt
- forumdisplay.txt
- htaccess.txt
- showthread.txt

Schorsch 02-15-2003 11:57 PM

what's wrong I only get text ??

click

TECK 02-16-2003 12:41 AM

Do you have the ForceType directive enabled? Don't think so. ;)
Check with your host if mod_mime is installed and that the directive is enabled.

Schorsch 02-16-2003 10:54 AM

Quote:

Originally posted by TECK
Check with your host if mod_mime is installed and that the directive is enabled.
Hi TECK,

mod_mime is installed. how can I enable the "directive" ??

mini2 02-16-2003 11:49 AM

Teck, just to restate this has been working a treat, not sure when Google will update it's index (is it 1st of each month) but Inktomi and Google in particular have been crawling ALL OVER the archived threads (also seen a few other drop by, but google's gone nuts).

www.mini2.com/forum or www.mini2.com/forum/archive/index.html

:)

Top marks.

tkeil69575 02-16-2003 11:17 PM

this is a great script teck - thanks.

one question though. would it be possible to only let admins see the the search engines on "who is online", while still showing normal guests to users?

tina

glenvw 02-17-2003 12:22 AM

oops....

Ignorant me. I just upgraded to VB 2.30
Guess what? I screwed up your hack that was working so good:
http://www.yes-its-free.com/vbbs/archive/

Is there an easy fix ( I hope!)

TECK 02-17-2003 06:47 AM

Ya, run the installer again, it will re-add the templates. :)

glenvw 02-17-2003 10:56 AM

thank you sir....


All times are GMT. The time now is 10:47 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.03895 seconds
  • Memory Usage 1,833KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (4)bbcode_php_printable
  • (9)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete