Go Back   vb.org Archive > vBulletin 3 Discussion > vB3 General Discussions
  #1  
Old 05-30-2004, 07:43 PM
joeychgo's Avatar
joeychgo joeychgo is offline
 
Join Date: Mar 2004
Location: Chicago, IL
Posts: 933
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default Problems I think are related to hacks ----

ok -Boardreader.com has been trying to crawl my site and are having problems..... They sent me this email..... Anyone have any ideas???



Quote:
Here is another problem we are seeing:

I'm getting two forum file paths for your site:

http://www.lincolnvscadillac.com/forum.php?

and

http://www.lincolnvscadillac.com/index.php?

The index.php is the forum file path that I'm normally seeing on vbulletins, that takes you right to the forums home page. Now in your case the forums file path for your forum home page is located at the forum.php forum file path. We have never seen that before, so I'm going to have to do some coding I think. I will check with my partner on Tuesday about this because that is out of my scope of programming.

Any ideas why your forum home page is like that? We have no problem recognizing and following your topic file path, http://www.lincolnvscadillac.com/forumdisplay.php?f=1
This path takes us to your forums threads page.

The strange thing is when you open http://www.lincolnvscadillac.com/forumdisplay.php?f=1 and scroll over the folders on top of this page:

Lincoln vs Cadillac.com (http://www.lincolnvscadillac.com/index.php?) you go to your homepage
and the topic path
Our Community Forums (Site Basics) (http://www.lincolnvscadillac.com/forumdisplay.php?f=85) you go to your forums page witch contains:

1) Site Basics Forum
2) About Us Forum
3) Suggestions & Comments Forum
4) Contest & Raffles Forum
5) Community Discussions Forum
6) Welcome Wagon Forum
7) Computers and Software Forum
8) General Discussion Forum
9) Anything Goes Forum
10) Politics & Current Events Forum
11) Sports Forum
12) Movies & Television Forum
13) Music & Concerts Forum
14) Original Artwork Forum
15) Events & Meets Forum
16) Home Theatre, Home Audio Forum
17) The Junk Drawer Forum
18) For Sale by Members Forum
19) Cars For Sale Forum
20) Auto Parts For Sale Forum
21) Non-Auto Stuff For Sale Forum

Compared to our grabbed forums following the Forum File Path and the Topics File Path: we pull back 64 unique forums. I think the structure of your forum is throwing us off. As I said before on Tuesday I will ask Al (my business partner) about this or maybe if you can set up a mirror site with just a list of forums using index.php and forumdisplay.php we can just crawl that on your backend. That might be a real pain for you so I'm thinking I will just ask Al about this on Tuesday.

Joey, I'm really sorry about all this trouble. I have never seen this happen before but on a good note we are finding one more way message boards are structured. The real funny thing is we can grab the forums just fine, but can't crawl the threads. STRANGE. Let me know your thoughts.

Scott
Reply With Quote
  #2  
Old 05-30-2004, 08:19 PM
tamarian tamarian is offline
 
Join Date: Oct 2001
Location: Canada
Posts: 1,205
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

When they access index.php they don't go to your forum, they go to the portal page. Your index.php shows vBAdvanced, instead of the default vBulletin home page.

This explains why they only get some forum links, from the "Latest Forum Topics" block.

To got access to your full forum display, they need to access your forum directly, which you have set to forum.php
Reply With Quote
  #3  
Old 05-30-2004, 08:42 PM
joeychgo's Avatar
joeychgo joeychgo is offline
 
Join Date: Mar 2004
Location: Chicago, IL
Posts: 933
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

shouldnt their crawler go from the VBA page to the forum page via the link on the nav bar??????? SHouldnt it just trickle down?
Reply With Quote
  #4  
Old 05-30-2004, 08:51 PM
tamarian tamarian is offline
 
Join Date: Oct 2001
Location: Canada
Posts: 1,205
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by joeychgo
shouldnt their crawler go from the VBA page to the forum page via the link on the nav bar??????? SHouldnt it just trickle down?
According to the email they sent you, no. They expect vBulletin boards to list all forums on index.php.

Quote:
Now in your case the forums file path for your forum home page is located at the forum.php forum file path. We have never seen that before, so I'm going to have to do some coding I think
Reply With Quote
  #5  
Old 05-30-2004, 08:53 PM
joeychgo's Avatar
joeychgo joeychgo is offline
 
Join Date: Mar 2004
Location: Chicago, IL
Posts: 933
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

yes I know -- but look at the bottom half of his email --- he only has the top half of my forums listed, and the only thing I did to my forums was to place them in columns -

Might be their problem --- I dunno -- its worth finding out dont ya think?
Reply With Quote
  #6  
Old 05-30-2004, 09:12 PM
tamarian tamarian is offline
 
Join Date: Oct 2001
Location: Canada
Posts: 1,205
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

Quote:
Originally Posted by joeychgo
yes I know -- but look at the bottom half of his email --- he only has the top half of my forums listed, and the only thing I did to my forums was to place them in columns -
That was because they followed the link to forum 85, which in your case is only one category (not a forum). This is standard behaviour for vBulletin, clicking a category name gives tou a page with forums under that category.

Quote:
Might be their problem --- I dunno -- its worth finding out dont ya think?
I don't know. I've never heard of them before, and don't know how they crawl forums.

You might want to send them a link to this thread, or a link to vBAdvanced, so they can adapt their crawler to recognize forums using portal pages for their homepage, and how categories behave.
Reply With Quote
  #7  
Old 05-31-2004, 02:51 AM
joeychgo's Avatar
joeychgo joeychgo is offline
 
Join Date: Mar 2004
Location: Chicago, IL
Posts: 933
Благодарил(а): 0 раз(а)
Поблагодарили: 0 раз(а) в 0 сообщениях
Default

ive done just that thanks tam
Reply With Quote
Reply

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT. The time now is 05:22 PM.


Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2024, vBulletin Solutions Inc.
X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.04430 seconds
  • Memory Usage 2,218KB
  • Queries Executed 11 (?)
More Information
Template Usage:
  • (1)SHOWTHREAD
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)ad_showthread_beforeqr
  • (1)ad_showthread_firstpost
  • (1)ad_showthread_firstpost_sig
  • (1)ad_showthread_firstpost_start
  • (5)bbcode_quote
  • (1)footer
  • (1)forumjump
  • (1)forumrules
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (1)navbar
  • (3)navbar_link
  • (120)option
  • (7)post_thanks_box
  • (7)post_thanks_button
  • (1)post_thanks_javascript
  • (1)post_thanks_navbar_search
  • (7)post_thanks_postbit_info
  • (7)postbit
  • (7)postbit_onlinestatus
  • (7)postbit_wrapper
  • (1)spacer_close
  • (1)spacer_open
  • (1)tagbit_wrapper 

Phrase Groups Available:
  • global
  • inlinemod
  • postbit
  • posting
  • reputationlevel
  • showthread
Included Files:
  • ./showthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/functions_bigthree.php
  • ./includes/class_postbit.php
  • ./includes/class_bbcode.php
  • ./includes/functions_reputation.php
  • ./includes/functions_post_thanks.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • showthread_start
  • showthread_getinfo
  • forumjump
  • showthread_post_start
  • showthread_query_postids
  • showthread_query
  • bbcode_fetch_tags
  • bbcode_create
  • showthread_postbit_create
  • postbit_factory
  • postbit_display_start
  • post_thanks_function_post_thanks_off_start
  • post_thanks_function_post_thanks_off_end
  • post_thanks_function_fetch_thanks_start
  • post_thanks_function_fetch_thanks_end
  • post_thanks_function_thanked_already_start
  • post_thanks_function_thanked_already_end
  • fetch_musername
  • postbit_imicons
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • postbit_display_complete
  • post_thanks_function_can_thank_this_post_start
  • tag_fetchbit_complete
  • forumrules
  • navbits
  • navbits_complete
  • showthread_complete