vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 3.5 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=113)
-   -   Google sitemap for the vB Archives. Redirect human and robots. (https://vborg.vbsupport.ru/showthread.php?t=93980)

lierduh 10-09-2005 05:46 AM

Quote:

Originally Posted by jdingman
Looks great so far. One question about mod_rewrite

using that redirects if you're using forums.domain.com. What about if you're using domain.com/forums/? What mod_rewrite would you use for that redirect?

(not exactly for me because I can probably get it working, but anyone else that might need this as well.)

Without testing, I think
RewriteRule ^forums/index.php$ forums/ [R=301,L]

Should do.

falter 10-09-2005 06:24 AM

Quote:

Originally Posted by lierduh
I have a new version ready to be released. If anyone wants, you can download this and try out before I put together the package.

I still need to do the documentation for the modifications of index.php and global.php files.

I don't know if this is due to any mods I have (which I'm pretty light on), but when I run your script directly (not using cron), I get the following output:

Quote:

Warning: array_keys(): The first argument should be an array in /path/to/my/stuff/forums/includes/class_core.php on line 1453

Warning: Invalid argument supplied for foreach() in /path/to/my/stuff/forums/includes/class_core.php on line 1453

Warning: array_keys(): The first argument should be an array in /path/to/my/stuff/forums/includes/class_core.php on line 1472

Warning: Invalid argument supplied for foreach() in /path/to/my/stuff/forums/includes/class_core.php on line 1472

Unable to add cookies, header already sent.
File: /path/to/my/stuff/forums/archive/forums_sitemap.php
Line: 1

Removing the "unset($_COOKIE);" from line 56 helps get the script to run, but, since my cookies are still there, all my private forums get sitemapped, too. so, I just moved down the stuff in the block above, and everything works.

So, I go from this:
PHP Code:

if (function_exists('log_cron_action'))
{
    global 
$index_zp;
    global 
$debug_log;
    global 
$max_url;
    unset(
$vbulletin->userinfo);
    
$vbulletin->userinfo['userid'] = 0;
}
else
{
    if (
$run_by_vb_Scheduled_Task_only)
    {
        exit(
"Script can only be run by vB Scheduled Tasks. Set \$run_by_vb_Scheduled_Task_only to 0 if you need to run manually");
    }

    unset(
$_COOKIE);
    
$specialtemplates = array();
    require_once(
CWD '/includes/init.php');


to this

PHP Code:

if (function_exists('log_cron_action'))
{
    global 
$index_zp;
    global 
$debug_log;
    global 
$max_url;
    unset(
$vbulletin->userinfo);
    
$vbulletin->userinfo['userid'] = 0;
}
else
{
    if (
$run_by_vb_Scheduled_Task_only)
    {
        exit(
"Script can only be run by vB Scheduled Tasks. Set \$run_by_vb_Scheduled_Task_only to 0 if you need to run manually");
    }

    
$specialtemplates = array();
    require_once(
CWD '/includes/init.php');
    unset(
$vbulletin->userinfo);
    
$vbulletin->userinfo['userid'] = 0;



lierduh 10-09-2005 07:04 AM

I remember now, someone else reported this as well. I think it might be php5 related. I don't have php5 to test, so I think I won't unset cookies then.:)

Thanks.

Quote:

Originally Posted by falter
I don't know if this is due to any mods I have (which I'm pretty light on), but when I run your script directly (not using cron), I get the following output:



Removing the "unset($_COOKIE);" from line 56 helps get the script to run, but, since my cookies are still there, all my private forums get sitemapped, too. so, I just moved down the stuff in the block above, and everything works.


dutchbb 10-09-2005 11:00 AM

Quote:

Originally Posted by falter
your robots.txt should be accessible at the root of your domain (http://www.mydomain.com/robots.txt). this is the only place that spiders know to check.

if you're trying to explicitly define specific files (ex. /forums/showthread.php), then you should define that entry in your robots.txt file. there's no point in not putting the ".php" at the end (ex. /forums/showthread), it doesn't buy you anything. it can actually have a negative impact if your entries aren't defined well. say you're trying to tell search engines to ignore "/forum/s.php" (this is just hypothetical). if you were to just put "/forum/s" in your robots.txt, then, in addition to blocking "/forum/s.php", you'd be blocking "/forum/showthread.php", "/forum/search.php", "/forum/showgroups.php", anything else where the url starts with "/forum/s" .... as you can see, it's important to be as specific as possible, otherwise you risk shutting spiders out of huge chunks of your site.

Thank you. I read it on this site, the guy seems to be some sort of guru about vbulletin SEO: http://forum.time2dine.co.nz/seo-vbu...lletin-98.html

I have a few questions (also for the author of this thread: )

What does http://www.vbseo.com have that this hack doesn't provide. Is this worth buying, or is it basically the same?

What do you think about the tips/hack provided on this site: http://forum.time2dine.co.nz/seo-vbu...lletin-98.html he has nr1 ranking on google for "vbulletin SEO" keywords.

lierduh 10-09-2005 11:32 AM

Basically my hack only lets Google index the real contents of the forums using vB archives. I do not think it is neccessary to let Google index both the full version threads and the archives. For more details and reasons, please read my open post.

Unreal Player 10-09-2005 01:53 PM

Ok, My site has been pending for almost 2 days. They say "several hours" wtf? anyone else get this?

jdingman 10-09-2005 02:16 PM

Is it crucial that I change permission for the root or my forum directory? I haven't changed them and it's been working fine. I did change my /archive/ to 755, but not ./

does it make that much of a difference?

trilljester 10-09-2005 04:01 PM

Quote:

Originally Posted by jdingman
Is it crucial that I change permission for the root or my forum directory? I haven't changed them and it's been working fine. I did change my /archive/ to 755, but not ./

does it make that much of a difference?

Well, as long as the web server "user" process has access to write to the root forum directory and archive/ then 755 is fine, assuming that the user owns the directories. The 55 part will keep others from writing to those directories.

xtreme-mobile 10-09-2005 04:50 PM

ummm all is goinbg well but what the helldo i have to do for step 3 it doesnt make any sense to me :(

any help would be fantastic :D

falter 10-09-2005 04:54 PM

hey lierduh,

I've been playing around a bit with the robot detection. I snagged a bunch of code from "online.php", hacked it up a bit, and came up with this (as a drop-in replacement for the "is_robot_visit" function. This one uses the spiders_vbulletin.xml file, which I recommend people updating. The 3.5.0 gold version is fairly vanilla. I got an updated one from here: http://www.vbulletin.com/forum/showp...5&postcount=12

Anyway, here's the change to global.php (this is assuming that you have the very latest version of lierduh's code :) )

PHP Code:

 /**
    * Return true if visited by a robot.
    */
    
function is_robot_visit()
    {
        require_once(
DIR '/includes/class_xml.php');
        
$xmlobj = new XMLparser(falseDIR '/includes/xml/spiders_vbulletin.xml');
        
$spiderdata $xmlobj->parse();
    
        if (
is_array($spiderdata['spider']))
        {
            foreach (
$spiderdata['spider'] AS $spiderling)
            {
                if (isset(
$_SERVER['HTTP_USER_AGENT']) AND preg_match("#"preg_quote($spiderling['ident'], '#') . "#si"$_SERVER['HTTP_USER_AGENT'])) {
                    return 
true;
                }
            }
        }
        unset(
$spiderdata$xmlobj);
        return 
false;
    } 

There's all sorts of extra markup in the xml for ip ranges and such, but I'm just goign to match against the user-agents, for now.


All times are GMT. The time now is 02:50 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01555 seconds
  • Memory Usage 1,774KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (3)bbcode_php_printable
  • (6)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete