vb.org Archive - Search Engine Friendly Archives and store

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vB4 General Discussions (https://vborg.vbsupport.ru/forumdisplay.php?f=251)

- - Search Engine Friendly Archives and store_sitemap directory (https://vborg.vbsupport.ru/showthread.php?t=324269)

Snowhog

01-23-2017 02:47 PM

Search Engine Friendly Archives and store_sitemap directory

How and where does vBulletin create and store Search Engine Friendly Archives?

We have the store_sitemap directory in our robots.txt file as a disallow:

disallow: /store_sitemap/

I'm not sure this should be a disallow. Should the files in this directory be available to spiders/crawlers?

bzcomputers

01-26-2017 10:30 PM

Quote:

Originally Posted by Snowhog (Post 2581237)

You should "allow: /store-sitemap/"; without allowing this directory to be scanned/indexed will stop Google and every other search engine from being able to download and see the site as you wish it to be seen.

Sample Page:

Quote:

<url>
<loc>http://www.domain.com /</loc>
<lastmod>2017-01-01</lastmod>
<changefreq>weekly</changefreq>
<priority>0.8</priority>
</url>

Snowhog

01-26-2017 11:05 PM

Thank you. I had already made up my mind that it shouldn't be a Disallow and removed it (as an entry) from our robots.txt file.

All times are GMT. The time now is 09:50 PM.

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.01156 seconds Memory Usage 1,710KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (2)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)post_thanks_navbar_search (1)printthread (3)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: