Log in

View Full Version : Question about Session URL and Spiders (the big G)


StarBuG
11-06-2005, 08:38 PM
Hi

I have just noticed that if I run gsitemap (a windows prog that spiders a website to generate a sitemap) catches all my forum urls with a session ID.

I added the Gsitemap to the spider indentification xml (spiders_vbulletin.xml)

<spider ident="Gsitemap">
<name>Gsitemap Spider</name>
<type>searchspider</type>
</spider>

The spider is now propaly displayed in the who is online however the Spider STILL gets the sessions.

Any Idea where this could be set?
I need my spider to get urls without session id to create the google sitemap

StarBuG
11-08-2005, 09:54 AM
anyone? come on PLEASE

Andreas
11-08-2005, 10:35 AM
// conditional used in templates to hide things from search engines.
$show['search_engine'] = ($vbulletin->superglobal_size['_COOKIE'] == 0 AND preg_match("#(google|msnbot|yahoo! slurp)#si", $_SERVER['HTTP_USER_AGENT']));


...

// Hide sessionid in url if we are a search engine or if we have a cookie
$vbulletin->session->set_session_visibility($show['search_engine'] OR $vbulletin->superglobal_size['_COOKIE'] > 0);

StarBuG
11-08-2005, 02:42 PM
perfect!

Thank you