The robots.txt file does not exist unless you create it.

Make sure you use a plain text editor and save it in ASCII or plain text format, NOT unicode.

Where do you get this information at so i dont have to ask 100 questions ?

What do i put on the txt file ?

If i put this text file on my root then set the robots up to read that file will it direct the bots to vbseo_sitemap/data/xml file

<a href="http://www.robotstxt.org" target="_blank">http://www.robotstxt.org</a>

Thanks for the link Ill read up on them ;)

Look like all the robots.txt does is Disallow things what do i need to put in there for it to find the sitemap.xmls

DM

Answered previously in this thread. See https://vborg.vbsupport.ru/showpost....5&postcount=98

Thank you ... Heres my robot.txt file Does it look right to you?

Not quite. Change it to the following:

Note the blank line before Sitemap: and no <> brackets. Also the URL must be a complete URL, including http:// etc.

See http://www.sitemaps.org/protocol.php#submit_robots

THANK YOU !!!!

Also See ...

http://www.vbseo.com/f44/newbie-here...o-hicky-13619/

Im really trying to get this to work

google bots are not indexing my pages for somthing like 2 weeks..
what could ne the problem? im using the generator for along time..
Results 1 - 10 of about 305,000 from atraf-f.net.
i do think that google is indexing new forums in my forums but there is no sign to new theards , there are only theards from 5\5 (maybe 1 or 2..) the rest are from april.. it a big issue for me.

what can i do? my forum is in hebrew, maybe one of the coders can see in the source if there is a problem that is cousing the bots to unindex my theards pages, thanks alot.

also, i went in one of the files, http://www.atraf-f.net/sitemap_46.xml.gz the latest one that was generated by the generator and i didnt see any pages from May 2007, there seemto be alot of pages from 2006 etc', how can i make the generator to include only theards \ pages from a certain date? thanks alot.

nice thanks

Any resons why this should not work also on 3.7.x?

Can some one please send correctly download link to the latest version, thanks a bunch!

There is no 3.7x yet. I assume you mean 3.67PL1 - it works fine in that version.

As far as I know, the link to the latest version of the add-on is right here.

after we make the site map we can delete th .htaccess file??

or not???

No. The .htaccess file tells spiders where to find your sitemap file. Only delete it if you use the robots.txt Sitemap: option.

Google sitemaps tell Google WHAT to index, not when. Generating a sitemap is no guarantee of increased spidering. I can't read Hebrew but your forum seems to be an active one. The problem is your home page is only PR1, which does have an influence on how often and how deeply Google will spider your pages. Of course, that may be different in your regional Google...

If you're using this add-on, that may be the incorrect URL for the sitemap.

By default, the add-ons saves current sitemaps at http://www.atraf-f.net/vbseo_sitemap - look there. Also make sure you either have the .htaccess file present and updated with the mod_rewrite instructions so the SEs can locate the correct sitemap or use the robots.txt instruction to accomplish that task.

Thanks so much DJ Baxter!

Yes I was referring to 3.6.7 PL1 (sorry)

I have now successfully installed sitemap generator v 2.0. And after running a sitemap generator report it confirms successful pings to Yahoo & Google.:)

I am concerned about one thing however and would appreciate feedback/confirmation on one thing:

When I try to access my http//:www.myforum.com/vbroot/sitemap_index.xml.gz page I get an 500-error page. : (Yes, I have set the permissions to 777 for my vbseo_sitemapfolder/data/ folder.

I have been advise to add "Rewrite/vbroot/" in my ".htaccess" file. Problem is my .htaccess file is invisible on my server. Any tips on that OR any other solution to the 500 -error page where the site map is supposed to show.


Thanks alot.
Aussie Regards
Patrick

Don't have it 777, have it 755, some servers have extra security enabled which causes this. Switching it to 755 would work fine.

Thanks so much for your tip Dismounted.

I have tried setting the CHMOD permissions to BOTH 777 and 755, however and I STILL get 500 error page... when trying to access the
http://www.myforum.com/vbroot/sitemap_index.xml.gz file

(I can of course see the actual file is inside its folder).

Try 644.

thanks, but google had stoped indexing new theards, it will index anything else, but threads, tomshing is wrong, the .htaccess file os fine, i can view the files by atraf-f.net/site... just beacause of the .htaccess file, they are stored in ttp://www.atraf-f.net/vbseo_sitemap/data, and the last sitemap that the generator made is the one i gave http://www.atraf-f.net/sitemap_30.xml.gz you can view it throw this URL but it's actually stored in vbseo_sitemap/data/ , and i don't see any new theards \ messages in the GZ files that the generator is generating..

a google bots had used to index our threads on a daily basic, and now we done some hanges in the theards pages and after redoing those changes google bots are not indexing new pages from my board, im telling you, the bots used to index our pages every day, and now there in no new thread page in google for sonthing like 2-3 weeks, what seems to be the problem?

If the directory isn't write enabled (766 or 777), how do you expect the add-on to actually write the sitemap file?

The .htaccess file is usually a hidden/system file. If you check it with an FTP program, you can usually set the program to show hidden files.

Can you post the portion of your .htaccess file with the mod_rewrite lines here?

Dj Baxter, Thanks,

Yes, The datafolder is set to CHMOD 777 (thanks).

I use Cute FTP Home ver 7.2 I am unsucessfully pulling my hair to find an option to enable viewing of hidden files-system files.. If I could I would be post the .htaccess code-s here and grateful for you you to have a look.

When and if I find a way to access the .htaccess file I will post it, thanks

Cheers

Patrick

I don't use Cute FTP. However, I've used several others, my current favorite being FileZilla.

Look in options or settings for something like "Show hidden files" (settings | interface settings | remote file list in FileZilla) or check the help file (I remember in WS_FTP one had to enter an obscure file string to enable this).

SE Bots Activity Log not working help me please

I had installed a previous version of this mod. Now that I upgraded to the 3.6.x version, I'm getting this message when I try to "run the Site Map Generator." I checked permission and changed them as the .doc instructed so.



Thanks

lets say the generator had generated the sitemaps, and there were somthing like 5000 posts that day, those posts will be in the last sitemap file? (let say num' 30 out of 30 )

cause i don't think that links to new posts are generated into the sitemap files, i cant find any new link to new theards in my forum in my sitemaps files..
what is the problem?

*Installed*

hi
why its saving the filre here vb/vbseo_sitemap/data/sitemap_index.xml.gz
i need it to be under vb/
i did as you instracted but no changes
i did change htaccess file
and every thing looks to be just fine but still i can not have the files added to my root folder

thanks

This version has a kind of serious bug if your forum is a big one:

it adds another cron job which do the exact same thing to the old one which will double the queries.

I think this only happens with an upgrade which is the most method.

To fix this bugs yourself forum owners just disable or delete one of the 2 tasks.

No. I thought the same but doing that will make your sitemap invalid. I think the second cron job is called by the first and is necessary to compress the sitemap files - it may be needed for other things too.

Anyone?

Sorry if I missed anyone else with this problem.

I seem to not be getting any sitemaps generated.
If you go to http://www.forumscitadel.com/sitemap_index.xml.gz you will see that it says "Sitemap file not found". Did I not change something I was supposed to? Would it be because I only have 1 user, 1 thread, and only a few replies?

I have generated my sitemap a few times, and it is not giving me any errors.

Is gzip installed and enabled on your server?

If so, check that your Scheduled Tasks shows TWO (2) cron jobs for this add-on. It seems that without both, the cron job doesn't complete properly.

Then double-check all your sitemap settings.

That's not where you'll find the sitemap, unless you changed it in the options. By default, you'll find it at http://www.forumscitadel.com/vbseo_s...p_index.xml.gz - that's why it's important to add the .htaccess lines.

I followed the directions, and it said to put the .htaccess in the forum root, which is where it is. I tried putting it elsewhere and it doesn't even find a file. The problem is that it is not generating a sitemap or something. Even when I pull up http://www.forumscitadel.com/vbseo_s...?sitemap=index directly it will not work.

Jedric,

check if you have sitemap files in your vbseo_sitemap/data/ folder. Do you get any errors if you execute Sitemap Generator manually?

Running vBulletin 3.6.7 PL1

I have (to the best of my knowledge) installed this per the instructions, uploading the upload folder and .htaccess file successfully, etc... I did not modify the sitemap files URL for example...

When I go to the URL specified for Google Sitemaps (http://forums.audioholics.com/forums...p_index.xml.gz) It's as if the plug-in never did anything after it was installed. I have re-run it and verified that I checked that vbSEO isn't installed... (first options item set to 'No')

Any ideas?

I figured it out. The data folder didn't have write access. Hawke, make sure that you create the data folder and CHMOD it 777. I had the exact same issue as you did.

thanks guys just installed i hope i did all the installs fine :)


Thanks once again for the hardwork

for something like 4 weeks, Google isn't indexing any new threads from my forum, i have something like 300,000 pages in google, somthing is wrong, Google is indexing any other page that i have on my forum like http://www.atraf-f.net/awards.php (was indexed a day after we added this page) but google bots are not indexing any new messages, what seems to be the problem ? here is an exmpale for a thread, http://www.atraf-f.net/showthread.php?t=225909 maybe something in the code is wrong and cousin bot to not index any new posts?
a link to my board http://www.atraf-f.net

please, try to help me ( p.s im using vbseo sitemap. ) , thanks.

Admin CP | *[vbseo]* | vbseo sitemap settings

Check the following settings:

Max URLs in Sitemap Files 50000
Specify the maximum number of URLs to be included in each sitemap file.
NOTE: Google allows no more than 50,000 URLs per file.

Include Show Post Pages Yes
Select "Yes" to include your show post pages in your sitemaps (showpost.php).

Include Member Profile Pages No
Select "Yes" to include your member profile pages in your sitemaps (member.php).

Include Archive Pages No
Select "Yes" to include your show archive pages in your sitemaps (archive/index.php).

Include Show Thread Pages Yes
Select "Yes" to include your show thread pages in your sitemaps (showthread.php).

Include Forum Display Pages Yes
Select "Yes" to include your forum display pages in your sitemaps (forumdisplay.php).

Include Poll Results Pages No
Select "Yes" to include your poll results pages in your sitemaps (poll.php).

Excluded Forums List ??????
Specify the forum ID for all forums that you do NOT wish to be included in your sitemaps.
NOTE: Separate multiple forum IDs with spaces.

Auto-Notify Google
Select "Yes" to automatically notify Google of your updated sitemap each time it is generated.
NOTE: Notification is sent via a HTTP request to the Google servers.