robots.txt for 3.8.2 - Any Ideas?

[vbplusme]

I can exclude it from the sitemap for sure but I thought it was pretty strange to be referencing archives from 1970 through current. The links looks like:

hxxp://www.mysite.com/blog.php?do=list&m=12&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=12&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=11&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=11&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=10&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=10&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=9&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=9&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=8&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=8&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=7&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=7&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=6&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=6&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=5&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=5&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=4&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=4&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=3&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=3&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=2&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=2&y=1970
hxxp://www.mysite.com/blog.php?do=list&m=1&y=1970
hxxp://www.mysite.com/blog.php?u=1&m=1&y=1970

The sitemap software is finding this for every year to current?

[BSMedia]

Those are sorting URL's that google finds and goes crazy on. Block off blog.php and call it a day.

It will also do this for the calendar, usually on newer vBulletin sites for some reason if you use the site:http://sitename.com you'll see 500 pages of calender sorting URL's.

Its fine to have the warning messages in GWT that tell you access is restricted to those URL's, they provide no value what so ever to your sites rankings, and should be blocked off as with most sorting URL's since it will just be duplicated content from else where.

[vbplusme]

Thanks very much for the reply. I appreciate it. It fits with what I suspected on the duplicate content issues I am trying to solve. I will add blog.php to my robots.txt file and exclude it from my sitemap.

thanks again.

--------------- Added [DATE]1240771303[/DATE] at [TIME]1240771303[/TIME] ---------------

Thanks very much for the reply. I appreciate it. It fits with what I suspected on the duplicate content issues I am trying to solve. I will add blog.php to my robots.txt file and exclude it from my sitemap.

thanks again.

--------------- Added [DATE]1240797229[/DATE] at [TIME]1240797229[/TIME] ---------------

Originally Posted by BSMedia

Those are sorting URL's that google finds and goes crazy on. Block off blog.php and call it a day.

It will also do this for the calendar, usually on newer vBulletin sites for some reason if you use the site:http://sitename.com you'll see 500 pages of calender sorting URL's.

Its fine to have the warning messages in GWT that tell you access is restricted to those URL's, they provide no value what so ever to your sites rankings, and should be blocked off as with most sorting URL's since it will just be duplicated content from else where.

Do Tags do the same thing, i.e. provide duplicate content with no value to the PR? I see a boat load of them in the sitemap too.

TIA for a reply.