[Zachery]

Originally Posted by calorie

If a bot isn't on a remove SID list, and then it crawls, it gets a SID, and when it comes back to respider, the bot has the SID in the respider request because it was assigned a SID initally. What is suggested may strip SIDs from bot requests that are new, but for respider requests, if the bot was initially assigned a SID, the bot remembers the SID, so the SID is in the respider request. In my situation, I didn't have msnbot on a remove SID list, so after some time, the bot was making quite a lot of respider requests for the same pages but using different SIDs. By the time I realized what was happening, I was out a good chunk of bandwidth so this is the type of situation where I think this hack is useful.

I dont think your logic is correct, i never had msnbot on the spiders list (untill after i noticed it on my forums) I checked its full location originaly and it did display a session id, and now after checking it yesterday(and it displaying as the MSNBot) it did not have a session id.