It seems to me like that would be incredibly annoying 95% of the time - If I understand correctly, you want the "__ unread private messages" link to open up in a new window. If this is the case, every time you click that link in the forums, a new window opens up. So if you're just browsing around the forums and need to read a PM, it opens in a separate window, regardless of whether or not you're writing a post.
Seems to me like you should just....not click on any links while you are working on a post
If you absolutely must read a PM while you're typing something, then hold down your shift key and click on the PM link...or just open a new window manually and navigate to the PM.