I have not tried vbulletin 5 and I won't. I suggest though that you repair & optimize the tables via phpmyadmin on both your vb3 db & your current vb5 db.
I also suggest that you make a post here as it might help to improve your servers performance.
http://www.vbulletin.com/forum/forum...-configuration
Actually I would suggest just regular installing of vb5 and then use impex to import the content.
You may find the download for it in your customer center on vb.com and support for it here.
http://www.vbulletin.com/forum/forum...-import-system