Questions regarding default vBulletin 4 should be asked on the vBulletin.com forums. See this sticky -
Installation Issues / Questions or bug reports regarding default vBulletin 4.0
However, I don't think the manual has been updated yet, so I don't think you are going to get detailed instructions on how to do it in vb4. there is the vb3 documentation regarding google adsense in the manual. Most likely it hasn't changed too much. (But I have no idea since I don't use it.)
As for Open X, vbulletin will have no documentation written on how to use it. You will have to check on the Open X site to see if they have documentation.