Yes, the site is turned off through the vBulletin options, then I went into the Usergroup/Userid Access To Closed Site Settings to only allow 2 additional groups, super moderators and 1 group that I've created for testing forum features. If you log into a user that's not part of any of those groups, you get the message that the forum is closed, when logged into a user that is part of the group, the forum displays as normal with the message at top saying that the forum is closed. Signed out of your account or open another browser that you're not currently signed into and go to the site, you see the full site with all it's glories (with the message about the forums being closed). I turn off the plug in and the site goes back to showing up as being closed for everyone (expect admins). And yes I've tried disabling all my other plugins and played with the settings to no avail.

Hmmm, I'll have to look into it.