Well, the editor has a button in the upper left corner that sort of looks like "A/A", that toggles wysiwyg mode, and I think you get either html or bbcode depending on how that is set. I don't know a lot about the editor so I don't know how to tell you to force a certain mode, but looking at newreply.php, it does this (after "cleaning" the wysiwyg parameter):
Code:
if ($vbulletin->GPC['wysiwyg'])
{
require_once(DIR . '/includes/class_wysiwygparser.php');
$html_parser = new vB_WysiwygHtmlParser($vbulletin);
$newpost['message'] = $html_parser->parse_wysiwyg_html_to_bbcode($vbulletin->GPC['message'], $foruminfo['allowhtml']);
}
else
{
$newpost['message'] = $vbulletin->GPC['message'];
}