The use of a vb conditional is a much better idea, and to make just the case where 1 vote is singular, and handle the alignment issue, use this in your "pollresult" template:
Code:
<div class="votes">
<div class="pollbarwrapper">
<div class="pollbar{vb:raw option.graphicnumber}" style="width:{vb:raw option.percentraw}%"></div>
</div>
<span class="numvotes" style="float: right;<vb:if condition="$option['votes'] == 1">padding-right: 1em;</vb:if>">{vb:raw option.votes} Vote<vb:if condition="$option['votes'] != 1">s</vb:if></span>
<vb:comment><span class="percentvotes">{vb:raw option.percent}%</span></vb:comment>
</div>