The issue I have found is that recently (within the past 2 months or so) Yahoo has begun using graylisting on all email received. I'm no expert on this stuff, but my understanding is that graylisting will bounce every email received once, and give the following reason:
Reason: Remote host said: 421 Message from (66.252.232.90) temporarily deferred - 4.16.50. Please refer to http://help.yahoo.com/help/us/mail/defer/defer-06.html
Basically what this does, is bounce the message back, and when the sending mail server receives it, it retries the message within 10 minutes. Upon the second receipt, yahoo's servers will allow it to pass through, then based upon the recipient's settings it will either end up in their inbox or junk folder. The premise of greylisting is that spammers rarely resend bounced spam email from their automated programs, so it does indeed significantly reduce the amount of spam received.
The problem which you're experiencing (i think all php applications using mail_send() experience this- not just vbulletin) is that php's mail_send() method of generating automatic emails currently has no method to know to resend the bounced emails from yahoo with the 421 error in the header.
I tried a workaround for this by changing the method by which email is sent to use SMTP instead of mail_send() which should fix the problem with yahoo bounces, and I get significantly less bounces from yahoo, but on occasion I still do receive 421 bounces from yahoo. I think that the main issue is to make sure your hosting company treats SMTP connections from your vbulletin scripts with the same permissions as from you when connecting through your ISP.