Email rejections due to DMARC

[pattycake]

I have done a LOT of researching on this. The problem is on the vbulletin side.
Look at this header from my site - it is the result of one member sending another a message. And yes, the actual emails have been changed.

Return-path: <ktmtalk_XXXX@ktmtalk.com>
Received: from localhost ([::1]:38730)
by server.ktmtalk.com with esmtp (Exim 4.82)
(envelope-from <ktmtalk_XXXX@ktmtalk.com>)
id 1WhKcS-0003zK-LQ
for copeXXXX@aol.com; Mon, 05 May 2014 11:15:52 -0400
Date: Mon, 05 May 2014 15:15:52 +0000
To: copeXXXX@aol.com
From: "ridenazi @ KTMTalk.com - The Absolute BEST KTM resource on the planet" <ktmXXX@yahoo.com>
Sender: ktmtalk_XXXX@ktmtalk.com
Message-ID: <20140505151548.74e58578d1e4@ktmtalk.com>
MIME-Version: 1.0
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8bit
X-Priority: 3
X-Mailer: vBulletin Mail via PHP
Subject: ktm300 4 sale

Ok, now look at the FROM. It shows:
"ridenazi @ KTMTalk.com - The Absolute BEST KTM resource on the planet" <ktmXXX@yahoo.com>

It's [SAYS] that it's FROM ktmXXX@yahoo.com when in reality, it is from MY server. vBulletin does the "from" this way so that the "receiver" of the message can hit reply and have it go to the original "sender".

So THAT is the problem... the header says it is from ktmXXX@yahoo.com when it's actually from my server (ktmtalk.com).

To confirm, on your next "bounced email", go look at the headers, specifically the FROM. If the entire "from" shows ANYTHING except your mail server, it will fail DMARC.

btw: AOL and gmail have both started using this "standard".

-pat-

[MGO_TOM]

I have been having the exact same problem for about a month now.

We have the "secure email" option enabled (if not, this wouldn't be a problem).

Originally, it was ONLY when a member having a yahoo.com email address attempted to send an email to another user. It doesn't matter what the receiver's email address is...what makes it bounce back is when the SENDER'S email address is a yahoo.com email address.

More info on why can be found here:
http://marketingland.com/email-sende...emailmarketing

For about the past week or so, I have started to see aol.com email addresses (SENDER) get rejected for the same reason.

I too am hoping for a good solution to this...if nothing else, simply reject the email attempt at the "send" stage so the member knows immediately their email didn't go though (I already have a bold red large text notification in the email interface, but again...many users seem to overlook it (somehow) and submit their email anyway...

[pattycake]

For about the past week or so, I have started to see aol.com email addresses (SENDER) get rejected for the same reason.

it's going to start happening more and more.... with more an more providers (aol, gmail, etc).

if the SENDER (ie, the FROM) contains ANYTHING except YOUR SERVER, it will fail DMARC.

I'll post a solution in a bit - I want to confirm a few other things first.

[Paul M]

We also have this issue on my vB3 forum, Ive been trying to figure out whats going on for 3 weeks (on and off) all our e-mails to Gmail are just getting bounced.

[makaiguy]

Originally Posted by pattycake

I have done a LOT of researching on this.

...

To confirm, on your next "bounced email", go look at the headers, specifically the FROM. If the entire "from" shows ANYTHING except your mail server, it will fail DMARC.

btw: AOL and gmail have both started using this "standard".

-pat-

Thanks, Pat, I've found the same. It may not exactly be DMARC causing the rejection, but an additional Sender-ID check being run by more and more servers -- not sure if this is part of DMARC or something separate, but no matter, it causes rejections either way.

I'm testing a hack of sendmessage.php right now that sends these user-generated messages with the default bbs address in the "From" field, and with the sending user's email address in an additional "Reply-to" header. This way the message should pass the Sender-ID test (the From address is on the sending server) but the recipient can still reply to the actual sender thanks to the Reply-to header.

Test messages I've sent via my test board to one of my other email addresses show the headers to be getting sent the way I want. Don't know yet if this really solves the problem when sending to gmail, yahoo, etc.

[pattycake]

Originally Posted by Paul M

We also have this issue on my vB3 forum, Ive been trying to figure out whats going on for 3 weeks (on and off) all our e-mails to Gmail are just getting bounced.

Paul... vb3, vb4, vb8000... it will happen on any vb version that puts anything other than your server as the from (ie, the sender).

The next time one bounces, go look at the header... you'll see the FROM issue.

--------------- Added [DATE]1399391922[/DATE] at [TIME]1399391922[/TIME] ---------------

Originally Posted by makaiguy

[snip]
I'm testing a hack of sendmessage.php right now that sends these user-generated messages with the default bbs address in the "From" field, and with the sending user's email address in an additional "Reply-to" header. This way the message should pass the Sender-ID test (the From address is on the sending server) but the recipient can still reply to the actual sender thanks to the Reply-to header.

Test messages I've sent via my test board to one of my other email addresses show the headers to be getting sent the way I want. Don't know yet if this really solves the problem when sending to gmail, yahoo, etc.

That should do the trick... it puts the REAL "from" (your server) in the "from" (ie, the "sender") and your "reply to" make a reply go to the correct email address (and not the forum's admin).

I need to go take a qwik check and find all occurances where VB changes this "from" and then I'll be able to post a "one size fits all" solution.

--------------- Added [DATE]1399392384[/DATE] at [TIME]1399392384[/TIME] ---------------

Click this link - http://www.dmarc.org/faq.html#g_13
It's a frigging dissertation on dmarc... just scroll down 40 or 50 pages to the section titled "Why are messages I send on behalf of visitors to my website being blocked?"

For those not wanting to read thru all that crap, it says:
Why are messages I send on behalf of visitors to my website being blocked?

This depends on how you are sending these messages. If you are simply taking the website visitor's email address and inserting it into the "From:" header of the message, and sending that message from your own servers, then you are impersonating the domain in their email address - in a way that is indistinguishable from spammers.

These practices may have worked previously - in many cases for decades - because before spam became a literally overwhelming problem, nobody checked. The most successful initial mechanisms to combat such spam were IP address-based blocklists, and so your site may have been allowed to continue because it did not appear on such a list. For the past decade, however email authentication has been introduced as a filtering mechanism, and is increasingly being used to detect and block such messages.

As a best practice, you should instead be using a domain you control in the address of the "From:" header, and use mechanisms like SPF, DKIM, and DMARC to show that this message is authorized to use your domain. In the example below, the site visitor's name is shown in the descriptive part of the "From:" header, and the "Reply-To:" header is set to the website visitor's address, but the actual address used in the"From:" header clearly indicates that your website is the origin of the message.
From: "John Doe via the Example Website" <service@website.example.com>
Reply-To: "John Doe" <john@firstmailboxprovider.com>
To: "Bob Smith" <bob@secondmailboxprovider.com>
Subject: "An article I thought you would find interesting"

[makaiguy]

Here's my hack to sendmessage.php. This is for vB3.8.7 - don't know if there have been syntax changes in later versions. The keys here are the 5th and 6th parameters passed to vbmail().

5th = the From email address. If not specified, or specified as a null (''), the default address for the bbs will be used.

6th = any additional headers you'd like to define. This is where we'll define the Reply-to header.

In my vB 3.8.7 vbmail() is called three times in sendmessage.php.

1. First time is for ContactUs mails. I don't currently have any trouble with these, probably because MY server isn't making DMARC/Sender-ID checks on incoming mail. Or maybe it's just that, even though it has the sender's email shown as From, in reality it is going from and to the same server (??).
   
   Find:
   PHP Code:

   vbmail($destemail, $subject, $message, false, $vbulletin->GPC['email'], '', $name); 

   Replace with:
   PHP Code:

   vbmail($destemail, $subject, $message, false, '', "Reply-To: {$vbulletin->GPC['email']}\n", $name); 

2. Second is for Send-A-Link-to-Friends mails. These now go thru okay because they show as coming from my admin email address. But would be MUCH better if they also included a Reply-to header so the recipient could reply to the real sender directly, instead of replies coming back to me.
   
   Find:
   PHP Code:

   vbmail($vbulletin->GPC['sendtoemail'], $vbulletin->GPC['emailsubject'], $message); 

   Replace with:
   PHP Code:

   vbmail($vbulletin->GPC['sendtoemail'], $vbulletin->GPC['emailsubject'], $message,'','',"Reply-To: {$vbulletin->userinfo['email']}\n"); 

3. Third is for Email-to-another-bbs-user mails. These are the ones currently giving me fits.
   
   Find:
   PHP Code:

   vbmail($userinfo['email'], fetch_censored_text($vbulletin->GPC['emailsubject']), $message , false, $vbulletin->userinfo['email'], '', $vbulletin->userinfo['username']); 

   Replace with:
   PHP Code:

   vbmail($userinfo['email'], fetch_censored_text($vbulletin->GPC['emailsubject']), $message , false, '', "Reply-To: {$vbulletin->userinfo['email']}\n", $vbulletin->userinfo['username']); 


[MGO_TOM]

Originally Posted by makaiguy

Third is for Email-to-another-bbs-user mails. These are the ones currently giving me fits.

Find:

PHP Code:

vbmail($userinfo['email'], fetch_censored_text($vbulletin->GPC['emailsubject']), $message , false, $vbulletin->userinfo['email'], '', $vbulletin->userinfo['username']); 


Replace with:

PHP Code:

vbmail($userinfo['email'], fetch_censored_text($vbulletin->GPC['emailsubject']), $message , false, '', "Reply-to:{$vbulletin->userinfo['email']}\n", $vbulletin->userinfo['username']); 


PERFECT! Thanks!
I tested both yahoo and aol email addresses (SENDER) and both went through fine now.
Replying to the resulting email does indeed direct it to the real sender.

By the way, my installation is 4.2.2 and there was no change in syntax on this hack.

[makaiguy]

Originally Posted by MGO_TOM

PERFECT! Thanks!
I tested both yahoo and aol email addresses (SENDER) and both went through fine now.
Replying to the resulting email does indeed direct it to the real sender.

Thanks for the feedback. It seems to be doing what it's supposed to here, too.

By the way, my installation is 4.2.2 and there was no change in syntax on this hack.

That's good to hear for when (if?) I finally get around to updating this beast.

I really hate to do this by hacking a php file, but I couldn't figure out how to do this via a plugin.

[Paul M]

Originally Posted by pattycake

Paul... vb3, vb4, vb8000... it will happen on any vb version that puts anything other than your server as the from (ie, the sender).

The next time one bounces, go look at the header... you'll see the FROM issue.

That wasnt the issue, as I said, it was rejecting all e-mails, the vast majority of them being subscription e-mails, which come from the sender domain.

Basically gmail has become very picky/sensitive to the slightest error. I went through our whole email set up and found a couple issues, one of which I suspect was probably the culprit, but Im not going to reverse them to confirm. Basically our IPv6 reverse dns was incorrect, but I dont think that was the problem, more likely was that the domain in our MX record (mail.mydomain.com) was pointed to a CNAME record (it CNAMED to mailserver.mydomain.com). Apparently this is not RFC compliant, so I changed them to use the direct server name which is defined as an A record.