Major Additions - Email Integration (New threads/replies by email)

This mod is based off of the Mail Reply modification by Colin F to which I have obtained permission to rewrite and release.

This modification allows you to mimic email lists such as yahoo groups through your forums!

After installing this modification you will have new settings in your forum manager where for each forum you can enable this modification and setup a separate email address to use for each forum that you have this enabled for.

Just like how there is a separate email address for each yahoo group.

After doing so, members can subscribe to each forum that this mod is enabled for to receive notifications for new threads and replies.

Any posts in that forum, will be sent to them via email. (Example email attached below). The users can then reply to that email (which will then be processed and posted to the forums through the cron job that runs every 10 minutes), or they can send a new email to the email address and a brand new thread will be created!

Essentially mimicing an email list! Suddenly those users that hate forums, can still be dragged into the conversations and help keep the forum's activity level up!


BUGS!
These are the known bugs to date:

• Confirmed bug that email processing does not handle Japanese characters.

Unsupported Items
These are the items that are not currently supported:

• Some custom bbcode modifications will not display like they do on the forums in the html emails. Like glow, and the table mod.

TO UPGRADE

• Please see the file in the zip for upgrading!! If your upgrading from version 2.3.X you will need to reverse some file edits that are no longer nessecary!

NEW INSTALL

• See instructions in the zip file.

VERSION HISTORY!
(See the file in the zip for a complete history!)

• 2.6 Release
  • Bug Fixes
    • Joining an unmoderated public group will now auto subscribe you per the usergroup settings.
    • NO CONFLICTS with Instant Thread Subscription! See details in below post.
  • New Stuff
    • User Option Allow Auto Subscribe - This is an admin allowable, user option that let's the user choose to not be auto subscribed.
    • User Option not receive own posts - A new user option that lets them select to NOT receive their own posts via email.
    • Editted the phrase for the Auto Subscription script to tell people what to do if there are no listings.
    • Added error checks when saving a usergroup for if auto subscriptioin is entered with a forumid for a forum that does not have email integration enabled and also checks that the usergroup you are saing is entered in the forum manager as allowed to use email integration.
• 2.6.1 Release (BETA VERSION)
  • Hot Fix
    • It appears I was unseting a variable too soon. I've corrected this and it has resolved the issue with multiple posted replies on my test server.

Most of these will be slow to make it in til I know I've got the code to handle just about every type of email we could receive But in anycase, here are the ideas for future versions!

• End User Option to subscribe to just receive new thread emails, or receive all new thread & reply emails (current version does all new threads & replies).

Thanks to RedTyger, Bob Denny, Ed Kohwley and Chris McKeever for their additions and assistance with portions of this code.

Huge thanks also goes to the many individuals that have helped test various incarnations of this mod.

MAD PROPS TO: cgmckeever for a great quoting regex for the old format, and all the mods at www.4winmobile.com for helping test version 2.4 prior to release!!

You guys rock!

Please feel free to donate to my continued work on this modification!! It let's me spend more time on it!
Make a Donation!

Version 2.6 will work with both 3.6.8 and 3.7!

The #.X versions are the solid tested versions.

The #.X.X versions are the beta versions. New features and bug fixes will be worked in there and then when I have a solid version it'll be released as the next #.X

Thank you!!

~ Cyricx

[OS Master]

Can you use this with Google Apps?

[Piggo]

v2.1 looking good, Cyricx.

I did find a little error: when in USERGROUP MANAGER, and "automatically add Instant Notification by Email subscriptions" is set to YES, but no FORUM IDs are selected, the *[ei_auto_sub]* link gives a nasty syntax error.

[dc3dreamer]

V2.1 - This is someting I looked at doing last year and forgot it "too much time required". I want to thank you very much for doing this! I'll definitely donate some $$$ to back that up. I do have a few ideas w.r.t threading replies (using the References: SMTP header and the post ID for example), but that's for later!

Right now I have only one significant bug. HTML posts coming from Thunderbird 2.0.0.6 (latest) start with {font=Arial,Helvetica,sans-serif} (but with [] not {}) so somehow the bbCode isn't getting translated.

Anyway, thanks so much again!

[dc3dreamer]

PS: Don't bust your butt on anti-spam. This would best be handled by the SMTP mail server that's receiving the incoming posts destined for the forums. You'll end up re-inventing that huge wheel, and the SMTP servers out there have all sorts of fancy and complex spam traps already. To them, the forums' email addresses are just user accounts to be protected just like any other "user".

I'd much rather see you work on the threading and HTML/bbCode functionality I'm going to see if I can work out threading here, but my initial look is a bit overwhelming given that I'm not a PHP/VB expert. Wow.

I just donated $50 - money well spent IMO!

[dc3dreamer]

Woo Hoo! I can now send out messages from newly created threads and replies to same (created in VB), and the resulting email messages thread properly in the email client. I probably have missed some corner case, but so far so good! I produce the Message-ID: header and (for replies) the References: header with IDs of the form threadid.postid@forum.host.com. Minimal mods to Notify_Email_for_New_Posts (replies) and Thread_Notification_Email (new threads) hooks. A lot of "just in time learning"!!

I have to say, your code is so well structured and so complete. I have a high level of confidence in your mod working reliably and handling all of the cases it needs to. Good work!

Now to see if I can get emailintegration.php to work with the new Message-Id/References headers to thread incoming email posts correctly. If I get this working, there'll no longer need to be all that stuff in the [] in the subject line, as the thread ID is part of the new Message-Id: header, and the parent to which to thread the incoming post will be in the References: header if it's not being threaded directly to the initial post of the thread.

I'm having fun at least.

[dc3dreamer]

Got it! Threading works in both directions, with the emailer's and the forum's thread trees being identical, regardless of where a message comes from. The "-t-nnnn" is now gone from the subject line (change to the ei_notify_xxx phrases).

One thing I ran into when posting HTML from T-Bird with Helvetica, Arial set as the default font, I kept getting stray [font=Helvetica, Arial, sans-serif] bbcodes in the messages posted in the forum. I found the problem, the "hail mary" cleanup for [font=xxx] needs a space in the character class of its regex. Once I put that in there, the problem disappeared.

Man that message "purty"-fication code is crazy. Well you must have gone crazy getting it to work with the different emailers out there. I don't really even understand some of it :-)

Tomorrow I'll do more testing and I have a few TODOs to pick up, then I'll roll it up and send it along. I need your email address, so I sent you a PM with mine so you can send me a message and I'll have it.

Thanks again!

[dc3dreamer]

With regard to spamming - check out Domain Keys and DKIM (forget Microsoft's competing proposal, it's dying/dead). Here's an easy to understand explanation

http://antispam.yahoo.com/domainkeys

The 'key' is, as I was talking about earlier, the mail server manages this, rejecting messages when the From: address is forged. One could impose the requirement that all incoming posts to their VB forums be DKIM-authenticated. This requires that the sender's mailserver support DKIM as well as the server hosting the VB forum mailboxes. I don't see a practical way of handling forged From addresses winthin individual clients (short of signing and verifing signatures, requiring everyone to have certs, etc.). Think of DKIM as a sort of driver-client architecture, where the common problem of address authentication is solved at the common points, the mail servers.

Don't burn time trying to do something in you stuff. Yahoo! uses DK in Yahoo! groups, so expect it to be widely available.

[dc3dreamer]

There's an unclosed <a> tag in the ei_notify_xxx phrases for the Replies to this topic link.

Also, in case you missed it, T-bird can produce a font list with spaces after the commas, so you need to add a space to the character class in the font tag hail mary cleanup.

[Cyricx]

Originally Posted by OS Master

Can you use this with Google Apps?

I'm uncertain, is this another modification offered here? if so, can you please provide a link?

Originally Posted by Piggo

v2.1 looking good, Cyricx.

I did find a little error: when in USERGROUP MANAGER, and "automatically add Instant Notification by Email subscriptions" is set to YES, but no FORUM IDs are selected, the *[ei_auto_sub]* link gives a nasty syntax error.

Noted, I'll program an error message that will stop the save process is that is set to use and forum ids is blank

Originally Posted by dc3dreamer

Got it! Threading works in both directions, with the emailer's and the forum's thread trees being identical, regardless of where a message comes from. The "-t-nnnn" is now gone from the subject line (change to the ei_notify_xxx phrases).

One thing I ran into when posting HTML from T-Bird with Helvetica, Arial set as the default font, I kept getting stray [font=Helvetica, Arial, sans-serif] bbcodes in the messages posted in the forum. I found the problem, the "hail mary" cleanup for [font=xxx] needs a space in the character class of its regex. Once I put that in there, the problem disappeared.

Man that message "purty"-fication code is crazy. Well you must have gone crazy getting it to work with the different emailers out there. I don't really even understand some of it :-)

Tomorrow I'll do more testing and I have a few TODOs to pick up, then I'll roll it up and send it along. I need your email address, so I sent you a PM with mine so you can send me a message and I'll have it.

Thanks again!

Yes, I went NUTZ!! I am going to slowly work it into one cleanup if I can. Your right about the space, that's fixed in my upcoming version as well as I'm doing some code to strip the size bbcode due to there being some confusion for the wysig converter making open size tags without closed tags.

Originally Posted by dc3dreamer

There's an unclosed <a> tag in the ei_notify_xxx phrases for the Replies to this topic link.

I'll check this out now! That may be what was causing the damnable occasional extra ! in some email clients.

[AyeCapn]

just curious how the ahck will react to hard and soft email bounces. Will it post the error messages in the forum?