Major Additions - Email Integration (New threads/replies by email)

This mod is based off of the Mail Reply modification by Colin F to which I have obtained permission to rewrite and release.

This modification allows you to mimic email lists such as yahoo groups through your forums!

After installing this modification you will have new settings in your forum manager where for each forum you can enable this modification and setup a separate email address to use for each forum that you have this enabled for.

Just like how there is a separate email address for each yahoo group.

After doing so, members can subscribe to each forum that this mod is enabled for to receive notifications for new threads and replies.

Any posts in that forum, will be sent to them via email. (Example email attached below). The users can then reply to that email (which will then be processed and posted to the forums through the cron job that runs every 10 minutes), or they can send a new email to the email address and a brand new thread will be created!

Essentially mimicing an email list! Suddenly those users that hate forums, can still be dragged into the conversations and help keep the forum's activity level up!


BUGS!
These are the known bugs to date:

• Confirmed bug that email processing does not handle Japanese characters.

Unsupported Items
These are the items that are not currently supported:

• Some custom bbcode modifications will not display like they do on the forums in the html emails. Like glow, and the table mod.

TO UPGRADE

• Please see the file in the zip for upgrading!! If your upgrading from version 2.3.X you will need to reverse some file edits that are no longer nessecary!

NEW INSTALL

• See instructions in the zip file.

VERSION HISTORY!
(See the file in the zip for a complete history!)

• 2.6 Release
  • Bug Fixes
    • Joining an unmoderated public group will now auto subscribe you per the usergroup settings.
    • NO CONFLICTS with Instant Thread Subscription! See details in below post.
  • New Stuff
    • User Option Allow Auto Subscribe - This is an admin allowable, user option that let's the user choose to not be auto subscribed.
    • User Option not receive own posts - A new user option that lets them select to NOT receive their own posts via email.
    • Editted the phrase for the Auto Subscription script to tell people what to do if there are no listings.
    • Added error checks when saving a usergroup for if auto subscriptioin is entered with a forumid for a forum that does not have email integration enabled and also checks that the usergroup you are saing is entered in the forum manager as allowed to use email integration.
• 2.6.1 Release (BETA VERSION)
  • Hot Fix
    • It appears I was unseting a variable too soon. I've corrected this and it has resolved the issue with multiple posted replies on my test server.

Most of these will be slow to make it in til I know I've got the code to handle just about every type of email we could receive But in anycase, here are the ideas for future versions!

• End User Option to subscribe to just receive new thread emails, or receive all new thread & reply emails (current version does all new threads & replies).

Thanks to RedTyger, Bob Denny, Ed Kohwley and Chris McKeever for their additions and assistance with portions of this code.

Huge thanks also goes to the many individuals that have helped test various incarnations of this mod.

MAD PROPS TO: cgmckeever for a great quoting regex for the old format, and all the mods at www.4winmobile.com for helping test version 2.4 prior to release!!

You guys rock!

Please feel free to donate to my continued work on this modification!! It let's me spend more time on it!
Make a Donation!

Version 2.6 will work with both 3.6.8 and 3.7!

The #.X versions are the solid tested versions.

The #.X.X versions are the beta versions. New features and bug fixes will be worked in there and then when I have a solid version it'll be released as the next #.X

Thank you!!

~ Cyricx

[ChurchMedia]

I'm really excited about releasing this to my community. They will LOVE it! So, great work!

I'm having a problem with BB Code not being formatted correctly and broken links (see attached screenshots). Is there something I'm doing wrong? I'm using MS Outlook and VB 3.8.

Thanks!

[fxs158]

Originally Posted by Cyricx

Like so

// this grabs an overview of all the messages in the mailbox and creates an array we can read
  $letters = imap_fetch_overview($mailbox,"1:$amountmessages",0);
 
//then later in the cron file it checks the array we made above and determines the sender and host to determine the email address.
   $fromaddress = $letter->sender[0]->mailbox ."@".$letter->sender[0]->host;
 
// then later it does this query to find a user that has that email address
   $userinfo = $vbulletin->db->query_first("
    SELECT user.*, usertextfield.*
    FROM " . TABLE_PREFIX . "user AS user
    LEFT JOIN " . TABLE_PREFIX . "usertextfield AS usertextfield ON (usertextfield.userid = user.userid)
    WHERE email = '".addslashes(htmlspecialchars($fromaddress))."'
   ");
 
// finally in the cron file it does this which is where if there is not a user found in the above query, it sends the person an error message.
 
   // If Email Address belongs to a user
   if (!$userid)
   {
    $userinfo['username'] = "Unregistered User";
      $userinfo['languageid'] = 0;
    eval(fetch_email_phrases('ei_error_nouser', $userinfo['languageid']));
    vbmail($fromaddress, $subject, $message);
    imap_delete($mailbox,$msgno);
    continue;
   }

You guys are killin me here

I think I've mentioned this a few times in this thread how it determines the user that is posting

Thanks for the info, I am no coder but if i am understanding correctly this is checking that the email comes from the correct address and or domain and then cross reference the email with one of my database.

I am still not understanding how the script would prevent someone with an email address on same domain and a fake header, from posting as someone else.

I would think that a better approach would be to issue a unique hash to each user on the forum and when the email is sent to them this hash must be somewhere in the email so that when users reply the software will authenthicate the user. Or something along the lines.


Great job by the way on the scrip, I have not been this excited about a hack in a long time!!!!!

[cynthetiq]

Originally Posted by Cyricx

O awesome.

Those are easy to code. I'll add those to the error codes for the next version I fire out

Thanks!!!!

Well, if you added a section within the forum setup where you have your email configurations to also include some additional regex items where one can put them in manually it would be a great way to stop spam.

I looked in depth at the form hack and the WoW hack based on form hack and they both require turning off the CAPTCHA image verification to allow guest posting

Here's some screenies of what the m2f backend settings and config are like and it really keeps the system from getting spammed.

[Cyricx]

Originally Posted by fxs158

I would think that a better approach would be to issue a unique hash to each user on the forum and when the email is sent to them this hash must be somewhere in the email so that when users reply the software will authenthicate the user. Or something along the lines.


Great job by the way on the scrip, I have not been this excited about a hack in a long time!!!!!

The massive downside to that is that they would have to remember and include that hash for when they wanted to create new threads. Significantly killing the user friendliness of this modification.

[Cyricx]

Originally Posted by cynthetiq

Well, if you added a section within the forum setup where you have your email configurations to also include some additional regex items where one can put them in manually it would be a great way to stop spam.

I looked in depth at the form hack and the WoW hack based on form hack and they both require turning off the CAPTCHA image verification to allow guest posting

Here's some screenies of what the m2f backend settings and config are like and it really keeps the system from getting spammed.

Thanks!! I'll review these and release a version that checks for these things like autoreply and such by early next week. (Have meetings all this weekend heh).

I estimate a release by tuesday

Also, yes, it'll be in the options

It'll have to be to allow for multiple languages heh

[fxs158]

I am also a little concerned about spam, if a spam boot gets hold of the email. They will start spamming it, the script will receive each email and basically return an error to each of the bogus email addresses sent by the spam software, this will cause my board to become a spamming hub as I will be replying to bogus addresses and in essence become a spammer myself. Maybe setup the software to delete any emails that are not in the db, or simply configure a forum to dump posts from unknown emails?

[fxs158]

Originally Posted by Cyricx

The massive downside to that is that they would have to remember and include that hash for when they wanted to create new threads. Significantly killing the user friendliness of this modification.

what if the hash was included on the subject line of the email on the way out of the server? so when they reply to the email then the hash for that user is already inserted for them?

I will do some testing on my board, but if there is a way for someone to bypass the filter and post as another user, then we are approaching this from the wrong angle. Security must prevail over functionality and ease of use. I love the idea of this mod and it is working flawlessly on my site so far!!!

[Cyricx]

Originally Posted by fxs158

I am also a little concerned about spam, if a spam boot gets hold of the email. They will start spamming it, the script will receive each email and basically return an error to each of the bogus email addresses sent by the spam software, this will cause my board to become a spamming hub as I will be replying to bogus addresses and in essence become a spammer myself. Maybe setup the software to delete any emails that are not in the db, or simply configure a forum to dump posts from unknown emails?

The code I posted above has this line

    imap_delete($mailbox,$msgno);

That flags the message to be deleted when the script is complete. So any message that errors out gets deleted.

As mentioned, the scripts next creation will handle any "returned mail" errors that you would get from an invalid email address

I really really appreciate your input!! Your bringing up some great points!

Originally Posted by fxs158

what if the hash was included on the subject line of the email on the way out of the server? so when they reply to the email then the hash for that user is already inserted for them?

That would work great for replies... but not for new threads

Or forwarded messages to create new threads would also not work for that system

I'm not opposed to a more secure system to receive posts... just wanna make sure it's feasible

The previous "Mail Reply" system by Colin F, created a new user profile field that people could put in a password. It hashed that onto the subject line of posts sent to users that when they replied validated the reply.

He then had users have to put something like

-pmypassword

at the end of subject lines of new threads if I recall right.

That seems infeasible, and definatley not user friendly enough to compete with yahoo groups.

I'm curious... will false headers from the same domain work to post to yahoo groups? has anyone tried?

[fxs158]

Well how about this, regular threads would get a random password attached to the subject line. New threads one must add a user password to subject line. If I have to type a whole email to make a post, I do not see why functionality would be sacrificed due to requiring that I type a password on the subject line right after my topic.

It would not be fully automated, but it would definitely beat, having a competitor spam the board with an email bomber. Can you imagine the work to clean that mess? Allot of sites are ran on shared accounts, all I would have to do is get an account on the shared server and then fake the headers and that could spell trouble for an unsuspected site.

I am not trying to be difficult, competition is though as it is, just do not want to give other sites a way to mess with my forum.

[Cyricx]

I'm mixed, I'd rather try to avoid having users that are sending threads to the forum have to do alot extra, or remember a password.

Heck, I have users now that have to reset their passwords weekly because they can't remember them

I'd love to hear from others on if they think that this would be nessecary.

Or if anyone else has any alternate ideas

Secondly, if you have a "competitor" on a shared server spamming your board... I'd really wonder why your host isn't doing something about that, as the end result would be them crippling their servers

I understand your not trying to be difficult.

I just don't find the system your suggesting to be feasible, and would like to find an alternative