Major Additions - Email Integration (New threads/replies by email)

This mod is based off of the Mail Reply modification by Colin F to which I have obtained permission to rewrite and release.

This modification allows you to mimic email lists such as yahoo groups through your forums!

After installing this modification you will have new settings in your forum manager where for each forum you can enable this modification and setup a separate email address to use for each forum that you have this enabled for.

Just like how there is a separate email address for each yahoo group.

After doing so, members can subscribe to each forum that this mod is enabled for to receive notifications for new threads and replies.

Any posts in that forum, will be sent to them via email. (Example email attached below). The users can then reply to that email (which will then be processed and posted to the forums through the cron job that runs every 10 minutes), or they can send a new email to the email address and a brand new thread will be created!

Essentially mimicing an email list! Suddenly those users that hate forums, can still be dragged into the conversations and help keep the forum's activity level up!


BUGS!
These are the known bugs to date:

• Confirmed bug that email processing does not handle Japanese characters.

Unsupported Items
These are the items that are not currently supported:

• Some custom bbcode modifications will not display like they do on the forums in the html emails. Like glow, and the table mod.

TO UPGRADE

• Please see the file in the zip for upgrading!! If your upgrading from version 2.3.X you will need to reverse some file edits that are no longer nessecary!

NEW INSTALL

• See instructions in the zip file.

VERSION HISTORY!
(See the file in the zip for a complete history!)

• 2.6 Release
  • Bug Fixes
    • Joining an unmoderated public group will now auto subscribe you per the usergroup settings.
    • NO CONFLICTS with Instant Thread Subscription! See details in below post.
  • New Stuff
    • User Option Allow Auto Subscribe - This is an admin allowable, user option that let's the user choose to not be auto subscribed.
    • User Option not receive own posts - A new user option that lets them select to NOT receive their own posts via email.
    • Editted the phrase for the Auto Subscription script to tell people what to do if there are no listings.
    • Added error checks when saving a usergroup for if auto subscriptioin is entered with a forumid for a forum that does not have email integration enabled and also checks that the usergroup you are saing is entered in the forum manager as allowed to use email integration.
• 2.6.1 Release (BETA VERSION)
  • Hot Fix
    • It appears I was unseting a variable too soon. I've corrected this and it has resolved the issue with multiple posted replies on my test server.

Most of these will be slow to make it in til I know I've got the code to handle just about every type of email we could receive But in anycase, here are the ideas for future versions!

• End User Option to subscribe to just receive new thread emails, or receive all new thread & reply emails (current version does all new threads & replies).

Thanks to RedTyger, Bob Denny, Ed Kohwley and Chris McKeever for their additions and assistance with portions of this code.

Huge thanks also goes to the many individuals that have helped test various incarnations of this mod.

MAD PROPS TO: cgmckeever for a great quoting regex for the old format, and all the mods at www.4winmobile.com for helping test version 2.4 prior to release!!

You guys rock!

Please feel free to donate to my continued work on this modification!! It let's me spend more time on it!
Make a Donation!

Version 2.6 will work with both 3.6.8 and 3.7!

The #.X versions are the solid tested versions.

The #.X.X versions are the beta versions. New features and bug fixes will be worked in there and then when I have a solid version it'll be released as the next #.X

Thank you!!

~ Cyricx

[Cyricx]

Yep!! Completely agree. I had to disable that due to the queries not agreeing well, but I've learned alot more about building, constructing and fiddling with arrays. So it'll definately be on its way

I love this tech though.. can you imagine how this could be adapted even further??

Reply to PMs via Email :P

Possibilities are endless!!

Anyway, I've sent the 2.x version to a few beta testers, after they beat it up and fire a few reports back to me, I'll release it here. Should be just a day or two til the public release of 2.x

Which will have:

Auto subscriptions (with links to update subs)
phrased error messages
admin options to enable/disable each of the links in the emails

[high]* Cyricx dances a jig![/high]

[tekguru]

And a report from me now in

BTW reply to PMs would be awesome- I was going to suggest that!

[rbgrn]

I installed per the installation instructions and am having no luck. I didn't do anything regarding attachment restrictions (left all default) and commented out the require in php as directed (I don't have whatever thing installed that it wanted)

I'm trying to pull via pop/w SSL (official cert) from pop.gmail.com
My login and password are definitely correct, the login also includes @gmail.com

I set up a forum and enabled email integration
I set it as only my admin group for now can post or reply by email

When I run the job manually, it just sits forever and the page never finishes refreshing.

My questions:
Is there any kind of log or debug info here? Nothing shows up off the cron job on the info column when it runs.

What user will the emails come in as posted by??

Does the email account need to coordinate with some account on the forum?

Has anyone tested this with gmail?

Thanks!

[rbgrn]

Another question:

Re: changes to "Instant Email Notification"

Does that apply globally now or is it only for forums that have Email Integration enabled?

I certainly don't want that behavior to change anywhere. I just want to pull emails into posts.

[rbgrn]

Ok got it to run, gmail requires the self-signed cert option which really means use port 995, which should be the standard port. This may require some renaming

anyway, it runs now, but no mail comes in even when there's new mail in the box. :/

[sdsvtdriver]

This mod looks great!

I do have a request, as this mod is the closest I've seen to do what I want.

We all know this mod allows a user to be on the forum and subscribe to a forum(or thread in the near future), then receive emails containing posts made since the subscription, and then reply.

What I'd like to see is for a user to be able to send email to their username.apasswordtheysetintheirCP@myvbforum.com and have a new post created in a predetermined forum.

This would be useful for people who like to take pictures and send them off to their blog, etc, however this would allow them to do so on the forums on the go.

Somewhat like textamerica. I have a plug-in from them on my blog. I sent my pic to myemail.mypassword@textamerica.com, they process it and it magically appears on my blog.

Would be useful for varying ideas, such as my car forum. If people see a cool car, they can shoot off the pic to their shorcut in their mobile phone's address book and a post is created with the MMS text and photo, instead of going home, emailing photo to yourself, putting it on photobucket, creating thread, etc. etc.

95% of the functionality (and hard part) seems to already exist in the code. A few mods such as the mobile post password field in the UCP, way to designate forum for mail with a username.password format @myvbforum.com, as opposed to their email address on file, etc.

Possible future enhancement?

[Tralala]

Why wouldn't they just send the pic to the mailboxes already used by this mod?

ie: general@myvbforum.com

Since the sending email address is cross-checked against the member's list, it's more or less just as secure. They can take pic, email it from the phone, and it gets posted. Am I missing something? What more does your request accomplish?

(In other words, this mod isn't just for replies, it's also for new threads too.)

[Cyricx]

Originally Posted by rbgrn

What user will the emails come in as posted by??

Has anyone tested this with gmail?

Thanks!

It'll match the sent email to the email address for the user on the board. If they match, rock on, it'll post as that user. If it doesn't they get an error message.

Tested with gmail incoming emails, but not with using a gmail account as the catch.

Originally Posted by rbgrn

Another question:

Re: changes to "Instant Email Notification"

Does that apply globally now or is it only for forums that have Email Integration enabled?

I certainly don't want that behavior to change anywhere. I just want to pull emails into posts.

It only works for those forums that have it enabled. If you don't enable it for any forums, you will notice no changes at all

For the forum you enable, it will have the new subscription option available.

Originally Posted by rbgrn

Ok got it to run, gmail requires the self-signed cert option which really means use port 995, which should be the standard port. This may require some renaming

anyway, it runs now, but no mail comes in even when there's new mail in the box. :/

This option is covered in the connection type dropdown.

Pop3 with self-signed certificate or IMAP with self-signed certificate both use port 995.

Please try all the connection types

Here is the break down of ports that are used.

 // ##### Determining Connection Type ######
 switch ($foruminfo['ei_connectiontype'])
  {
   case 0:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":110/pop3/notls}INBOX";
    break;
   case 1:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":143/novalidate-cert}INBOX";
    break;
   case 2:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":993/pop3/ssl}INBOX";
    break;
   case 3:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":995/pop3/ssl/novalidate-cert}INBOX";
    break;
   case 4:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":993/imap/ssl}INBOX";
    break;
   case 5:
    $connectmailserver = "{" . $foruminfo[ei_hostname] . ":995/imap/ssl/novalidate-cert}INBOX";
    break;
  }

Originally Posted by sdsvtdriver

What I'd like to see is for a user to be able to send email to their username.apasswordtheysetintheirCP@myvbforum.com and have a new post created in a predetermined forum.

This would be useful for people who like to take pictures and send them off to their blog, etc, however this would allow them to do so on the forums on the go.

This is possible with the current code.

If a user is in a usergroup that is set in this setting in the forum manager -

Please enter the usergroup IDs that can post and reply by email.

They can send an email to the email address for the site and it will post. They do NOT have to be subscribed to the forum in order to start a new thread from email

[rbgrn]

Cyricx - I've got everything working with a few modifications, here's my general experience feedback and breakdown:

1) The feature I need is simply to pull mailing-list emails into threads/posts
1a) I modified the code to figure out which thread an email should go into using the subject
1b) I commented out error mails or response mails as I don't want interaction

2) There is no cron logging working for this
2a) I tried adding some in the cron job php and changed the phrase to have {1} in it so that it'd show up, but for whatever reason (I'm new to vb) it does not still
2b) I recommend adding some informational and debugging logging as it's frustrating when things aren't working and you can't figure out why
2c) My recommended categories - Email Received, Email Sent, Error Email Sent, Other Errors

3) Configuration
3a) It'd be ideal to have configuration as a separate menu item rather than hang off the forum itself. Multiple select of forums would be excellent on a single configuration
3b) For my set up, I'd like to have an override user that ALL emails get posted as
3c) Also I'd like to pull email from one box and depending on who it's from, get delegated into different forums (for me that translates to multiple mailing list support off 1 account). Another solution would be supporting a box other than INBOX
3d) I'd like an option to globally shut off all outbound email, as I don't use that.
3e) I don't want any other behavior changed within my board

4) Security
4a) Email inherently is insecure to use for this as I can forge my From address and post as someone I'm not. There's not a lot you can do about that, unfortunately. Just a concern.

5) Code
5a) Your code is one monolithic block. It'll be easier for you to add features if you refactor it a bit, breaking it into logical blocks using methods or classes if it gets even bigger.
5b) Consider adding some debugging ability. It'd be nice for us hackers.

Of course it's easy for someone to sit back and criticize another's work, which is not what I'm trying to do. I really appreciate your time and think this is very valuable. I have different uses for it than what you intended initially and I'd consider polishing this off doing all of my above things and forking it off into a different project maybe called vBulletin Mail Pull or something of that nature. I may consider that if I get some more time to work on this in the future. For now, I'll create multiple mail boxes and users to go into the different forums for different mailing lists.

Thanks again for all your work.

Here's my modification to group threads together

            // find thread with matching title 'Post'
            // make reply if match
            $baseTitle = preg_replace("/re: /", "", $title);
            $baseTitle = preg_replace("/Re: /", "", $baseTitle);
            $baseTitle = preg_replace("/RE: /", "", $baseTitle);
            $queryResult = $vbulletin->db->query_first("
                SELECT threadid
                FROM " . TABLE_PREFIX . "thread
                WHERE title = '$title' OR title = '$baseTitle' LIMIT 1
            ");
            if ($queryResult['threadid']) {
                $threadid = $queryResult['threadid'];
                $type = "reply";
                $cronlog .= "Found reply for thread $threadid";
            }
            // ##### Time To Post The Reply #####
            if ($type == "reply")
            {
                // Building post info

[rbgrn]

I added one more change in quick - hiding email addresses

            // hide email addresses
            $mailmessage = preg_replace("/[a-zA-Z0-9\-_]+@[a-zA-Z0-9\-_]+.[a-z]{2,4}/", 'xxx@xxx.xxx', $mailmessage);