vb.org Archive - Major Additions - Email Integration (New threads/replies by email)

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 3.6 Add-ons (https://vborg.vbsupport.ru/forumdisplay.php?f=194)

- - Major Additions - Email Integration (New threads/replies by email) (https://vborg.vbsupport.ru/showthread.php?t=151222)

Email Integration (New threads/replies by email)

This mod is based off of the Mail Reply modification by Colin F to which I have obtained permission to rewrite and release.

This modification allows you to mimic email lists such as yahoo groups through your forums!

After installing this modification you will have new settings in your forum manager where for each forum you can enable this modification and setup a separate email address to use for each forum that you have this enabled for.

Just like how there is a separate email address for each yahoo group.

After doing so, members can subscribe to each forum that this mod is enabled for to receive notifications for new threads and replies.

Any posts in that forum, will be sent to them via email. (Example email attached below). The users can then reply to that email (which will then be processed and posted to the forums through the cron job that runs every 10 minutes), or they can send a new email to the email address and a brand new thread will be created!

Essentially mimicing an email list! Suddenly those users that hate forums, can still be dragged into the conversations and help keep the forum's activity level up!

BUGS!
These are the known bugs to date:

Confirmed bug that email processing does not handle Japanese characters.

Unsupported Items
These are the items that are not currently supported:

Some custom bbcode modifications will not display like they do on the forums in the html emails. Like glow, and the table mod.

TO UPGRADE

Please see the file in the zip for upgrading!! If your upgrading from version 2.3.X you will need to reverse some file edits that are no longer nessecary!

NEW INSTALL

See instructions in the zip file.

VERSION HISTORY!
(See the file in the zip for a complete history!)

2.6 Release
- Bug Fixes
  - Joining an unmoderated public group will now auto subscribe you per the usergroup settings.
  - NO CONFLICTS with Instant Thread Subscription! See details in below post.
- New Stuff
  - User Option Allow Auto Subscribe - This is an admin allowable, user option that let's the user choose to not be auto subscribed.
  - User Option not receive own posts - A new user option that lets them select to NOT receive their own posts via email.
  - Editted the phrase for the Auto Subscription script to tell people what to do if there are no listings.
  - Added error checks when saving a usergroup for if auto subscriptioin is entered with a forumid for a forum that does not have email integration enabled and also checks that the usergroup you are saing is entered in the forum manager as allowed to use email integration.
2.6.1 Release (BETA VERSION)
- Hot Fix
  - It appears I was unseting a variable too soon. I've corrected this and it has resolved the issue with multiple posted replies on my test server.

Most of these will be slow to make it in til I know I've got the code to handle just about every type of email we could receive :) But in anycase, here are the ideas for future versions!

End User Option to subscribe to just receive new thread emails, or receive all new thread & reply emails (current version does all new threads & replies).

Thanks to RedTyger, Bob Denny, Ed Kohwley and Chris McKeever for their additions and assistance with portions of this code.

Huge thanks also goes to the many individuals that have helped test various incarnations of this mod.

MAD PROPS TO: cgmckeever for a great quoting regex for the old format, and all the mods at www.4winmobile.com for helping test version 2.4 prior to release!!

You guys rock!

Please feel free to donate to my continued work on this modification!! It let's me spend more time on it! :)
Make a Donation!

Version 2.6 will work with both 3.6.8 and 3.7!

The #.X versions are the solid tested versions.

The #.X.X versions are the beta versions. New features and bug fixes will be worked in there and then when I have a solid version it'll be released as the next #.X :)

Thank you!!

~ Cyricx

FAQ

How do the emails work?

Quote:

This is a basic rundown of how the emails work..

When a new thread is created on the forum, plugins check for the email integration settings and then send an email out to everyone that has subscribed to that FORUM for instant email notification.

When a new reply is added to a thread, the plugins check to see what forum that thread is in, and then if that forum has email integration settings, and then sends an email out to everyone that is subscribed to that FORUM for instant email notification.

The cron job does the processing of the emails.

It checks your forums for email integration settings and then connects to each of the mailboxes one at a time. Pulls all the messages out, decodes them, cleans them up, checks for errors etc.

To determine who the user for the email is it checks the email header for who it came from. If you have strict email turned on, it grabs the username and then verifies what domain sent the email. If you have strict email turned off, it just grabs the email address of the person that sent the email. Note: this can be easy to fake.

O.. and yahoo mailing lists... do not use strict email checking. You can post as just about anyone you want in seconds from microsoft outlook :)

It then checks the "in_reply_to" in the message header. This should contain a predefined group of characters that let's us know it's a reply and assigns the post to the correct thread and parentid. Some virus scanners change this so in the subject line of reply emails is the threadid number. We check for that next as a backup :) If it doesn't match either of these criteria, it's classified as a new thread.

If it is determined that the email is a reply, it uses the datamanager to send the reply and uses the plugins to send out the emails.

If it is a new thread, then it uses the datamanager to post the new thread and uses embedded code in the file to send the email.

It then flags the message to be deleted and before closing each mailbox, expunges all marked messages from it. (Just incase an email came to the box after it had grabbed the list, this way one doesn't get lost! :) )

So the only time you'll get an email when you run the cronjob is when it is grabbing an email :)

Additionally, manually running the cron job will sometimes cause the quote boxes to not appear in the mailbox. This is due to the bbcode_quote template not being cached in the admincp ;)

Error - Undefined Function

Quote:

If you get the error the below error you will need to enable IMAP support on your server. If you host your own server, check your php.ini file, the line for imap support will be commented out. If you do not host your own server, request your host to enable this.

Fatal error: Call to undefined function: imap_open() in
/*****/includes/cron/emailintegration.php on line XXX

To verify if you have IMAP installed go to your admincp -> Maintence -> View Php Info and you should see something like this:

additional .ini files parsed /etc/php.d/imap.ini,

OR

imap

IMAP c-Client Version 200X

To correct this problem you will need to have the php.ini file recompiled with IMAP functions enabled -
http://us.php.net/manual/en/ref.imap.php

You do not need to have IMAP installed, but if you wish to also install IMAP, instructrions for doing so can be found here -
http://us.php.net/imap

Error - Couldn't open stream

Quote:

If you get this error while manually running the cron job (suggested to do the first time to be sure it's running right!)

imap_open() [function.imap-open]: Couldn't open stream

Your connection type setting for one or more of your forums are incorrect. I suggest only setting up one forum the first time. Edit that forum, and try one of the other 6 connection types. Then rerun the cron. It's a trial and error if you do not know how your server is setup. Some people use IMAP, some POP3 and others still use POP3 with Official Certificate, so try all 6.

Can I use a Gmail account to pull the emails from?

Quote:

You most certainly can.

Login to your mailbox at gmail, go to Settings -> Forwarding and POP/IMAP

Enable POP for the account, either from here on, or for all mail. Be sure to select archive or delete for what to do after the messages are downloaded. Otherwise the system will continue to repost the emails!!

In your forummanager use the folowing settings:

email address to reply to: username@gmail.com
hostname: pop.gmail.com
username: username@gmail.com
password: password
Type of connection: select "POP3 with SSL (Self signed certificate)"

Done!

Smilies don't show up in html emails!

Quote:

Images are stored in html files with the url for the location of the image. Edit your smilies in the smilie manager and give them all the true url of the image location. For example instead of /smilies/smilie.gif change them to http://www.yourdomain.com/forum/imag...ies/smilie.gif

That will allow them to show up correctly in email.

How does it work with Thread Subscriptions?

Quote:

This plugin does NOT change how thread subscriptions work. You get the same emails, same format for thread subscriptions.

It just adds an option to instant subscribe to the forum and those emails you can reply to.

However... it does have a series of checks to be sure that these two don't conflict. Here's a fairly concise list of how this plugin integrates with the subscribe thread function.

When an email reply is processed, it checks if user's are subscribed for instant notification for the forum AND the thread.

If they are, it removes their subscriptions to the thread and sends them a forum email.
If a user is only subscribed to the forum, then they get the forum email.
If a user is only subscribed to the thread, they get vbulletins default thread email.

When a reply is posted on the forums for a forum that email integration is turned on for, it runs the same checks as above.

Additionally, if a user has it setup to be instantly subscribed to all threads they reply to it checks then to see if they are subscribed to the forum. If they are, it doesn't allow the system to also subscribe them to the thread.

If a user tries to subscribe to a thread for instant notification for an EI Enabled forum and they are also subscribed for instant notification to the forum, it'll give them an error. However, they may still subscribe to the thread for daily or weekly updates while still having instant notification to the forum.

Is that confusing enough?? Trying writing the code for it! ;)

Look forward to your more stable version Cyricx as think this may well work for my forum. I like the idea so will watch this one grow in version number ;)

Sounds like a great start well done.

Funboy

I wonder how this would work for us with vBseo. :)

Looking forward to seeing this one with the bugs fixed. :D

This is a great idea! I'm looking forward to see'ing this out of beta!

this is an awesome idea for my mobile users. I will try this out on my test board in the next few days and report back

will need a link in the foot of the email to disable subscription of threads...

this is great tool.... thanks!

I like this idea too, I've always wanted to do this for text messages.

I've been waiting for something like this for some time. Hope development goes smoothly.

I need this! :)

once it became stable.

Wow -- I'll be keeping my eye on this! Thanks for working on it! :)

Awesome! Thank you very much! :D

Great idea! :) Will be keeping an eye on this.

Quote:

Originally Posted by nexialys (Post 1281622)

will need a link in the foot of the email to disable subscription of threads...

this is great tool.... thanks!

Great idea! I'll be workin on those templates some while I work on the format of the emails that are posted and the blank bug. I'll add that in :)

I'm also going to add a little blurp that says "You may also start a new thread by sending an email to $foruminfo[ei_replyaddress]." :)

EXCELLENT man

when do you think it`ll be out of it`s beta stage i`m dying to try this hack :)

It depends alot on how much time I have to work on it. I've been drilling it pretty religiously all day and got alot more accomplished. :)

Hoping for a few donations that will continue to fund time spent on this. Regardless of that though, I'll continue to hammer on it :)

I can't really offer an ETA due to that though, but I'm sure as you all watch my progress with the betas as they roll out, you'll see this is a definate passion to get released as I think this is a core function that should be included in vbulletin. :)

Quote:

Originally Posted by Cyricx (Post 1282063)

Excellent!

This is a great idea, but I'd rather have the people coming back to my site to reply to threads in hopes that another thread title will catch their attention and to get them more active on my site.

This is fab, and there is one thing that will turn this to another level IMO.

Would it be possible for this to receive email attachments (images) and attach them to the new posts? This would allow anyone with an MMS phone to email their Media Message to the forums/site e.t.c.

Keep up the good work, thanks!

Quote:

Originally Posted by FleaBag (Post 1282855)

This is planned for the next few betas after I've got some bugs fixed. I plan on having a global on/off for accepting attachments and a on/off for accepting all attachment types (for example, for forwarded emails) or restricting it to only allowed image types like images :)

Quote:

Originally Posted by AzzidReign (Post 1282630)

This is a great idea, but I'd rather have the people coming back to my site to reply to threads in hopes that another thread title will catch their attention and to get them more active on my site.

True.. the problem that we as forum lovers run into is that there are many people that due to evil work environments can only access email during the day and rarely check anything else. This is the group that LOVES mailing lists and email groups. We are missing out on a whole nother level of interaction in users with this group. If we can bring these people into the fold as well, then we increase the posts on the forums, the stuff to read, and likely convert a few of those mailing list lovers to forum readers :)

Eagerly awaiting this mod to reach a stable point!

Quote:

True.. the problem that we as forum lovers run into is that there are many people that due to evil work environments can only access email during the day and rarely check anything else

So true, content is king, and if people can post in working hours by replying emails,
forum activity will certainly increase. Furthermore, the awareness of the existence of
the forum will increase.

The next beta version will be released soon. It looks like I was actually able to totally remove the file edits! :)

I'm working on a new width bug I found with the messages, and then I'll work on the usergroup bug. The usergroup is the big one, but if the messages look like crap, it hardly matters how well it works ;)

Just wanted to give you all an update :)

This looks sweet, please get this out of BETA, please :)

subscribed!

Something this big, I don't wanna use beta! ;)

That's a great thought. But in practice we lose a lot of visitors to Yahoo Groups because they like getting emails and they reply to the email. I am waiting for this hack to be completed and I will donate to the cause.

Quote:

Originally Posted by FleaBag (Post 1282855)

Thanks for the support gang :)

I haven't released the next beta yet as I'm hoping to jump straight to a solid version. I've fixed both the blank bug, and the quote boxes, as well as I have recoded almost all of the code that I had permission from RedTyger to use.

I'm left with Attachments and Forum Permissions. Attachments should be quick and easy. The share class file that I'm using (same one that Colin used) already pulls out the attachments from the message, so I just need to add a few catches in there for those that don't want to allow attachments and for those that want to restrict the type of attachments :)

Getting close!!

Great idea >>>> you are great }}}}} waiting for ur final release :)

I am definitely going to be watching this with great interest, I can't wait for the stable version!

hmmm i will be watching this it goes final! good work! :)

Thank you so much for developing this feature. My past online communities formed originally from mail lists -- ONElist which became eGroups which preceded Yahoo Groups. Once we outgrew the free sites, I bought my vBulletin license and it's been online forums ever since.

I really miss being able to hang out on my own machine without being tied to a browser page -- while getting immediate messages (or daily/weekly digests) and sending out immediate replies right there in my inbox (without having to go to the site all the time). BUT I still want to have an online forum/community to visit too. It'll be great to offer both forms of interaction for my membership so it would better cater to each individual's preferred method.

Thanks again.. this is an excellent idea.

great idea looking forward to it

This is one of the first mods in a while to actually make me DROOL.

I seriously cannot WAIT to see this in action!

I have a forum that has migrated recently from a Yahoo! mailing group ... and participation is down. These folks, for whatever reason, were more comfortable with the "push" email interaction, and haven't taken the initiative to actually visit the forum on their own. It's an uphill battle, and now they're all asking why we made this switch in the first place. This mod would give us the best of all worlds, it seems!

I will certainly donate when it's made available.

Yeah we're going through the exact same transition, trying to move users from a mailing list to a forum and have some users in rather remote areas.

For example one user lives in Yap, Micronesia and only has 10 hours internet access a month, so this product would be IDEAL for what we need!

Waiting with great anticipation!

Just to give everyone an update.

My current build has all known bugs resolved. I'm currently workin on the attachment code. I'm bustin on this pretty hard to get a non-beta version released :)

I actually dreamed about php code last night :eek:

Cyricx... just wanted to chime in with encouragement. I can't tell you how excited I (and my forum's members) are about the prospect of this mod. Comes at such a great time, too, since they're struggling with the transition from a Yahoo! mailing list. Your work could solve so many problems and really bridge a gap that vBulletin has had for a long time now.

So, thanks.

Well a donation sent from me to

Phrase Groups Available:

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.03910 seconds Memory Usage 1,875KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (12)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)pagenav (1)pagenav_curpage (2)pagenav_pagelink (1)pagenav_pagelinkrel (1)post_thanks_navbar_search (1)printthread (40)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start pagenav_page pagenav_complete bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: