vBouncer: reduce server load by unsubscribing bouncing members from threads

vBouncer is a handy tool for busy forums (or those that do not require email verification) to control the amounts of bogus emails sent from and to your server.

What vBouncer does: vBouncer monitors your mail spool file, and identifies members responsible for large amounts of bounced emails. It then issues a PM warning, and when the unsubscribe threshold is met, it will will unsubscribe them for all threads, and sets them to the "no email" subscription level.

After a couple of runs, this should significantly reduce your server load.

Features:
o Manual or auto processing option
o Collect logs via IMAP/POP or mail spool file
o Custom filters to match patterns in bounced emails
o Uses vB's cron files (Scheduled tasks)
o Uses encrypted header for authentication (suggested by Paul M).
o Stats on members with bouncing emails, and error codes
o Tuning options
o Verbose cron logs for diagnostics
o Option to send PM warning prior to reaching the limit
o Option to send PM notifications after reaching the limit
o Option to change the member's subscription default
o Option to all thread subscriptions to "no email"
o Option to all forum subscriptions to "no email"
o Option to move offenders to a different primary usergroup
o Option to process 5.0.0/550 errors (email unknown) upon a single email, isntead of observing the bounce limit

Installation:
1. Extract the files into the corresponding directories (admincp => admincp, includes => includes)
2. Run the installation script in the admincp
3. Fill out the settings menu in the vBouncer section of the admin panel (see the how-to post below)
4. Import and activate the plugin file through the plugin manager.
5. Once ready, add the cron scripts to your scheduled tasks: vbouncer-collect.php (should be run daily, or more frequently) and vbouncer-clean.php (weekly, equal to Bounce limit period)
6. If you are using vB's SMTP email option, make sure to also enable the "-f" option.

Change log:

- August 12, 2005 v1.3 Update to the cron files to match new changes in vB's cron/tasks. To upgrade, just re-upload the contents of includes/cron.

- July 11, 2005 v1.2 Bug fix for some server/PHP setups; resolve aliases for return keys, otherwise some aliases would be considered forged.

- July 5, 2005 v1.1 Bug fix in standard report pattern matching

- July 4, 2005 v1.0 Added encrypted header for authentication (suggested by Paul M).

- July 3, 2005 v0.9.1 fixed a bug preventing IMAP/POP log collection.

- July 2, 2005 v0.9 added IMAP/POP option and custom filters file vbouncer.ini

- June 27, 2005 v0.5 Added custom header setting option, for re-written headers.

- June 26, 2005 v0.4 Found a way to make it work as a plugin, without code edits (thanks to Paul M)

- June 26, 2005 v0.3 No changes, just removed the plugin

- June 25, 2005 v0.2 Added NDR/Reply code errors to stats, and option to act upon 5.0.0/550 (user unknown)

- June 21, 2005: v 0.1 Initial release. Marked beta, but it's working fine on PostFix and SendMail. Need to hear from those who tried on QMail and Exim

[merk]

good idea, solves the query problem

[tamarian]

Originally Posted by Paul M

I am actually testing what seems a very simple answer to the above, which also gives me something consistant to look for in the returned mails.

I use the following code to add two headers to every outgoing e-mail

PHP Code:

$mycode = "qwertyuiop"; // change this to something unique //
$headers .= "X-Tracker-Dst: " . $toemail . $delimiter;
$headers .= "X-Tracker-Chk: ". MD5($toemail . $mycode) . $delimiter; 


Every system that bounces mail should return the original headers (all that I have tested do) so you get these headers back with all failures.

The bouncer code then scans the inbox looking for all the X-Tracker-Dst headers to get the original destination, and pulls the X-Tracker-chk for each one. It then does the same MD5 calculation and if it's a match it knows it was a genuine bounce. To spoof this - the spoofer would need to know your unique code in order to get the MD5 correct. It also means that all I need to look for in the rejects inbox is these two headers.

I like the idea. But I'm not sure you will get all your headers back, I think it will depend on the ISP. I think AOL would be one of the ISP's that won't give you back all the headers, and some would give it back depending on the bounce type.

But I will test it out, and see if the percentage of those who do return it. If it's reasonably high enough (50+%), I think it would be worth it. An option can be added to see if the admin want to to rely on it, or ignore it.

[merk]

Originally Posted by tamarian

I like the idea. But I'm not sure you will get all your headers back, I think it will depend on the ISP. I think AOL would be one of the ISP's that won't give you back all the headers, and some would give it back depending on the bounce type.

But I will test it out, and see if the percentage of those who do return it. If it's reasonably high enough (50+%), I think it would be worth it. An option can be added to see if the admin want to to rely on it, or ignore it.

Maybe a 3rd option, process automatically any email that has the headers to confirm it, and if it doesnt have them, put them in a "manual processing queue"?

[tamarian]

Originally Posted by Paul M

On a seperate note - I'm going to see if I can figure out why the forum cannot see the mail box - since my vB can see my attachments folder, which is at the same level as the mail folder - it maybe nothing more than permissions.

Edit, okay, it's not permissions. A deeper look required over the weekend.

I strongly suspect it is due to a (chroot)ed environment. chhroot setup will cause different users to have a distorted view of the system directories. If I login as superuser, my /home /etc /var directories are "real", and mapped to the main directories. If I login as a regular user, my directories are mapped. So when as a regular user I look for /var/spool/mail, in reality it's /home/users/chroot/var/spool/mail. So the path name to the file is not identical to both users.

[tamarian]

Originally Posted by merk

Maybe a 3rd option, process automatically any email that has the headers to confirm it, and if it doesnt have them, put them in a "manual processing queue"?

That's doable. Maybe give it a status/error code of 'no sure', and let the admin decide from the stats page.

I plan to test these headers on a live system, just to see if I really get back a decent percentage of the original headers, this may take a couple of runs to calculate.

To give you guys some heads up, I may release 0.9 tonight for the 3.0.7 version. It has two new features:

1. IMAP/POP support, so those on control panels, or sites with external email servers, or those who just prefer not to deal with mail spools can use it.

2. Custom bounce filters. This is an optional config file that allows you to insert multiple custom regular expressions to match the patterns on bounce emails on your server that vBouncer couldn't catch. There's just too many special cases for each server MTA setup, that I though t's best to allow custom regex, rather than hardcode them. Plus, it will not require me to keep on releasing a new version for each pattern

[Christine]

Hi tamarian,

Sorry to offer late, but just getting back to this. Do you still need cPanel access and/or a POP account?

[tamarian]

Originally Posted by Christine

Hi tamarian,

Sorry to offer late, but just getting back to this. Do you still need cPanel access and/or a POP account?

cPanel access is not needed, but would love to test the POP module on an Exim. So if you have a POP account that has bounced emails (or empty and not being used), please PM me the access info

[Paul M]

Originally Posted by Paul M

On a seperate note - I'm going to see if I can figure out why the forum cannot see the mail box - since my vB can see my attachments folder, which is at the same level as the mail folder - it maybe nothing more than permissions.

Well after further testing - it is a permission problem.

In your cpanel account, for address rejects@domain.com

The path needs to be " /home/<cpaccount>/mail/domain.com/rejects/inbox "

The mail, domain.com & rejects folders all need chmod'ing to 755.

The mailbox file can then be set to 666, and it will work - except - that as soon as another mail is received, exim chmod's the inbox file back to 660, cutting off php's read/write access. I now need to see if I can stop exim doing this.

[tamarian]

Originally Posted by Paul M

Well after further testing - it is a permission problem.

In your cpanel account, for address rejects@domain.com

The path needs to be " /home/<cpaccount>/mail/domain.com/rejects/inbox "

The mail, domain.com & rejects folders all need chmod'ing to 755.

The mailbox file can then be set to 666, and it will work - except - that as soon as another mail is received, exim chmod's the inbox file back to 660, cutting off php's read/write access. I now need to see if I can stop exim doing this.

How about changing the owner (chown) as well? If exim is no longer the owner of the file (but still has write permission) it should not be able to chmod it.

[Paul M]

Originally Posted by tamarian

How about changing the owner (chown) as well? If exim is no longer the owner of the file (but still has write permission) it should not be able to chmod it.

What or who would I set the owner to ? I have fixed it on my test server by changing one line in the exim.conf file - but this may not be possible if you are not on a dedicated server.