vBouncer: reduce server load by unsubscribing bouncing members from threads

vBouncer is a handy tool for busy forums (or those that do not require email verification) to control the amounts of bogus emails sent from and to your server.

What vBouncer does: vBouncer monitors your mail spool file, and identifies members responsible for large amounts of bounced emails. It then issues a PM warning, and when the unsubscribe threshold is met, it will will unsubscribe them for all threads, and sets them to the "no email" subscription level.

After a couple of runs, this should significantly reduce your server load.

Features:
o Manual or auto processing option
o Collect logs via IMAP/POP or mail spool file
o Custom filters to match patterns in bounced emails
o Uses vB's cron files (Scheduled tasks)
o Uses encrypted header for authentication (suggested by Paul M).
o Stats on members with bouncing emails, and error codes
o Tuning options
o Verbose cron logs for diagnostics
o Option to send PM warning prior to reaching the limit
o Option to send PM notifications after reaching the limit
o Option to change the member's subscription default
o Option to all thread subscriptions to "no email"
o Option to all forum subscriptions to "no email"
o Option to move offenders to a different primary usergroup
o Option to process 5.0.0/550 errors (email unknown) upon a single email, isntead of observing the bounce limit

Installation:
1. Extract the files into the corresponding directories (admincp => admincp, includes => includes)
2. Run the installation script in the admincp
3. Fill out the settings menu in the vBouncer section of the admin panel (see the how-to post below)
4. Import and activate the plugin file through the plugin manager.
5. Once ready, add the cron scripts to your scheduled tasks: vbouncer-collect.php (should be run daily, or more frequently) and vbouncer-clean.php (weekly, equal to Bounce limit period)
6. If you are using vB's SMTP email option, make sure to also enable the "-f" option.

Change log:

- August 12, 2005 v1.3 Update to the cron files to match new changes in vB's cron/tasks. To upgrade, just re-upload the contents of includes/cron.

- July 11, 2005 v1.2 Bug fix for some server/PHP setups; resolve aliases for return keys, otherwise some aliases would be considered forged.

- July 5, 2005 v1.1 Bug fix in standard report pattern matching

- July 4, 2005 v1.0 Added encrypted header for authentication (suggested by Paul M).

- July 3, 2005 v0.9.1 fixed a bug preventing IMAP/POP log collection.

- July 2, 2005 v0.9 added IMAP/POP option and custom filters file vbouncer.ini

- June 27, 2005 v0.5 Added custom header setting option, for re-written headers.

- June 26, 2005 v0.4 Found a way to make it work as a plugin, without code edits (thanks to Paul M)

- June 26, 2005 v0.3 No changes, just removed the plugin

- June 25, 2005 v0.2 Added NDR/Reply code errors to stats, and option to act upon 5.0.0/550 (user unknown)

- June 21, 2005: v 0.1 Initial release. Marked beta, but it's working fine on PostFix and SendMail. Need to hear from those who tried on QMail and Exim

[tamarian]

Originally Posted by merk

Great

Does PHP need to be compiled with some extra support to connect to POP mailboxes?

Yes, it needs "--with-IMAP". The good news is that control panel configurations (who mostly would need the POP method) usually compile a lot of stuff into PHP.

For those who need to check: See vB's control panel under the phpinfo part in the maintenance/diagnostic section. If you have IMAP compiled, it would have a section titled IMAP and show status as "enabled".

[CB|Steffen]

I like the idea, but there might be some kind of security problem: How do you make sure that the bounce is valid? I think this system could be abused to annoy users by selectively sending fake bounces resulting in account deactivation.

[tamarian]

Originally Posted by CB|Steffen

I like the idea, but there might be some kind of security problem: How do you make sure that the bounce is valid? I think this system could be abused to annoy users by selectively sending fake bounces resulting in account deactivation.

That is true.

What I do is enabple a PM warning first, which means the member has 7 days to contact me if these are not really bounced emails.

I don't use the option to change user group, it was only added by request (I also made a warning under that option in the setting )


On my forums, the worst that can happen is to change subscription from instant email to "no email". Which means they can still see subscribed threads from the userCP.

[merk]

A random number/hash could be generated for each email sent and stored so that the system knows its a legimitate email?

Im much worse about bounced email, if it bounces once they get moved to awaiting email confirmation (and a big red warning appears on every page in that group).

[tamarian]

Originally Posted by merk

A random number/hash could be generated for each email sent and stored so that the system knows its a legimitate email?

I gave this some thought, and concluded that it's best not to do that. Here's why: vBouncer is optimized not to do anything unless a bounced email is found. To implement a hash method, vBouncer will need to run an insert query for every email sent. The hash number will need to be stored in the database for every single thread notification sent. vBulletin will send multiple thread notifications for each new post, relative to the number of members subscribed to that thread, plus an extra qury when a bounced email is received, to match with the hash number sent. This is a huge number of queries, and will defeat the purpose of reducing the server load.

Im much worse about bounced email, if it bounces once they get moved to awaiting email confirmation (and a big red warning appears on every page in that group).

That's why I don't like that option I much prefer to let vBouncer run it's course. Send a warning on the first run, and take action on the second run (defult run period is 7 days).

[merk]

Fair enough, though the system is open to abuse if you think about it. I wonder if there is another solution to either make it harder or impossible to abuse.

[tamarian]

Originally Posted by merk

Fair enough, though the system is open to abuse if you think about it. I wonder if there is another solution to either make it harder or impossible to abuse.

I think the abuse would require all the following conditions:

1. Enabling auto processing option
2. And enabling usergroupid change option
3. And allowing members to see other members email addressess (or them finding out the email addresses from the members themselves)
4. And being able to forge emails with correct bounce syntax.

But an alternative would be to ignore forged email headers. I may write some pre-checks, and/or allow the option to call user-defined scripts, like SpamAssasin and the like, to allow returning a true or false answer, and ignore emails based on that answer.

[merk]

Originally Posted by tamarian

I think the abuse would require all the following conditions:

1. Enabling auto processing option
2. And enabling usergroupid change option
3. And allowing members to see other members email addressess (or them finding out the email addresses from the members themselves)
4. And being able to forge emails with correct bounce syntax.

But an alternative would be to ignore forged email headers. I may write some pre-checks, and/or allow the option to call user-defined scripts, like SpamAssasin and the like, to allow returning a true or false answer, and ignore emails based on that answer.

Good point. I dont allow users to see other users email addresses, so that should be enough to make sure its okay?

[Paul M]

I am actually testing what seems a very simple answer to the above, which also gives me something consistant to look for in the returned mails.

I use the following code to add two headers to every outgoing e-mail

PHP Code:

$mycode = "qwertyuiop"; // change this to something unique //
$headers .= "X-Tracker-Dst: " . $toemail . $delimiter;
$headers .= "X-Tracker-Chk: ". MD5($toemail . $mycode) . $delimiter; 


Every system that bounces mail should return the original headers (all that I have tested do) so you get these headers back with all failures.

The bouncer code then scans the inbox looking for all the X-Tracker-Dst headers to get the original destination, and pulls the X-Tracker-chk for each one. It then does the same MD5 calculation and if it's a match it knows it was a genuine bounce. To spoof this - the spoofer would need to know your unique code in order to get the MD5 correct. It also means that all I need to look for in the rejects inbox is these two headers.

[Paul M]

On a seperate note - I'm going to see if I can figure out why the forum cannot see the mail box - since my vB can see my attachments folder, which is at the same level as the mail folder - it maybe nothing more than permissions.

Edit, okay, it's not permissions. A deeper look required over the weekend.