vBouncer (PHP): reduce server load by unsubscribing bouncing members from threads
 

There's also a vB 3.5 version: https://vborg.vbsupport.ru/showthrea...threadid=83486

vBouncer is a handy tool for busy forums (or those that do not require email verification) to control the amounts of bogus emails sent from and to your server. This was initially released for vB 2.x and vB 3.0.x as a Perl script, and is now fully integrated with vB as scheduled tasks with admin panel control.

What vBouncer does: vBouncer monitors your mail spool file, and identifies members responsible for large amounts of bounced emails. It then issues a PM warning, and when the unsubscribe threshold is met, it will will unsubscribe them for all threads, and sets them to the "no email" subscription level.

After a couple of runs, this should significantly reduce your server load.

Features:
Manual or auto processing option
Collect logs via IMAP/POP or mail spool file
Uses vB's cron files (Scheduled tasks)
Custom filters to match patterns in bounced emails
Encryption header to authenticate bounces.
Stats on members with bouncing emails, and error codes
Tuning options
Verbose cron logs for diagnostics
Option to send PM warning prior to reaching the limit
Option to send PM notifications after reaching the limit
Option to change the member's subscription default
Option to all thread subscriptions to "no email"
Option to all forum subscriptions to "no email"
Option to move offenders to a different primary usergroup
Option to process 5.0.0/550 errors (email unknown) upon a single email, isntead of observing the bounce limit

Installation:
1. Extract the files into the corresponding directories (admincp => admincp, includes/cron => includes/cron)
2. Run the installation script in the admincp
3. Fill out the settings menu in the vBouncer section of the admin panel (see the how-to post below)
4. Edit includes/mail.php
Find:Replace with:
In admincp/index.php, find:
Add before::
5. Once ready, add the cron scripts to your scheduled tasks: vbouncer-collect.php (should be run daily, or more frequently) and vbouncer-clean.php (weekly, equal to Bounce limit period)

Change log:

- July 11, 2005 v1.2 Bug fix for some server/PHP setups; resolve aliases for return keys, otherwise some aliases would be considered forged.

- July 5, 2005 v1.1 Bug fix in standard report pattern matching

- July 4, 2005 v1.0 Added encrypted header for authentication (suggested by Paul M).

- July 3, 2005 v0.9.1 added fixed a bug preventing IMAP/POP log collection.

- July 2, 2005 v0.9 added IMAP/POP option and custom filters file vbouncer.ini

- June 27, 2005 v0.51 for 3.0.7 Corrected installer file.

- June 27, 2005 v0.5 for 3.0.7 Add setting for custom header strings.

- June 26, 2005 v0.2 for 3.0.7 Initial release No changes, just removed the plugin

Screenshots from 3.5 beta, but it looks the same on 3.0.7...

This post will be used for an updated FAQ and How-To's

How it works
1. vBouncer makes your forum send emails with a modified "Return-Path" header including the vBouncer emaila ddress, so most ISP's will send bounce notices to that email (instead of the webmaster email).
2. When ISP's send the bounced notices, those notice emails are stored in the mail spool of vBouncer. Most will tag those notices with a "Final-Destination" header stating the email address that bounced.
3. The vBouncer cron task (vBouncer-collect.php) checks the spool file and extracts all email adresses that are bouncing, and store them into a database table, if they belong to a forum member. It wil then empty the spool file.
4. If you selected auto processing, the vBouncer cron task (vBouncer-clean.php) reads the table and stats, and executes the options you chose to apply in your vBouncer settings. The database table will pruned from any old data.

Email setup: For vBouncer to work, you need to:
1. Create a new email account on your server, and that email account has to be unique, and only used for this purpose, collecting bounced emails.
2. This account should not be POP'ed or deleted by any user, only vBouncer should delete them.
3. The vBouncer emails should be in a regular spool file (text file). This is standard on SendMail and PostFix. Not sure about Qmail, Exim or Windows, but they should able to do it. (Anyone with experience in those systems, please let me know)
Please check with your server admin how to accomplish this.

Tips:
If you can't make your mail server spool to a single text file, you may want to try the (dot)forward file method:
http://publib16.boulder.ibm.com/pser...es/forward.htm
See post #3 below for further details on mail servers.


vBouncer Settings The comments on the vBouncer settings page are verbose, but here are the important ones with additional comments.

Post notification sender email
This is the email vBouncer will use to send new reply notification. You need to create a unique email address for this. Bounced emails will be returned to this email address on your site/server domain. This email account should not be downloaded, and must be left waiting, vBouncer will pick it up.
Example: subscriber_notify@mydomain.com

Full path to email spool file
This is the full path to where the mail spool file resides. This is where bounced email notifications are sent back to the post notification email.
Example: /var/spool/mail/subscriber_notify
(Use the diagnostics option to sure the file is readable and writable)

Maximum unique members to process per run
Processing the mail log file will stop when this limit is reached. If your logs repeatedly excede this number, it is better to run the scheduled vBouncer collect task more frequently, instead of making this limit too high, for performance reasons. Example 100

Auto processing mode?
You have the option to either let vBouncer automaticly change subscription levels for members with bouncing emails, or choose no if you prefer to simply decide based on the stats of bouncing emails.
Yes No
(Choose no, if you just want to monitor the stats)

Bounce limit
If you selected auto-processing, how many bounced email notices should be acted upon?. (Only works in auto mode) Example: 50

Bounce limit period
Enter the number of days in which the bounce limit set above will be applied. (Only works in auto mode) Example: 7

The above two options results in setting a limit of X email bouncer per Y days period.

Send Notification PM
PM Sender ID
PM Sender Username
Send PM warning
Warning Limit
Change default subscription level to no email
Change subscribed threads to no emails
Change subscribed forums to no emails
Change usergroup for bouncing members?
Destination usergroup
Protected usergroups
Untouchable usergroups
These options control what actions to do with members who excede the set limits.

Note: If you use vBouncer's IMAP/POP method, you do not need to follow the instructions here. This is only for those who prefer to use mail spool files.

Spool files How-To

This post addresses the location and setup of spool files on different mail server software. You will only need it if you can't find your mail spool file, or don't have one.

Sendmail: Sendmail by default spools the emails for each email address into a text file. Usually found in /var/spool/mail/account_name (account_name is the account in the email address, like subscriber_notify, for example)


Postfix: Postfix, like Sendmail, spools the emails for each email address into a text file. Usually found in /var/spool/mail/account_name (account_name is the account in the email address, like subscriber_notify, for example)

QMail: (Thanks to Merk for testing and providing the information) QMail, by default stores emails in Maildir format, new file for each email. In order to spool emails to a single file, you need to do the following steps:

In the home directory of the user account (subscriber_notify), create a file called .qmail the content of that file should be a single line indicating the file path where you want the emails to be stored:

/var/spool/mail/subscriber_notify

Or any path you choose. The file needs to exist, so we can first create it by using the touch command:

touch /var/spool/mail/subscriber_notify

And it needs be readable and writable by PHP, so use the chmod command:

chmod 666 /var/spool/mail/subscriber_notify


Vpopmail+QMail: (Thanks to Merk for testing and providing the information) For vpopmail, a user doesnt have a home directory, and it dosent go into the users virtual mail directory either, it sits just below that in the domain's "root" directory. /home/vpopmail may be different for some users.

/home/vpopmail/domains/domain.com/.qmail-subscriber_notify for subscriber_notify@domain.com (contents as above)

The contents of the .qmail file are still the same described above for QMail. You just need to follow the same steps to create .qmail file and the file to spool the emails too.



Exim+cPanel: (thanks to Paul M for this tip)

Note, before you do this, check this post:

https://vborg.vbsupport.ru/showpost....&postcount=101

It may be best to avoid changing the config of Exim, and just use the IMAP/POP option in vBouncer.

In the exim config file ( /etc/exim.conf ), near the bottom is a section that begins with virtual_userdelivery: - in that section is a line which says mode = 0660 - change this to mode = 0666

In your cpanel account, for address rejects@domain.com

The path needs to be " /home/<cpaccount>/mail/domain.com/rejects/inbox "

The mail, domain.com & rejects folders all need chmod'ing to 755.

The mailbox file can then be set to 666, and it will work.


Windows Mail Servers: Still waiting for testers and information


----------------------------

If your server is running Exim or a Windows mail server, please post here:

https://vborg.vbsupport.ru/showthread.php?t=83407

Once the information is provided (in that thread), and we get you up and running, the instructions for those specific mail servers will be posted here.

Note that it may actually work out of the box even if you don't have SendMail or PostFix. If you have a text file on your server that can store the emails for a specific account, then you're all set. It's just that I have no experience with those systems, so I don't know.

i will install with Exim. let u know.

Great, I look forward to the results. :)

I've installed the PHP version as I loved the PERL version but was always too lazy about running it.

I've got a Cpanel server with Exim, and I've setup an account (for the old vbouncer) named subscription_bounces@example.com (obviously not example.com, use your imagination and insert domain name for the rest of this post ;)).

The old one used this file:
/home/example/mail/example.com/subscription_bounces/inbox

Which worked fine, and was run from SSH under the example account.

The diagnostics for the PHP version doesn't like this though.

It tells me that the file does not exist, that it is not writable nor readable. But logged on via SSH I can see it, and I've given read permissions to everyone for the file.

The PHP process tends to run as nobody as I'm running eaccelerator and that requires PHP to run as a module rather than CGI. However I'm unconvinced that's the issue as the file is world readable.

Any ideas why the diagnostics doesn't like it?

when i run diagnostics, it tells me that the files doesn't exist and is not readable or writeable.

appears Exim is one file.

/home/name/mail/domain/subscriber_notify/inbox

edit: just read the post above. same issue here.

buro9 and 7thgenCivic.Com - the reason is in my post here (problem 1).

https://vborg.vbsupport.ru/showthread.php?p=726338

The file should be also writable by the apache process. Use chmod 666

Paul also mentioned this might be due to restrictions of PHP access outside the HTML root.

You might want to try a sym link

ln -fs /home/example/mail/example.com/subscription_bounces/inbox /path/to/admincp/subscriber_notify

Now you should be able to use /path/to/admincp/subscriber_notify as the file location (also make it chmod 666

Let me know if this fixes the problem

The symlink didn't work. I set the permission for both the file and the link as 666. Diagnostics still failed.

I've also tried disabling the open_basedir setting for PHP within httpd.conf but that fails.

How much effort do you reckon for creating a PHP POP3 variant of this hack for those who have trouble reaching the spool file?

I'd consider that option, but first we need to know why the file can't be read. What are the apache error logs showing?

Not a thing unfortunately. It's not logged as an error in Apache.

The vBulletin scheduled task log says this for the Collect task though:

version 0.5 released: Option to use custom header strings to identify bouncers. (Example, some Exim users may need to enter 'X-Failed-Recipients:'.

To upgrade: uninstall, then re-install :)

I posted this in the 3.5 thread, but maybe soemone here can test it:

---

I would like someone using Exim+Cpanel to test the .forward (dot-forward) file method :)

Here's how it works:

1. Create a file somewhere in your public_html directory /path/to/forum/admincp/subscriber_notify
this file should be readable/writable (666)

2.In your (subscriebr_notify) user home directory, create a .forward file. The .forward file should have one line in it:
/path/to/forum/admincp/subscriber_notify

The theory is that Exim will start forwarding emails to that file.

Not sure what Exim considers "home" directory, so you may need to try creating it in different directories for that user account (subscriber_notify) to test which directory is the right place for .forward

Send an email to subscriber_notify, and check if it was copied to that file. Exim manual states that this should work, so let's keep our fingers crossed. :)

Another request for Cpanel+Exim users. I'm writing a POP maodule as an option instead of reading files. I'll need to test my code (from my home server) with your bouncer email box. If you'd like to volunteer :) please PM me the new account's (subscriber_notify) POP username and password.

I don?t see that option...

My bad. Updated zip 0.51 should have it.

i am moving servers, so I am going to wait. but it will still be cpanel.

When i run the install file it says:

Fatal error: Call to a member function on a non-object in **********/public_html/forums/admin_cp/vbouncer_install.php on line 20

And line 20 is $vbulletin->input->clean_array_gpc('r', array(

And when i uncomment that line i get an error; Any ideas what's wrong?

And im using 3.0.6 with security fix from 3.0.7

I made the 0.51 update, but forgot to upload the file. Get the new zip, and let me know how it goes.

That error came from the 051 update

That just can't be :) Line 20 is not the same as the one you posted.

Is it possible you downloaded the 3.5 version?

Tnx! Now it?s working!

The next version will have a POP/IMAP option. This will be useful to those in chroot environments, or those on multiple servers, or using external mail servers.

I have it working, but I need some test volunteers. If you have a POP account for the subscriber_notify account, send me the POP access info by PM. (The worst I can do is delete a couple of bounced emails!) :)

I can't install Exim on my system, since I never used it, and would be time consuming for me.

I have a problem to get the email address of certain bounced mails, for example this:

I don?t know what custom header have to use for this bounce...

Can u help me?
Tnx in advance.

Psico.

Yes, this is currently what I'm working on for the POP part, where some MTA's don't show full error-code headers, just content. What is your server's email software? Exim?

If you can send a large file containing all the bounces, it would help me test the new code. Better yet, temporary access to the subscriber_notify POP account. :)

Here is the bounce file.

I can´t give u access to the pop account because i´m using qmail with something like vpopmail, so the mail ends in the file not in the pop folder...

I really need this hack...badly...my server have too much load because of bounced mails...

Tnx a lot!

No problem, I'll send you temp copy to test. Please save me a copy the bounced email file, as the bigger the sample the beter.

The sample above is not enough?

Here is another...

I prever something really big, just to see how your MTA rewrites headers from different ISP's. Think something in the tune of 100 megs :)

No, impossible...this will take weeks or more...

Hello tamarian.
I've set this hack up using the latest version.
My host has set all emails with the particular email address to go to /var/mail/admin.
It's 1,547,032 bytes in size at the moment and -rw-rw-rw-.
I've gone through the vbounce settings and selected my preferences and I have added collector and cleaner to the scheduled tasks.
Everything passes the diagnostics.
To speed things along I have run the collector but in the vbouncer report it shows nothing has happened.
Is there any way I can check to see why nothing is happening?

Are you using Exim? Was the mail file exclusively full of bounced emails? (I mean not SPAM and stuff, just hard and soft bounces). Was the scheduled task log really empty, no text or status header?

I'm working on handling Exim reports (or those with bounce messages without bounce reports). If you have a decent sized sample, please email it to me :)

Sorry, I don't know what Exim is or how to view the contents of the mail file. Though we've sent some test emails to that account.
The scheduled task log for collect reports 'No matches found'.

What email software is installed on your server?

A couple of things might be happening:

1. The file does not contain bounced emails, just regular emails, spam and the usual viruses, etc.

2. The file contain bounced emails, but no MTA bounce reports.

If it's #1, you may need to create a new email account for the sole purpose of using for vBouncer, so the file will contain only bounced emails.

If it's #2, your MTA doesn't generate bounce reports. This should be handled by the next version I'm working on.

Working perfectly .. Thanks alot dude :)

psico and spinball: PM me your email addresses. I'll send you a copy of vbouncer-collect.php to test and see if it works on your setup. If all goes well, I'll release it along with the POP/IMAP code.

I emailed your last message to my host and he 'tweaked' and the hack now works!! Yeah!! Thanks for your help. 893 bounced emails right there!
I'm not sure I understand how altering the Bounce Limit and Bounce Limit Period affect how this hack works. For example, if a user has just a few bounced emails - say 5, I want to deactive their emails right away. I don't understand what the bounce period does.

One other thing - and was it you I had this conversation with re vbusy? The graphics don't appear in the chart because the specified location in the template is 'http://www.mydomain.com/forums/images/blahblah' and you might want to test for this in your hacks so the graphics work for people who have vBadvanced installed.

I assume he turned on bounce reports in the MTA, which I think would be the best solution. As this is very specific to each server setup, I can't offer tips on how others can do it, but if you can share what he did, that would be great.

Here's an example, say you set up:

Bounce limit = 50 emails
Bounce limit period = 7 days

Then only those who reach 50 email bounces within 7 days period will be processed.

You can set it to:

Bounce limit = 1 email
Bounce limit period = 1

This will auto process the user for any bounced email received

so what is the status with Exim? I moved servers, but it still doesn't work. I have an 853kb inbox file if you want to look at it. But i don't think that is the issue. permissions is the problem.

Derek