Just upgraded from the previous version (which worked fine) and installed perfectly so many thanks....What would be a great improvement would be be this IMO...

1 - A random default avatar for guest NNTP posts - Would tie in with forum based posts better and look better.

2 - Use the same quotation code as in the forum...i.e the tidy box that surrounds quotes normally within forum posts...I don't know how this would transfer back to the newsgroups however....??? Anyway, the Admin cp section really helps...really nice stuff...

Now keep in mind not every servers handles the same newsgroups... but here is a list I go by...

http://205.215.62.111/groups-list.txt (2mb in size).

I like the avatar idea... The quotes might be sorda hard because of the different ways the news clients quote messages... maybe a default for the standard ones could be built in?

Mark

My Newsgroup stuff gets a tad ignored because all of the NNTP guest post members don't/can't have an avatar which sort of gives the game away...I would imagine that this would better be placed as a "generic" hack as even it it were a static guest avatar it woud be a plausable idea being that boards could use a site logo as default...???

1. I am not sure it will work, but if someone wants to try it out. Add a user with the user ID of 0. And add avatars etc for the user 0.

2. What really matters is what contents is ended up on Usenet. If the normal usenet users see BB tags etc, they will start to hate your forums. So a text based message will be more acceptable. Even the script gets rid of the BB tags before sending them, there are always users who miss one "[" or "]" and stuff up the formating.:)

thanks for the list

SUGGESTION on next update
Create a script (ala cron job) to remove certain post(s) (by subject) that contains "porn", "$", "buy", etc, etc...

I've been playing with getting the email notifications to work with this script. If you find

in gateway.php and after it put:

It seems to work (well, it did the few times I tested it :speechless: )
Let me know if it works for you.

After a long battle with my host and finding a good NSP, I finally got this hack to work but only through port 23... which is fine with me as long as it works.

I would like to add to what Princeton has suggested. We absolutely need a filter system to screen out not only offensive headers but also attachments. One of my subscribed groups is a binary group. (alt.binaries.3d.lightwave) Lately, I have been deleting virii that come through and plant themselves as simple attachments. The main type has been the .scr variety. I would hate to see one of my members click on one of these attachments before I can find it and del it. Perhaps we can intergrate the attachment file extension permissions from VB into this hack, no?

www.cghaus.com

I have been writing that today (coincidence). I am just testing it, and will post the instructions tomorrow. It checks the extension type aginst what has been allowed in the vb control panel.

These code changes use vb's attachment permissions (extensions only) before inserting attachments into the database - so no more .scr files :squareeyed:

If the file extension is not set up in the 'Attachment Manager' 'Extensions and Sizes' section of the control panel then the attachment is ignored.

I have tested it on version 2.1.0. All changes are to 'gateway.php'. There are 6 additions or changes to the script to be made to 'gateway.php'.

Find

CHANGE it to:

Find:

AFTER it put:

Find:

AFTER it put:

Find:

AFTER it put:

Find:

CHANGE to:

Find:

CHANGE to:

And that is it :nervous:

This script doesn't check against the maximum file size or image dimensions permissions, but it would be fairly easy to add on however I'm not sure of how useful this would be, as people posting to newsgroups would be unaware of your boards limits.

AWESOME!!! Thanks Kevin! Your hard work is appreciated. This will go along way towards making this hack a little bit more secure and BB friendly.

Hmmm.... it seems I am now getting an error. I'm not sure if this is related to the new attachment perms or not.

I also upgraded to the new version this morning.

Maybe I made a mistake writing the instructions. I have attached a version of gateway.php which is working for me with the attachment permissions.

Try this.

thank you Kevin ... no problems on my end

Imhotep,
ReCheck your code ... you probably made a mistake copying / pasting

Kevin, thanks, this file fixed the problem. I am usually very careful about how I copy/paste, so I am not sure if I made a mistake or not. I suppose if princeton got it working, then I probably fulbbed somewhere along the line.

thanks again

In the NNTP Gateway setting, under the "Email address domain" option, I'm not quite sure what to put there. What do you mean by Do I need to change the default value?

Also, for the "Mailing list E-mail" setting, do I put the email address of the mailing list (eg, coolgroups@yahoogroups.com) or is it the email address that will be sending messages to the mailling list (eg, me@mydomain.com). The yahoo mailing list that I want to archive only accepts emails from verified email addressess. Is that going to be a problem?

Finally, two suggestions:

1. For mailing lists, all emails will have a common word or phrase in the subject. For example, the subject of all messages for the "New DVD Releases" mailing list would look something like:

[NewDVD] Spider Man deluxe edition
RE: [NewDVD] Spider Man deluxe edition
etc...

It would be great if you could add a setting in the gateway setup that would allow the user to specify the subject phrase (eg, [NewDVD]) and then strip that from the subject text so that it doensn't show up in the title of every post. Also, even more importantly, have a check box that would allow us to only import message that has the subject phrase. This would allow us to filter out messages that are not actually mailing list messages.

2. Every mailing list message that I get has the same text in the footer. It would be great if there was a setting somehwere where I could specify this footer text (its over 13 lines!) and have it stripped before importing the message.

Thanks, this hack is really useful for me!

I've been thinking about attempting a 'killfile' feature (a common newsreader feature) which would allow the administrator to set some parameters and stop the downloading of certain posts into the forum.

I propose it works like this:

In the admin control panel, you enter the phrases (surrounded by {} ) which you would not want downloaded (similar to the censorship feature).

As each post is processed, it checks the headers, author and title of a message to see if it matches with the phrases, and if it does, discard it.

Possible uses:
You could stop a specific troll's posts from reaching your boards by putting the posters name into the killfile (although any replies by other usenet users will get through).

You could block messages originating from a specific server (for trolls that keep changing their posting name), although you risk blocking anyone else using the same server (small risk).

You could set up common 'spam' message titles to block.

Downsides:
Performance. Putting dozens of words or phrases into the killfile will slow down the processing of messages while the gateway script is being run.

Pureists. Some people using the newsgroups may consider this a form of moderation in an unmoderated group.

I'd really appreciate others users of this scripts thoughts.

Is this worth pursuing and is it the right approach?

Sounds good. Would be great if it worked the other way as well; only posts with a certain word(s) in the title would be posted to the board (eg, see my request #1 in my above post).

This could work for mailing lists, but not newsgroups. Also, each mailing list would require a different keyword to be matched in the title.

I agree about the footer. Again you would need to accomodate multiple different footers (for each mailing list, or for groups like yahoogroups which regularly change their footers from day to day). Any risk to possible violations of the terms and conditions from the list operators?

Yes, I was referring to mailings lists. Ideally, I would like to see a form for every mailing list, which would be required for any "catch phrases".

At the moment none of the posts that I get for my mailings lists are being threaded; they are all only one post deep and each reply shows up as a seperate post. It looks the "RE:" portion of the subject title might be causing problems since it can show up in several different spots in the title. For example, for the original thread, [NewDVD] Spider Man deluxe edition, the replies show up as :

RE: [NewDVD] Spider Man deluxe edition
[NewDVD] RE: Spider Man deluxe edition
RE: [NewDVD] RE: Spider Man deluxe edition

I'm guessing it's just how the various mail programs that people use handle inserting "RE". Looking at several hundred messages, the best way to determine which reply goes with which message would be to look at the title starting from the end of the message and working backwards. You don't usually see the [xxx] or the RE: part at the end of the subject. A quick browse through about 300 messages showed that if I used the last three or four words of the subject line it would have accurately put the replies with the correct original thread.

I am using the old version of gateway.php (The one that worked with the Beta versions of VB3 where you had to edit the database by hand to add newsgroups)

I am thinking of upgrading to this version. :)

Do I need to do anything to make this version work, such as

Delete all current Newsgroup forums? (To prevent messages in these forums from being sent back out to the Internet)
Do I need to delete any of the old database enteries?

Anything else I need to do to upgrade from the old version?

Thanks for your help and for what appears to be a great hack!

Not 100% sure but I do not think there has been any DB changes... I went from the edit by hand DB to the new version with no issues.

Great did you have to empty your existing newsgroup forums? Or was it fine?

I just dont want to send all the messages I have now to be sent back out to usenet. :)

I''ve been using this since vb 2.30 - upgraded to vb3 and then 3.01. Have upgraded the script several times and have *never* had to empty existing newsgroup forums. The script just works :)

Nope... did not have to empty... The major changes are just the way the gateway works and pulls the new messages as well as the admin i face now.

I am having a look at this now, however I don't have any mailing lists which makes it difficult to test.

The way I have it working is, the administrator enters the characters or phrases that starts the footer message into a setting in the cpanel (not the whole footer, just enough that it is 'unique' and is not likely to form part of a genuine message).

You can enter as many different footers as required.

Every mail message gets checked against these, and if there is a match, strips everything from that point onwards from the message. Usenet messages are ignored.

The only situation where this wouldn't work is where a footer is quoted in someone elses reply, and they then bottom post their message beneath this. From memory of using mailing lists I don't recall this happening often, but I need a guide from you. (It would be ton easier to not have to worry about this :nervous: .)

Let me know if this approach will work for your lists, and will you be able to test the script?

KevinM, that is exactly the kind of solution I was hoping for :) The footers in the mailing lists I use alwasy start with the same text and all text after that can be ignored. I'm not too concerned about the quoted footer text myself, so whatever would be easier for you. I would be happy to test it.

Lierduh, are you still supporting this hack? I haven't seen you comment in over a week so I was just wondering.

What this does:

If you are using this hack to mirror mailing lists you will often find that the mailing list operator adds their own footer to each message which can be quite large.

This function will strip this from the message, prior to insertion into the forum. It can handle multiple different footers (just surround each one with { and }).

I have tested this on a test forum, but I am looking for feedback from a live enviroment.

I'll give this a go, thanks!

It worked just as I had hoped it would, thanks for taking the time to write that. :D Messages look much less cluttered now.

One issue that I'm still having is that the vast majority of the mailing list posts are not being threaded and are showing up as individual posts. I thought it may have been an issue of the title changing when people reply and it adds an "RE:" to various places in the subject (see my previous post #140), but after looking at it more closely, posts with the exact same subject are showing up as seperate threads:

http://orthotechforum.com/images/mlist.gif

I can't figure out why some (very few) posts are threaded but others with the same subject are not. Admittedly, I know zero about coding, but shouldn't it be pretty straightforward to determine which replies should be threaded based on the subject title? I'm wondering if it might have something to do with the "Use Post Time" setting. The default is set to use the time stamp when the messages are imported, which means that all of the posts at the time of import have the same time stamp. I changed it to use the original post time, thinking that posts with the same title and same timestamp might somehow be throwing off the threading. I'll test this and see how it works. Any idea though why the posts are threading correctly?

Yes, this has been mentioned before. I remember from the early days of this hack when I used it to mirror a yahoo group, I spent half my time manually merging threads. It drove me mad.

As far as I can tell, the script doesn't rely on matching thread titles at all, but by references in the headers (which works great for usenet, which is built on this standard).

If there isn't a reference or a match then a new thread is started. I should think it is possible to do a secondary check against a thread title before starting a new thread.

I will have a look and see how difficult this would be to implement. I am no expert at php - I just look at other peoples scripts and figure it out (sometimes!).

That would be great, let me know how you make out.

1. Doesn't mailing lists add prefix to any messages posted to it? The two type of lists I have admined do. they are sympa and mailman.

2. I consider it rude to strip the footers.:) The lists attach them for a purpose, either advertising or rules. They should be presented as they are received in an email mail box. If the list adds 13 lines of footer to the messages, perhaps the subscribers can complain to the list owner to have it shrinked down.

The threading works by checking references in the headers. All the email clients I have used set this field in the header correctly. Although I have not checked, but I am pretty sure that is the requirement set out by the RFC for the email clients. Could you please post the headers of the messages that are not threaded correctly?

I was just out of town for a week.:)

If anyone is using the (beta) versioin from the early days. There is nothing needs to be done. The AdminCP will do the rest for you when you fire them up the first time.

The AdminCP interfaces are also: Installation script, upgrade script. It is designed in a way not to slow down the normal operation when the version is already up to date.

1. I'm not sure. I'll post the header though.

2. At the moment I am archiving the group just for my own purposes and the forum is hidden to everyone but myself. If I ever made it public I would likely add the footers back in.

Here are four messages (from a Yahoo group) with the exact same subject line that were not threaded:

I notice the 3rd one didn't even have a message ID or a "In reply to". That's not a typo, I wonder if it has something to do with his AOL client. None of the Message-ID's or In-Reply-To match up.... I guess that is the problem :surprised:
If that's true, would it be possible to thread the posts based on the subject title? Then maybe the post order within the thread could be determined by post time.

Welcome back :)

I have something on my test system on the threading-by-subject which is about 70% complete. It turned out to be easier than I thought.

The way it is working out:

1. It check against references (as it always has - no change here).
2. If there is no reference match, it checks if "Re: " appears in the subject title
3. If it does, it checks for a match by subject title (with the "Re: " removed) and forum and inserts the post.
4. If not, it creates a new thread (back to the existing script)

If Lierduh finds a way by using the references alone, then that will be a far more efficient way of threading as this will add two more queries to the script.

How does it handle multiple "Re:" in the subject?

Good question. I've got it working fine on my test system now, and it works with multiple "Re: " in the subject.

I will do a final test tomorrow on a clean 'gateway' script, write some instructions and then you can be do some testing on a real mailing list if that's ok. :nervous: