NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

[lierduh]

The latest version: V2.3.2 (31 Oct 2004)

What is it?

This hack provides a gateway from vBulletin forums to the USENET. It allows forum members to read newsgroup messages as well as to send posts to the newsgroups. In a nutshell, it turns vBulletin forums into a basic function news reader.

For an added bonus, this package also supports mailing list archive and forum to mailing list gateway. The software retrieves messages from a pop3 account and import them into the forums threaded. It also allows a forums user to send messages to the mailing list by the forum posting interface.

Why?

The gateway will be appreciated by forum members who do not know how to set up a news reader, or people who do not have port 119 open in their network. A forum interface to read and post news are often considered to be better than a news reader.

The gateway adds more contents to your forums and allows the members to interact with wider audience. It is common to see new users signing up to use this feature.

History

This hack is basically a hacked vB2 NNTP gateway written by Gilby to work with vB3. The original hack can be found here:

https://vborg.vbsupport.ru/showthread.php?t=35247&page=1&pp=15


Features

Some of the features and improvement include:


Handles both newsgroup posts and mailing lists.
Handles MIME encoded messages and attachment.
Handles UUEncoded messages.
Two Admin Control Panels for all settings.
Automated installation.
Correct thread implementation. All imported messages and posts sent to newsgroups will be threaded correctly according to References and Message-ID headers.
Optional setting to thread by subject for mailing lists that strip references header.
Supports TABLE_PREFIX.
Imported messages will be searchable just like normal forum posts.
Similar threads are built when the messages are imported if the system setting is on.
Multiple attachments importing.
Save attachment to database or file according to vB setting.
Thumbnail created while importing image files.
Multiple attachment post to newsgroup is also supported.
Selectable X-No-Archive header honouring system.
Light on system resources.
Global Killfile to filter out unwanted messages. Killfiles can have OR and AND effectiveness.
Strip footer by setting for incoming messages. Strips out repeated message footers.
Built options to disable signatures and footers for out going messages.


Future development


AdminCP interfaces. (completed)
Installation/upgrade script. (completed, coded into AdminCP)
Traditional "> " lead posting interface.


What do you need and do not need?

You do not need to edit vB php files if you use two bookmarks
You need to edit one vB php file to insert two lines of code if you want direct click links within AdminCP.
No need to run SQL.

Installation

Basically the AdminCP interfaces take care of installation as well as upgrade in the background. Back up your database first. The scripts can modify your database without alerting you!

Please follow the instructions in the readme.txt file for more details.

Support

Please post your support questions to this thread. Please check the FAQ in the third post within this thread first.

================== UPDATES ======================
Jul 31 2004 (release version 2.2.0)

Provided option of not importing attachments
Provided option of pause between messages
Provided option of not sending messages to USENET
Provided mechanism to skip importing a message if it was crashed on it
Provided "Default Footer" option
Provided strip_footer function for mailing lists
(written originally by KevinM)
Provided Killfile functions with Header/Body/AND/OR settings
Provided ability to thread messages by subject for mailing lists
Provided ability to use a single POP3 account for different mailing lists
(use prefix)
Merged KevinM's attachment routine, save thumbnail to hard drive
as vB3.0.3
Official support to have the script run by Scheduled Tasks
Many bug fixes


Please note, when you manually run the script, you must turn on the debug mode. Otherwise nothing will be displayed on the screen.

To run as debug mode: http://forums.yourdomain.com/gateway.php?debug=1

25 Oct 2004

Provided the option of sending posts using the real email address
Provided the option of sending posts using a fixed email address for
each group.
Posts are now held back if they are in the moderating queue
Scheduled task log is now working
Provided the option of sending the debug messages to the vB3 cron log
Bug fixes


31 Oct 2004

Fixed bugs running the gateway by the vB3 Scheduled Tasks

1 March 2004

Uploaded the new zip file. It contains one or two bug fixes.

[Intex]

Very nice work - I'm yet to install, but I'm sure many members will find it useful as with the first version.

[lierduh]

I reserve this post position for support FAQ. This way it will be easily found and referred to.

If you have problem retrieving and posting messages, please make sure your web server can access the NNTP server.

If you have shell access to your web server either by telnet or ssh. You can test the server access using:

=============================
First, ssh or telnet to your web server.

Then telnet the news server from your web server. For example:

[ns@mb eyoforums]$ telnet msnews.microsoft.com 119
Trying 207.46.248.16...
Connected to msnews.microsoft.com (207.46.248.16).
Escape character is '^]'.
200 NNTP Service 6.0.3790.1146 Version: 6.0.3790.1146 Posting Allowed
GROUP microsoft.public.access
211 13988 1 86055 microsoft.public.access
ARTICLE 86054
==============================
ie. after telnet to the NNTP server by using "telnet msnews.microsoft.com 119"
type
GROUP group.name
then type
ARTICLE xxxxx

(xxxx is the post number)

You should get the whole article including the header. If the telnet does not even work. There is no point trying to fix the gateway.

You can do other things while you are under the telnet session. Type "help" will get a list of the commands you can use. To quit a telnet session, type push Ctrl key together with "]" without the quotes.

Server Crashed during the creation of gateway tables, now I am getting MySQL errors when open AdminCP interfaces

The best way to do when this happens is to delete both nntp_groups and nntp_settings tables. Open the AdminCP interfaces. These tables will be recreated by the AdminCP interfaces. Tables will also be upgraded to the latest version automatically.

It tries to download instead of displaying the contents on the screen when I run it manually

Please fix the Windows registry following the instruction here:

http://support.microsoft.com/default.aspx?scid=kb;en-us;329661&Product=ie600

The gateway retrieves the messages our forums sent

This usually happens if the news server rewrite your Organization header, eg. to insert advertising etc.

Please set the "No Organization Check" to '0' without the quotes.

Which newsgroup providers do other forum owners use?

Please check out this thread:

https://vborg.vbsupport.ru/showthread.php?t=75232

[Erwin]

To upgrade from the beta vB3 version in the vB2 hack thread to this new vB3 version, you need to upload the new files here, and then run these queries before running the new gateway to add the new nntp settings variables:


INSERT INTO `nntp_settings` VALUES (NULL, 'Is Gateway Running', 'is_running', '0', 'Check if another instance of gateway.php is running', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Last Time Run', 'last_run', '1084316100', 'Last time the gateway.php was run', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Honour No-Archive', 'honor_no-archive', '0', 'Do not import the message if the message has the X-No-Archive header set to yes. Google Group honours this Non-standard header.', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Last Max postid', 'last_postid', '0', 'The max postid in post table when the script was last run', NULL, 0, 0);


Otherwise, this new gateway.php won't work with the old settings. :)

[Cap'n Steve]

Cool. Anything's better than google's interface.

[Majik?]

What exactly do I need to be abl to run this? I use SSH to connec tto my Web server and while in the SSH session I tried the command telnet news.swiftel.com.au 119, but got the error bash: telnet: command not found.

[lierduh]

What exactly do I need to be abl to run this? I use SSH to connec tto my Web server and while in the SSH session I tried the command telnet news.swiftel.com.au 119, but got the error bash: telnet: command not found.

That means your host has removed telnet program from your server, or disallowed the users to use telnet.

Above steps are only for fault finding. No need to try if you have no problem.

By the way, in the example, I used my own news server. You need to use your own.

[Erwin]

lierduh's upgraded USENET hack seems to work a whole lot better than the old ones. :) No more blank threads, it actually works faster, and there is less load on the server. Thanks!

[allan grossman]

Hey, lierduh -

Suggestion for a future release:

I've commented out the entire signature routine in gateway.php to keep my users from being flamed hard by real Usenet users - since you have to be pretty durned old to remember when a signature was

<dash><dash><space><cr>
No more than
four
lines of
text

Setting a database field to skip the whole signature routine to keep Usenet neophytes from getting their feelings hurt might be kinda cool - but commenting out the routine doesn't take a whole lot of technical expertise.

Just a thought ;)

[lierduh]

Hey, lierduh -

Suggestion for a future release:

I've commented out the entire signature routine in gateway.php to keep my users from being flamed hard by real Usenet users - since you have to be pretty durned old to remember when a signature was
Just a thought ;)

That should be quite easy to implement. At the moment, the users can uncheck the "Show your signature" when post. I can have a "disable signature" option in the settings table to disable the signature altogether.:)

Another thing I have on mind is to create a posting interface which behaves just like a news reader. ie. No BB codes available, just the "> > " for quoted text. It will be kind like WYSIWG of a news reader. This way, it will be much easier to trim the quoted text, and it will reduce the chance of forum news posts getting flamed. :devious:

Right now, I just want to get the second part of the Admin CP interface finished.

[Imhotep]

I have something to add to this discussion... something I never thought about until someone pointed it out. I have had this hack installed for at least a week and have never been able to get my posts out to Usenet. I just checked with my host and, sure enough, they have outgoing port 119 blocked! :ermm: To make matters worse, they refuse to open it up for my domain.

So, if anyone here is having problems with their posts not showing up, first thing I would do is check to see if your host is blocking outgoing port 119.

I'm screwed unless someone wants to code POP3 to NNTP or something to that effect. :ermm:

this sucks...

To those people who have this thing working, what hosts are you using? Mine is Lunarpages.

[lierduh]

I'm screwed unless someone wants to code POP3 to NNTP or something to that effect. :ermm:


That won't help your situation, your web server needs to talk to a NNTP server on port 119 to send the news to it. pop3 to nntp is used to post news by email instead of vB.

If you can retrieve news but not post, that means the port is open. Does the news server accepts posts? or does the server accepts posts from your web server's IP range?

If you can ssh to your web server, just try to use one of the Unix news readers from the shell. eg. tin. You can even use lynx to read news. Check Google, I can't remember.:)

[Imhotep]

If you can retrieve news but not post, that means the port is open.

Open for incoming only! The server is firewalled and port 119 outgoing is blocked! From what the support staff has told me, there is no way in hell I am getting anything out 119... ever. This is their reply.

Unfortunately, we will not open port 119. Usenet has had many longstanding
abuse issues, and opening port 119 would open it for anyone on the server,
which could cause many problems. I am sorry we will not be able to accomodate
you.

[allan grossman]

Some news providers will also let you run NNTP on other ports - like port 80. I'd contact my news provider.

At work the NNTP *protocol* is blocked :(

[Imhotep]

Some news providers will also let you run NNTP on other ports - like port 80. I'd contact my news provider.

At work the NNTP *protocol* is blocked :(


Good idea, Allan... I am going to look into it.

thanks!

[chas_h]

Lierduh,
I'm glad to see you moving this project forward! I'm among the ones who have had your earlier version running for several weeks on my VB-3 forum and I am really looking forward to these updates, especially the admin section.

Thanks!

[KevinM]

Lierduh,

Just got this up and running on a fresh install. Fantastic job. I've used the vb2 version for a year and this is a vast improvement. Particularly appreciated are the no-archive option and UU-encoded attachments (This now opens up all the binary groups which was too problematic with just mime encoded features). Control panel works great as well.

Two questions related to the attachments:

Is there a way of creating the thumbnails at the same time the the messages are downloaded? If you have the thumbnails option active, it shows the broken red x in the message. When you manually run the re-create thumbs option from the update counters menu it does create them, but it means re-creating the entire attachment table.

I can't get the attachments to work when using the 'attachments as files' option (this would be my preferred option with a vast number of attachments). the files download, but when you click the link on the post it can't find the file. When you reverse the process (i.e. switch back from files to db storage) the images reappear.

Thanks for a great upgrade.

[Imhotep]

Ok, so, for those who are following along with my connection "soap opera" woes, here's a bit of good news...

From Giganews...

Feel free to connect on the following ports: 119, 23, and 80(nntp only).

I am dropping Newsguy just as soon as I get home. I asked them about port 80, but apparently I can only use 80 if I go with their $75.00/month blah blah plan. LOL! Um... NO THANKS!

For those who are having problems posting from VB to Usenet, take note, port 119 on your web server may be blocked for OUTGOING data.

[allan grossman]

Ok, so, for those who are following along with my connection "soap opera" woes, here's a bit of good news...

From Giganews...



I am dropping Newsguy just as soon as I get home. I asked them about port 80, but apparently I can only use 80 if I go with their $75.00/month blah blah plan. LOL! Um... NO THANKS!

For those who are having problems posting from VB to Usenet, take note, port 119 on your web server may be blocked for OUTGOING data.

I've been using this hack with Giganews for quite awhile. Works just fine :)

Take a look if you like - I've got one newsgroup set to autoprune every day so it only holds 90 days worth of posts, but it works just fine.

Forumhome is at http://ebassist.com/vb3

The only mod to the gateway I've made is to disable signatures in gateway.php - but we discussed that earlier in the thread :)

edit: Before you switch news providers it might be a good idea to see whether your ISP blocks only port 119 or the entire NNTP protocol. Telnetting to a news server won't tell you whether your ISP blocks NNTP, I'm afraid :(

[chas_h]

Made the upgrades (including the added sql parts per Erwin, thanks) and the latest gateway hack is working just fine, I like the new interface in the admincp too.

Two questions;

1) Lierduh, when you talk about adding a 'new usenet posts' to the navbar, is that as part of the 'quick links' menu or is it a whole new button on the navbar? I need to add it to quicklinks because I have removed the other 'new posts' thing.

2) when Allan was talking about commenting out the entire signature routine in gateway.php, could someone show exactly what I need to comment out? I would definitely like to remove that part for now.

Thanks Lierduh!

[chas_h]

FWIW, I am using EasyNews as a Usenet provider, seems to work pretty well.

[mcncyo]

have a couple of questions. I would like to disable all post from a couple of users that are in the usergroup. Is there away of deleting thread topics that start with certin users?

[lierduh]

Just got this up and running on a fresh install. Fantastic job. I've used the vb2 version for a year and this is a vast improvement. Particularly appreciated are the no-archive option and UU-encoded attachments (This now opens up all the binary groups which was too problematic with just mime encoded features). Control panel works great as well.

That is good. I have not actually tried a binary group (now I tell you.:))


Is there a way of creating the thumbnails at the same time the the messages are downloaded? If you have the thumbnails option active, it shows the broken red x in the message. When you manually run the re-create thumbs option from the update counters menu it does create them, but it means re-creating the entire attachment table.

I will have a look at this.

I can't get the attachments to work when using the 'attachments as files' option (this would be my preferred option with a vast number of attachments). the files download, but when you click the link on the post it can't find the file. When you reverse the process (i.e. switch back from files to db storage) the images reappear.

Sorry for my ignorance, where is this option located? are we talking about outgoing or incoming. I can check it out, but I am feeling lazy after working on the AdminCP interfaces.

[lierduh]

1) Lierduh, when you talk about adding a 'new usenet posts' to the navbar, is that as part of the 'quick links' menu or is it a whole new button on the navbar? I need to add it to quicklinks because I have removed the other 'new posts' thing.

2) when Allan was talking about commenting out the entire signature routine in gateway.php, could someone show exactly what I need to comment out? I would definitely like to remove that part for now.

Thanks Lierduh!

"New Usenet Posts" will be at the main navigation bar next to the "New Posts" link if you followed the readme file. If you have moved the original link to quick link, you should be able to do the same for the USENET link.

The next version will contain the signature option.:)

[lierduh]

have a couple of questions. I would like to disable all post from a couple of users that are in the usergroup. Is there away of deleting thread topics that start with certin users?

Can't you disable them posting in the usenet forums altogether?

[mcncyo]

Can't you disable them posting in the usenet forums altogether?


i am talking about newsgroup users that i don't want to display on my messageboard.

[lierduh]

To upgrade from the beta vB3 version in the vB2 hack thread to this new vB3 version, you need to upload the new files here, and then run these queries before running the new gateway to add the new nntp settings variables:

Otherwise, this new gateway.php won't work with the old settings. :)

Update: With the AdminCP interfaces becoming available, this is no longer a problem. The first time the panel is openned, the script will detect and carry out the upgrade.:)

[Erwin]

To disable the forum signature, find and remove this code in gateway.php:


if ($newthread['showsignature']){
$get_userinfo=$DB_site->query("SELECT field1 AS signature,usertitle,customtitle ".
"FROM " . TABLE_PREFIX . "user AS user LEFT JOIN " .
TABLE_PREFIX . "userfield AS userfield ON user.userid=userfield.userid WHERE
user.userid={$newthread['userid']} LIMIT 1");

$userinfo=$DB_site->fetch_array($get_userinfo);

$signature .= "\n\n";
$signature .= format_text($userinfo['signature']);
}


The newsgroup signature (with links to your forum and the thread) will still be present. :) The large forum signatures will no longer be added.

[allan grossman]

Or - if you want to, just comment out the whole thing - change this // get user's signature
if ($newthread['userid']){
$signature = "\n-- \n";
$signature .= $newthread['username'];
//if ($userinfo[customtitle]){ $signature .= " - $userinfo[usertitle]"; }
//Only show signature if showsignature is true for this post
if ($newthread['showsignature']){
$get_userinfo=$DB_site->query("SELECT field1 AS signature,usertitle,customtitle ".
"FROM " . TABLE_PREFIX . "user AS user LEFT JOIN " .
TABLE_PREFIX . "userfield AS userfield ON user.userid=userfield.userid WHERE
user.userid={$newthread['userid']} LIMIT 1");

$userinfo=$DB_site->fetch_array($get_userinfo);

$signature .= "\n\n";
$signature .= format_text($userinfo['signature']);
}

$signature .= "\n------------------------------------------------------------------------\n";
$signature .= "{$newthread['username']}'s Profile: {$settings['profileurl']}{$newthread['userid']}\n";
$signature .= "View this thread: {$settings['threadurl']}{$newthread['threadid']}\n";
} else {
$userinfo = false;
$signature = "\n-- \n{$newthread['username']} - Unregistered User";
$signature .= "\n------------------------------------------------------------------------\n";
$signature .= "View this thread: {$settings['threadurl']}{$newthread['threadid']}\n";
}to this // get user's signature
// if ($newthread['userid']){
// $signature = "\n-- \n";
// $signature .= $newthread['username'];
//if ($userinfo[customtitle]){ $signature .= " - $userinfo[usertitle]"; }
//Only show signature if showsignature is true for this post
// if ($newthread['showsignature']){
// $get_userinfo=$DB_site->query("SELECT field1 AS signature,usertitle,customtitle ".
// "FROM " . TABLE_PREFIX . "user AS user LEFT JOIN " .
// TABLE_PREFIX . "userfield AS userfield ON user.userid=userfield.userid WHERE
// user.userid={$newthread['userid']} LIMIT 1");
//
// $userinfo=$DB_site->fetch_array($get_userinfo);
//
// $signature .= "\n\n";
// $signature .= format_text($userinfo['signature']);
// }
//
// $signature .= "\n------------------------------------------------------------------------\n";
// $signature .= "{$newthread['username']}'s Profile: {$settings['profileurl']}{$newthread['userid']}\n";
// $signature .= "View this thread: {$settings['threadurl']}{$newthread['threadid']}\n";
// } else {
// $userinfo = false;
// $signature = "\n-- \n{$newthread['username']} - Unregistered User";
// $signature .= "\n------------------------------------------------------------------------\n";
// $signature .= "View this thread: {$settings['threadurl']}{$newthread['threadid']}\n";
// }or - you can remove the entire routine. I choose to leave the code in place - which causes a tiny performance hit, but at least it's easy to remember what I did when a new release comes out :)

cheers,

allan

[mcncyo]

i am getting this error below



Fatal error</b>: Maximum execution time of 1800 seconds exceeded in <b>/home/forum2/public_html/forum/includes/functions_databuild.php</b> on line <b>477</b><br />

[1]+ Done

[allan grossman]

i am talking about newsgroup users that i don't want to display on my messageboard.

I'd like to weigh in here if folks don't mind.

Except for the .moderated newsgroups, Usenet is completely unmoderated - you can certainly do whatever you like with your forum, but IM frequently less than HO moderating an unmoderated newsgroup goes pretty much against what Usenet is all about.

What I do is run a disclaimer as an announcement at the top of the Usenet forums that reads like this -

==========

Folks -

For those of you who are unfamiliar with Usenet, the best description I've seen is at

http://www.faqs.org/faqs/usenet/what-is/part1

and

http://www.faqs.org/faqs/usenet/what-is/part2

No one "owns" Usenet and the groups that are gatewayed into ebassist are completely unmoderated. The feed that we provide is a service to ebassist users, and is just a gateway into another discussion area. You're liable to see flaming, links to ads and porn sites and a whole bunch of really objectionable stuff if you look hard enough.

But, as you continue to look, you'll see some real gems of information and some *very* knowledgeable folks.

Once again, ebassist doesn't own Usenet, has no control over its content and the feeds are provided without any interference (or moderation) from the folks on ebassist dot com.

Usenet can be harsh at times - and it's definitely not for the fainthearted, but I think providing an uncensored newsfeed provides more of a service to the ebassist community than filtering content would.

To sum things up, internal forums on ebassist dot com are moderated. External feeds like Usenet are not. Read at your own risk :)

the management

==========

But - it's your board and your call ;)

[KevinM]

Sorry for my ignorance, where is this option located? are we talking about outgoing or incoming.

In the 'attachments' menu on the control panel there is an 'attachments storage type' option, which enables you to store the attachments as files rather than in the db.

When this option is used, the link to the attachment in the post breaks (although the file still gets downloaded to the db, the script is searching for the file in folder ont eh server). Hope this makes sense.


I am feeling lazy after working on the AdminCP interfaces.


No problem, have some sleep first.... :nervous: !

[lierduh]

i am getting this error below



Fatal error</b>: Maximum execution time of 1800 seconds exceeded in <b>/home/forum2/public_html/forum/includes/functions_databuild.php</b> on line <b>477</b><br />

[1]+ Done


More information please. Is this the first time you run? how many posts roughly are we talking about importing? I have set the script maximum run time to 1800 seconds. If you can not import the whole lot, restart again, it will pick up where the last run left off. This is one of the features this new version has. Also how is your server load. Can it handle a lot of posts to be imported in one go? With each post imported, the server needs to build search index and it is resource intensive.

[mcncyo]

More information please. Is this the first time you run? how many posts roughly are we talking about importing? I have set the script maximum run time to 1800 seconds. If you can not import the whole lot, restart again, it will pick up where the last run left off. This is one of the features this new version has. Also how is your server load. Can it handle a lot of posts to be imported in one go? With each post imported, the server needs to build search index and it is resource intensive.


i have to run the script about 3 times and to get all of the information. I am still planning on adding more newsgroups to the site.


by the way i used the script to import about 30,487 post and it work fine. The only probem i had is everytime it did it is i have to change the "Is Gateway Running" option to 0 each time. My server load only went up to 2.0 which is 100% on both cpu.

[Erwin]

lierduh, you've released 2 updates already. Did you make any changes to gateway.php, or are the updates purely for the AdminCP?

[lierduh]

lierduh, you've released 2 updates already. Did you make any changes to gateway.php, or are the updates purely for the AdminCP?

The two beta files are only the AdminCP interfaces. The main script has been not updated since the ___b.zip .

I will do something tonight to add the signature option routine and may be others.

[Sanjiyan]

A few questions....

I am still running vb2.3.0, and gilby's version of this hack, the reason for staying with vb2.3.0 is because of the lack of a 'suitable' nntp hack addon, which you have now provided, so as long as it works with what I am about to ask, I shall update my forums to vb3.0.1

WIll this allow/import/whatever gilby's database modifications? as is from gilby's own version for vb2

Also as gilby's version allows me to pull from both NNTP and a pop3 mail box (downloads posts etc from a mailing list), will this version also allow it?

I do not use the pop3 to post from the forums to the mailing list, but to pull from the mail server the posts and then put them into the forums.

If you want to know what I mean about pulling posts from a pop3 mail box, please go here, and view the subforums, I have and the comments below explaining what they are:

http://www.sanjiyan.co.uk/forums/forumdisplay.php?forumid=35

Many thanks...

[lierduh]

A few questions....

WIll this allow/import/whatever gilby's database modifications? as is from gilby's own version for vb2


There is one extra field in the post table needed. The first time you open the AdminCP interface, it will detect what needs to be done and upgrade for you. This will also be the way it works with future upgrade.


Also as gilby's version allows me to pull from both NNTP and a pop3 mail box (downloads posts etc from a mailing list), will this version also allow it?
Many thanks...

I have disabled this because I have not checked whether it will work. Also no one else has mentioned the need to use this. I will have a look at it in the next few days and enable it in the next minor version. It is something pretty simple.

[Sanjiyan]

Thanks, because the pop3 function is something I rely very heavily on with this hack, as 40% of my need for this hack is for the pop3 mail pulling ability.

EDIT:

Also which file do I need to download? the nntp_gateway_v2.0.zip or vb3_NNTP_Gateway_May16b.zip

[Sanjiyan]

I have disabled this because I have not checked whether it will work. Also no one else has mentioned the need to use this. I will have a look at it in the next few days and enable it in the next minor version. It is something pretty simple.

Not trying to be big headed here, but the reason this option was put into the Gilby hack, was mainly because I asked it to be, I think to date no one else uses that function in the hack bar me, but it is a very nice function to have.

Anyway, can not wait for you to get this option working again, THEN I will upgrade my boards.

[lierduh]

Also which file do I need to download? the nntp_gateway_v2.0.zip or vb3_NNTP_Gateway_May16b.zip

The May16 release is a barebone release without the AdminCP and lack of couple of other functions.

The v2.0.1 is the full release with all the bells and wristles.:) I leave the May16 release in there just in case the new version breaks somehow. The May16 has been stable (...well, for 3 days now:)). So you should use the new one unless something is broken.

[yabba]

My gateway.php give this back from the newsserver :

1 group(s) gatewayed.

Connecting to server, server says: 200 feed.news.schlund.de InterNetNews NNRP server INN 2.3.5 ready (posting ok).

Info for rec.music.industrial at news.puretec.de:
411 No such group rec.music.industrial


:-( But the newsserver have this group, why found the gateway.php not such group ?

Greets yabba

[Erwin]

Just let us know when you update the gateway.php file - I am using the one that you posted when you started this thread.

[lierduh]

Just let us know when you update the gateway.php file - I am using the one that you posted when you started this thread.

The v2.0.1 contains the new gateway.php version.

[lierduh]

My gateway.php give this back from the newsserver :



:-( But the newsserver have this group, why found the gateway.php not such group ?

Greets yabba

Unless you made a typo for the newsgroup name, it should have nothing to do with gateway.php. :)

Try the fault finding method described in here:

https://vborg.vbsupport.ru/showpost.php?p=512830&postcount=3

Send the result to your web host, politely ask them why you can not access that group from your web server.

[mcncyo]

i would also like the pop3 accont to work incomming and outgoing. I would like to put a few mailing list on my message board.

[KevinM]

The POP3 function on the VB2 version is something I used, but it was really picky about the type of POP3 email account used, and from memory nobody could work out what caused it. It was a a case of trying different POP3 account suppliers until finding one that worked.

Actually, I would appreciate a POP3 recommendation that works with this hack. My BT account has just been taken over by Yahoo and no longer works and all my oneandone accounts (who do all my hosting) also don't work.

[Imhotep]

IMHO, it would be cool if we could add variables for incoming and outgoing ports. My host allows incoming on port 119 but not outgoing. So, with my new NSP, I am experimenting with getting posts to get out to Usenet.

Giganews allows connections to ports 119,80, and 23

Am I to assume that I can search the PHP files for "119" and change them to whatever I want? This is what I have done for my first test.... changing all instances of 119 to 80. So far, no luck. I hope my host did not block ALL outgoing NNTP data. :ermm: I still have 23 left to test.

[KevinM]

I am getting this error whenever it hits a message with an attachment (with the latest script)

<b>Warning</b>: fopen("/tmp/temp.bin", "x+") - Invalid argument in <b>/home/httpd/vhosts/ukgardenforum.co.uk/httpdocs/gateway.php</b> on line <b>970</b><br />
failed create a temp file

Is this related to creating the thumbnail?

[Odoin]

I have two newsgroup feeds setup... seems to be one of the forum pulls the posts made to it back into the topic so there is double posts.. one from the local user and the same post over acting as if it was from a outside user... Any idea why it would do this on just one forum?

[lierduh]

I have two newsgroup feeds setup... seems to be one of the forum pulls the posts made to it back into the topic so there is double posts.. one from the local user and the same post over acting as if it was from a outside user... Any idea why it would do this on just one forum?

Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?

[lierduh]

I am getting this error whenever it hits a message with an attachment (with the latest script)



Is this related to creating the thumbnail?

I will come up with a solution very soon.

[lierduh]

IMHO, it would be cool if we could add variables for incoming and outgoing ports. My host allows incoming on port 119 but not outgoing. So, with my new NSP, I am experimenting with getting posts to get out to Usenet.

Giganews allows connections to ports 119,80, and 23

Am I to assume that I can search the PHP files for "119" and change them to whatever I want? This is what I have done for my first test.... changing all instances of 119 to 80. So far, no luck. I hope my host did not block ALL outgoing NNTP data. :ermm: I still have 23 left to test.

The best way is to use the telnet method I described in the first page to find out exactly what is the problem.

As I mentioned before, if you can recieve the message, but not post. It mean the 119 port is OPEN! It means your web server can connect to the news server on port 119 using a high port on the web server.

What the host support might try to say to you if the guy really knows what he is talking about is. The 119 port is not open on your web server to let other people connect to it as a news server. That is, other people using a high port to connect to port 119 on your web server. This more likely is disabled!

For a TCP/IP to connect.

server port:119 <========> client port : high port (above 1024)

Also post some screen dump or test result always helps others to help you.

[Imhotep]

As I mentioned before, if you can recieve the message, but not post. It mean the 119 port is OPEN!

Agreed, but open one way only! Have you ever heard the term "firewall"? Like I said before, my host has blocked outgoing on port 119. They could care less about incoming. They block outgoing and cite "ongoing usenet abuses" as their excuse. Port blocking is a two lane road. You have incoming and outgoing. Just because I can receive posts does not mean it's open both ways. I hope I don't have to explain this again. :ermm:

And, since I did not get an aswer to my questions, but rather a misguided and thinly-veild scolding, I will have to email my host instead and see if they have a global block on all outgoing NNTP.

SERVER <====NNTP INCOMING====< NSP
SERVER >====NNTP BLOCKED?====> NSP

[lierduh]

Agreed, but open one way only! Have you ever heard the term "firewall"? Like I said before, my host has blocked outgoing on port 119. They could care less about incoming. They block outgoing and cite "ongoing usenet abuses" as their excuse. Port blocking is a two lane road. You have incoming and outgoing. Just because I can receive posts does not mean it's open both ways. I hope I don't have to explain this again. :ermm:

And, since I did not get an aswer to my questions, but rather a misguided and thinly-veild scolding, I will have to email my host instead and see if they have a global block on all outgoing NNTP.

SERVER <====NNTP INCOMING====< NSP
SERVER >====NNTP BLOCKED?====> NSP

It seems you understand about firewall and TCP/IP, please explain what port is blocked and what is open, be it UDP or TCP protocol. How about an iptable/ipchain rule table for the firewall?

You receive and post news on the same port! I like to see the result of the telnet session from your web server to the nntp server's port 119. Do you know you can post news in the same session.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.

I am here trying to help you, you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.

[Odoin]

No nothing has changed since the forum was first setup. It only double posts in one of the forums... the other one has no problem. I went as far as to remove the forum and newsgroup but it still did it... I am going to try a few other groups and see what comes up.


Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?

[lierduh]

No nothing has changed since the forum was first setup. It only double posts in one of the forums... the other one has no problem. I went as far as to remove the forum and newsgroup but it still did it... I am going to try a few other groups and see what comes up.

Can you get the full header for one of the messages in that group?

[recumbent615]

Not trying to be big headed here, but the reason this option was put into the Gilby hack, was mainly because I asked it to be, I think to date no one else uses that function in the hack bar me, but it is a very nice function to have.

Anyway, can not wait for you to get this option working again, THEN I will upgrade my boards.
I would have to disagree with you on this, I have been using this hack for quite some time under Vb2.x and use it only for the import of mailing lists. I've also used to to grab the archrive of the mailing lists from news groups that have done the archival. The one thing that is keeping me at VB2.x has been this ability...

Thanks for bringing this up to rev ...


Kevin

[Imhotep]

It seems you understand about firewall and TCP/IP, please explain what port is blocked and what is open, be it UDP or TCP protocol. How about an iptable/ipchain rule table for the firewall?

You receive and post news on the same port! I like to see the result of the telnet session from your web server to the nntp server's port 119. Do you know you can post news in the same session.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.

I am here trying to help you, you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.
From Lunarpages FAQ...

Do you support ssh / telnet access?

NO

Although the c-panel has an ssh/telnet section, it is disabled on all of our servers for security reasons.

please explain what port is blocked and what is open, be it UDP or TCP protocol.

My host has informed me that all outgoing traffic, be it UDP or TCP, on port 119 is blocked.

How about an iptable/ipchain rule table for the firewall?
What about it? Do I look like I am wearing the sysop hat? Do you really think they're going to setup special rules for me?

You receive and post news on the same port!
I am acutely aware of this fact.


I like to see the result of the telnet session from your web server to the nntp server's port 119.
see above.


Do you know you can post news in the same session.
Again, I am aware of this fact.

Now if they filter your tcp contents, then it is a different story, nothing to do with port blocking.
Which is why I am now wondering if they have a global filter on all outgoing NNTP packets.

you do not seems to understand my intention, rather, you think I gave you misguided information just for the fun of it. I shall not bother.

I understand your intention completely. I also understand that you might be as stubborn as a mule. My host writes to me and tells me all outgoing on port 119 has been blocked by their firewall. You seem bent on telling me it's wide open (both ways)... simply on the premise that I am receiving Usenet posts in my forum. IMHO, this is inherently wrong.

[Odoin]

I have not changed useragent... but I have changed organization but that was when I first set it up...

I added another newsgroup and still getting double posting... however the first newsgroup does not do this...

Any ideas?

Have you changed "useragent" or organization? The latest version checks both useragent and organization in the header. The message only get imported if they do not match at the same time.

Otherwise do you have special charactors in the organization setting?

[lierduh]

I have not changed useragent... but I have changed organization but that was when I first set it up...

I added another newsgroup and still getting double posting... however the first newsgroup does not do this...

Any ideas?

It is possible your news server changed your organization to a default one. Please check the header of the messages posted from your forums. You should be able to check with your normal news reader, failing that, use Google.

The original script only checks the useragent. The reason I added the organization check is if two forums are using the same news server with the same useagent, then the messages posted from the fellow forums will be skipped! I can add an option to disable organization check with the next version. Weekend is coming...:) In the meantime, find:

if ($message['user-agent'] == $settings['useragent'] AND
$message['organization'] == $settings['organization']) {

and replace it with:

if ($message['user-agent'] == $settings['useragent']) {

[Odoin]

Yes they change my organization to: 100ProofNews.com - Unlimited Downloads - $8.95/Month ARG....

So can I change the useragent to prevent skipping from another forum?

[lierduh]

Yes they change my organization to: 100ProofNews.com - Unlimited Downloads - $8.95/Month ARG....

So can I change the useragent to prevent skipping from another forum?

Yes, but first you need to change the code as indicated above. Gee, I can't believe they put advertising in your message header. I am sure that is against RFC1036!

[Odoin]

That worked :)... Thank you so much for the quick replys :).

[Odoin]

One other questions... Is there a way to change the display named on the who is online to something other then showing the gateway.php file being ran from a guest with the cron runs it?

[Highlander]

Hello :)

i just installed everything (uploaded files and tried making some settings)

But everytime i insert news server:
SERVER: msnews.microsoft.com
GROUP: microsoft.public.access

..for testing and calling gateway.php he want to donwload more than 30MB of data. should i let him finish the download?

Please help just one time till i check out how it works .. what settings , wich server and wich group with wich Last Message ID should i type in the settings?

i just need a more detailed guide for the first setup .. my forum Id ist 520 but nothing is imported there even if i downlaod 30Mb of data with gateway.php .. i just abort every time cause 30Mb of data wont fit never in the sql database.

What can i do and wich group/server i can use to try it out a little bit ..

greetings
Daniel

[lierduh]

Hello :)

But everytime i insert news server:
SERVER: msnews.microsoft.com
GROUP: microsoft.public.access

Please help just one time till i check out how it works .. what settings , wich server and wich group with wich Last Message ID should i type in the settings?
Daniel

Set Last Message ID to 73610. If there is anything, please cut and paste the screen dump.

[Highlander]

i set everything and the message id i set also .. than i called the setlastmsg.php and after it the gateway.php ..

but no messages appear and my test messages dont semms to be uploaded.. i will test just a little bit more ..

but for another try it maybe could be useful to copy your settings for testing (for ex. another news server with group)

..thx for you quick reply on this :D

[Highlander]

it is normal that hw want to downlaod with gateway,php more than 50MB ? (i aborted it after it reqached 50MB) .. <- its the buffering you described? .. i inserted the message id you gave me :)

[xb_]

hi

I see alot of discussion about the nntp / usenet features of the hack, while i primary use it for a mailing list archive, but not without problems (unfortunatelly).

There is this one guy on the list, whos' messages don't show up (i was thinking, he uses an encoding of some sort, which isn't supported by the hack), however they appear as an empty post, including a title and author. An example email of that specific author, can be found here (http://marc.theaimsgroup.com/?l=full-disclosure&m=108499271817045&q=raw).

When i don't manually edit the gateway.php to include a valid userid in the posts, the messages won't show up at all on the forumdisplay. I'm not entirelly sure what or why this is happening. (could be a permission thing in vB, as guest users don't have permissions in that specific forum)

And last, it appears that i'm having alot of 0 byte attachements; It are actually the only attachements i'm getting. So something is wrong with the insertion of those, in my setup.

in the admin page, theres some routine which creates a new table, if it thinks that the original nntp_groups doesn't excist. However this routine doesn't make notice of the TABLE_PREFIX variable, so if it is set, you'll end up with an additional empty table..


But besides the above, everything seems to work quiet alrite. Nice work so far, although i would like to see some more discussion about mailing list archive setups.

[Highlander]

whoops.. the gateway now reached 120MB download and downloading just more .. is this normally? i just entered the forelast message id (you gave me three posts before) from the group on the msserver! so it normally had to download only 2 topics there .. hmpf.. anyone know why gateway is downloading over 120MB of data and still not stopping .. ?

should i abot it now?

also posted one message and get this error:
1 group(s) gatewayed.
Connecting to server, server says: 200 NNTP Service 6.0.3790.1146 Version: 6.0.3790.1146 Posting Allowed
Info for microsoft.public.access at msnews.microsoft.com:
211 16354 2 73633 microsoft.public.access
Posting Message 'Beta Testing' from Highlander. Result: 441 (629) Article Rejected -- Ill-formed message id '<Highlander.16lqrp@>' in field 'Message-ID:'
Posting Message 'Beta Testing' from Highlander. Result: 441 (607) Article Rejected -- The 'References:' field requires one or more values

[lierduh]

should i abot it now?

also posted one message and get this error:

Yes, stop it.

What is your "Email address domain" "email" setting?

I am confused. You said before after you ran setlastmsg.php, you do not get any posts. Which is correct behaviour. Now you are getting 120MB download?

Please post your settings, and screen dumps. That will be better than any description. See, last post you included the error message, I know straight away need to ask what is your setting regarding "email".

[lierduh]

hi

When i don't manually edit the gateway.php to include a valid userid in the posts, the messages won't show up at all on the forumdisplay. I'm not entirelly sure what or why this is happening.
And last, it appears that i'm having alot of 0 byte attachements; It are actually the only attachements i'm getting. So something is wrong with the insertion of those, in my setup.

But besides the above, everything seems to work quiet alrite. Nice work so far, although i would like to see some more discussion about mailing list archive setups.

I will do up the mailing list function on the weekend. If possible, please post a sample raw message that you have problem with. I need the entire header, you can cut out the attachment, but leave the beginning and the end of the attachment.

[lierduh]

One other questions... Is there a way to change the display named on the who is online to something other then showing the gateway.php file being ran from a guest with the cron runs it?

Will need to hack into the vB scripts which I am trying to avoid as much as I can. There is no harm even someone clicks on the link though with the new version.

[lierduh]

I am getting this error whenever it hits a message with an attachment (with the latest script)

Is this related to creating the thumbnail?

What platform is your server based on? Windows? Linux?

Please email me the phpinfo() output. I will have a crack at it.

[xb_]

I editted the above post. As for the attachements, non of the attachements are getting through.
Looking in Mysql, the filename isn't included in the attachement table, nor is the filesize and filehash. The actuall filecontents of the attachements are there.

[lierduh]

I editted the above post. As for the attachements, non of the attachements are getting through.
Looking in Mysql, the filename isn't included in the attachement table, nor is the filesize and filehash. The actuall filecontents of the attachements are there.

Did you hack the script to work with mailing list? that probably explains it.:) Because I had to fix nntp.php and mime.php. I have not checked the pop3 class file.

What I need is the raw message from one of the problem list you have. You need to get it from a normal email reader.

[KevinM]

What platform is your server based on? Windows? Linux?

Please email me the phpinfo() output. I will have a crack at it.

I have sent a PM with the info on it (I couldn't get your email). It worked on your last version (except for the creation of the thumbnail). It is built on a redhat Linux system.

[xb_]

The only hacks/changes i made to it, were changeing the userid into a valid/excisting userid, and added TABLE_PREFIX to the table creation/check routine.
As for the message, i have received it from someone else's mailer (even tho i signed up with a seperate account for this :D ), but i suppose it is what you requested.

[Highlander]

lierduh i can give you (if you want check for bugs) administrtation account so you can look yourself.

I added an screenshot of the settings!
Also got this error message if i post an QUICK REPLY answer to an post in one of the USENET Forums:
1 group(s) gatewayed.
Connecting to server, server says: 200 NNTP Service 6.0.3790.1146 Version: 6.0.3790.1146 Posting Allowed
Info for microsoft.public.access at msnews.microsoft.com:
211 16354 2 73633 microsoft.public.access
Posting Message 'Beta Testing' from Highlander. Result: 441 (629) Article Rejected -- Ill-formed message id '<Highlander.16lqrp@>' in field 'Message-ID:'
Posting Message 'Beta Testing' from Highlander. Result: 441 (607) Article Rejected -- The 'References:' field requires one or more values
Sometimes i get no posts an only the message:


Connecting to server, server says: 200 NNTP Service 6.0.3790.1146 Version: 6.0.3790.1146 Posting Allowed
Info for microsoft.public.access at msnews.microsoft.com:
211 16354 2 73633 microsoft.public.access
and somteimes its downloading and downlaoding without stoop and any point of return :D .. my posts are aaways rejected :( .. and i did alwys set the lastmessage id before i use gateway.

greetings

[Highlander]

its me again :D
i just changed the emailsettings and now i get this when i set the ladt message id and try gatewy.php:
1 group(s) gatewayed.

Connecting to server, server says: 200 NNTP Service 6.0.3790.1146 Version: 6.0.3790.1146 Posting Allowed

Info for microsoft.public.access at msnews.microsoft.com:
211 16383 2 73657 microsoft.public.access

Posting Message 'Der Test' from Highlander. Result: 240 Article Posted OKso it seems its posted.. and now what to do now to receive the newsest answers otr new articles.. that i did not understood really ..

EDIT:
ok, now 10 minutes after i posted my post and searched in the newsgroup with outlook i found it :D yeah it wortked .. but than to receive new articles i call gateway.php thats the point when he downloads and dont stop anymore downloading any stuff but not alwys, receiving its nothing .. just my test posts are showing up in the group .. :(

[Highlander]

..still not working correctly.. now tried for many hours .. wont work .. maybe my php 4.0.6 or MySQL ist to old to work 100% with this script.

as i said i can post in the group (sometimes) and many times it downloads without stopping the gateway.php. I tried to finish a download but then i reached 400Mb and stopped it now .. something is crashing . the gateway.php dont want to download messages but sometimes upload my messages i wrote or other wrote in the subforum for the newsgroup.

making a break here and getting a cup coffee .. it should not be that the script works for me .. it would be damn good ;)

[lierduh]

..still not working correctly.. now tried for many hours .. wont work .. maybe my php 4.0.6 or MySQL ist to old to work 100% with this script.

as i said i can post in the group (sometimes) and many times it downloads without stopping the gateway.php. I tried to finish a download but then i reached 400Mb and stopped it now .. something is crashing . the gateway.php dont want to download messages but sometimes upload my messages i wrote or other wrote in the subforum for the newsgroup.

making a break here and getting a cup coffee .. it should not be that the script works for me .. it would be damn good ;)

Highlander,

You could not post because you did not set "email" in the settings table as you have found out. What is your newsgroup settings for the MS Access group?

KevinM,

Got your PM, it seems your php set up does not have a temp directory for php to use. Do you have a directory under your control that php can write temp files to it?

xb,

Thanks, the empty attachments are the PGP signature. I will deal with this in the new version. (I should have time to work on the script tonight, and hopefuly tomorrow.) Which version of nntp_groups.php do you use that you had the TABLE_PREFIX problem? please check the cvs version at the top of the script. I went through the latest script, but could not find any missed "TABLE_PREFIX". The default user ID is 0 for guest. What did the error say when you used 0 instead of another userid?

[xb_]

Thanks, the empty attachments are the PGP signature. I will deal with this in the new version. (I should have time to work on the script tonight, and hopefuly tomorrow.) Which version of nntp_groups.php do you use that you had the TABLE_PREFIX problem? please check the cvs version at the top of the script. I went through the latest script, but could not find any missed "TABLE_PREFIX". The default user ID is 0 for guest. What did the error say when you used 0 instead of another userid?


This is my version of nntp_groups.php:
$Id: nntp_groups.php,v 1.3 2004/05/17 09:14:31 ns Exp $
* xb_ hits himself on his head.
just now i see that there is already a newer version. 1.6. Wow +0.3 version updates in 2 days...
Thanks for resolving this :) and sorry for the inconvinience.


I don't seem to get an error, and the posts are inserted. The problem i had with uid 0 is that the posts don't get displayed on the forumdisplay.php script. My guess would be that this is because of the permission settings on that specific forum (guests have no access at all there, while the valid account has).
If i opened up the showthread.php script with parameters, i could see them just fine, but this could also have been due to my administrator access).
I also think that the posts are correctly inserted, only they would appear when i edit the userid into a valid one in phpmyadmin. I thought its only in the threads table, but not entirelly sure of this atm; i made all the uid 0s' a valid one in gateway.php, while i was at it.

While we thought about this over a few posts, i'll try setting guests allowing to post new threads in that forum, and allowing to reply, while i'll upload the newer version of your scripts, for testing purpose. I'll check up on it when i wake up :)

[Highlander]

Highlander,

You could not post because you did not set "email" in the settings table as you have found out. What is your newsgroup settings for the MS Access group?
the settings are attached in my previous post before with a jpg image .. i set the email and i am now able (but not every time i run gateway.php) to post in the newsgroup .. when i check it out with outlook i found my test articles in it .. my settings for MS Newsserver are following:

SERVER: msnews.microsoft.com
LAST MESSAGE: 16534 (10 articles still left)
GROUP: microsoft.public.de.german.spiele
USER/PASS: none setted (there is no need)

you see i changed to ms games group0 .. its the same .. every free group i tried has the same problem ..

in the attachemnt i attached an image of my settings ... (email is set the picture is older so dont take care of this :D !)

question:
can you give me another newsserver for testing? your swiftel server want to have user and pass , cant use them for a test.

whats the issue with downloding all the 100mbs? why is gateway an lastmessage.php everytimne saving themself? is this neccessary?

Hope you can help me making the thing work :D

[lierduh]

SERVER: msnews.microsoft.com
LAST MESSAGE: 16534 (10 articles still left)
GROUP: microsoft.public.de.german.spiele
USER/PASS: none setted (there is no need)


Stick to the "microsoft.public.access" group on the "msnews.microsoft.com" server. Set the lastmsg to 73800. Disable all the other groups, only enable this group!

What is on your screen if you run it manually. If you use IE and it tries to download instead of displaying it on the screen. Please fix the Windows registry following the instruction here:

http://support.microsoft.com/default.aspx?scid=kb;en-us;329661&Product=ie600

[xb_]

While we thought about this over a few posts, i'll try setting guests allowing to post new threads in that forum, and allowing to reply, while i'll upload the newer version of your scripts, for testing purpose. I'll check up on it when i wake up :) This went without result.


I checked the hacks i have inside forumdisplay.php, which is only a warning system (https://vborg.vbsupport.ru/showthread.php?t=59718). By removing this warning system (https://vborg.vbsupport.ru/showthread.php?t=59718), the problem was solved.

[KevinM]

Do you have a directory under your control that php can write temp files to it?


Lierduh,

It's my own leased server (although I'm stuffed when it comes to knowing how to configure it :ninja: ). I tried creating the directories and changing the path in the script but it had the same error. I then tried changing the v+ to w+ in the fopen statement and the script loops through the attachments without breaking, but produces the following error upon with each attachment:

Warning: imagecreatefromjpeg: '/tmp/temp.bin' is not a valid JPEG file in /home/httpd/vhosts/ukgardenforum.co.uk/httpdocs/includes/functions_image.php on line 117

The attachment is downloaded and inserted correctly for viewing, except for the thumbnail.

[Intex]

Can someone confirm this works on 3.0.1. Thx.

[Odoin]

Yes works very well on 3.0.1

[Highlander]

THX LIEDERUH:


its nots completely working, but it was an IE issue and with the registry fix i added manually in the registry the IE is now wirkoing corectly, but now i have an error message.

the site gateway.php is loading and messaging folling line:
<b>Warning</b>: Wrong parameter count for rtrim() in <b>/www/home93/html/board/includes/nntp.php</b> on line <b>132</b><br>

it seems that the RTIM is now a PHP 4.0.6 command??


any ideas .. its the code in uincludes/nntp.php at line 132
$line = rtrim(fgets($this->fp), "\r\n");

thank fpr the help, i think we are nearly on a working script :D

[Highlander]

okai i fixed it but maybe not really correctly :D

changed this lines:
$post = '';
while(!feof($this->fp)) {
//$line = trim(fgets($this->fp, 256));
//Took out the length, some lines are longer than that
$line = rtrim(fgets($this->fp), "\r\n");

to this lines:
$post = '';
while(!feof($this->fp)) {
$line = trim(fgets($this->fp, 256));
//Took out the length, some lines are longer than that
//$line = rtrim(fgets($this->fp), "\r\n");

if there is a better way fixing this LASt issue so tell me please, but with this canges now everything is working, the posts apear now and i can read and post ionto newsgroups .. and it writes the group messages into the forum!

many thanks to liederuh he is giving a excellent support here!!! really .. you are after logician the second one i would install hacks without reading :D

Questions

How to create a cron job, when i dont have shell, telnet access? Can i use the cron manager of the vb Software? and add the /vb-directory/gateway.php to it .. so it would call the gateway once hourly.
Why i cant call gateway.php for 30 minutes after it loaded out and imported successfully articles from the groups?
When i post with prefix in articels they wont appear as anser, they will show up as news articels, is there any fix?
thanks for the great help :D

[mcncyo]

i can answer #2 for you. It didn't finish going though all of the messages and time out. The reason why there is 30 mins is so two programs will not run at one time. When one program gets down it is supose to change the 1 to 0 in a table. which will let it run again.

[lierduh]

$line = rtrim(fgets($this->fp), "\r\n");[/php]
Questions

How to create a cron job, when i dont have shell, telnet access? Can i use the cron manager of the vb Software? and add the /vb-directory/gateway.php to it .. so it would call the gateway once hourly.
Why i cant call gateway.php for 30 minutes after it loaded out and imported successfully articles from the groups?
When i post with prefix in articels they wont appear as anser, they will show up as news articels, is there any fix?
thanks for the great help :D

Download the new version, it should fix the rtrim problem, it had problem working with an older version of php.

1) Last time I checked, you can just include the gateway.php in the vB cron job. It will be run every hour.
2) The only time you have this problem is the script crashed. This is to prevent a race condition from more than one running script.
3)You don't want to add prefix to the posts, otherwise, they will not be threaded properly.

[lierduh]

Lierduh,

It's my own leased server (although I'm stuffed when it comes to knowing how to configure it :ninja: ). I tried creating the directories and changing the path in the script but it had the same error. I then tried changing the v+ to w+ in the fopen statement and the script loops through the attachments without breaking, but produces the following error upon with each attachment:
The attachment is downloaded and inserted correctly for viewing, except for the thumbnail.

Please try the new v2.1.0. It may solve your problem.

[chas_h]

Bravo Lierduh! I am very impressed with how far you have taken this in the last few weeks and very happy that you have decided to continue developement.

Thank you!

[Erwin]

The new versions - can you explain in a post here if people need to upgrade, or are they only for people who want mailing list support?

[lierduh]

The new versions - can you explain in a post here if people need to upgrade, or are they only for people who want mailing list support?

Erwin, I would say use the latest one. Other than mailinst list support. I have improved handling for mime encoded messages, even if you do not use binary group (not really designed as a binary gateway I must say). People use mime encoding for PGP signatures and other purposes. There are a few new options available now in the setting table. I am not sure which version are you using, but chances are there were bugs fixed with the newer version. I had a quick browse through my cvs logs, there are quick a few areas changed since last release. It is not urgent though to upgrade, this release does not contain major bug fixes. Naturally, there might be new bugs created.:) However, the upgrade only needs you to log in to the AdminCP and open the pages, other than uploading the files.:)

[Odoin]

Good idea, Allan... I am going to look into it.

thanks!
I did not see a answer to my post... How can I get this to show up as something other then:

Unknown Location
/forum/gateway.php?log=cron

In the user lists with CRON runs the gateway?

[Mike11212]

Hi

I started to install the script when my server went down and only some of the tables for the database were created. When I run the settings I get a MYSQL Error.

Can someone tell me what Tables were created so I can delete them manually and reinstall it.

[Odoin]

This should be the tables in a SQL file for you :).
Hi

I started to install the script when my server went down and only some of the tables for the database were created. When I run the settings I get a MYSQL Error.

Can someone tell me what Tables were created so I can delete them manually and reinstall it.

[Mike11212]

Thanks so much odion. I tried looking in the files for it and just now saw them in the nntp settings file but with the file you supplied I won't have to look for it.

[Highlander]

liederuh you forgot the nntp.sql in the new version ;)

and thx again for your help ..

The Cronjob i do is onlyy the path to gateway in vb cron added?

got another error, maybe you know something about this :D .. i hope i am creating not to much errors and asking you holes in the stomach :D
<b>Warning</b>: open_basedir restriction in effect. File is in wrong directory in <b>/www/home93/html/board/gateway.php</b> on line <b>970</b><br>
<br>
<b>Warning</b>: fopen("/tmp/temp.bin","x+") - Operation not permitted in <b>/www/home93/html/board/gateway.php</b> on line <b>970</b><br>
failed create a temp file

this error appears if an article in the group has many answers (over 30) or i porting more than 100 articles. It seems he wants to buffer the imported data into a file? but it also seems if i would chmod gateway.php to 777 than he would overwrite it with the buffered data, right?! so the gateway.php would be overwritten and unuseable for the nexttime, but anyway, if i start it again with the next message id it will import correctly , so that means the last message id creates this error ..

.. BUt hey its a great script .. i really like it is very useful!

Questions
1. Do you know where i can find a list of the newsgroup servers? Only search engines or similar sites wich posts the articles itself i found, the nntp servers are really under heavy secret .. nobody wants to share this information. I searched google for days now and i only found some open, but not very good, newsgroup adresses.
Where i can find the Adresses for the nntp servers?

2. The "Guests" who wrotes the usenet articles in the specified Forum i choose is signed as guest, i want to know if its not possible to integrate it in the script while it imports the articles to sign the "guest" as "usenet Guest" it would look much better :D .. i will crawl trough the code :D maybe i can answer this myself!


This is minimum the hack of the month, better would sound the hack of the YEAR 2004!

greetings from germany
Daniel (Highlander)

Want take a look of the imported articles:
Usenet Forums with the imported articles (http://spielplanet2.de/board/forumdisplay.php?f=519)

[Odoin]

No... it makes the tables for you when you run the script from the admin panel the first time...

[Erwin]

Erwin, I would say use the latest one. Other than mailinst list support. I have improved handling for mime encoded messages, even if you do not use binary group (not really designed as a binary gateway I must say). People use mime encoding for PGP signatures and other purposes. There are a few new options available now in the setting table. I am not sure which version are you using, but chances are there were bugs fixed with the newer version. I had a quick browse through my cvs logs, there are quick a few areas changed since last release. It is not urgent though to upgrade, this release does not contain major bug fixes. Naturally, there might be new bugs created.:) However, the upgrade only needs you to log in to the AdminCP and open the pages, other than uploading the files.:)
Alright, I'll install the latest version. :)

[lierduh]

I did not see a answer to my post... How can I get this to show up as something other then:

Unknown Location
/forum/gateway.php?log=cron

In the user lists with CRON runs the gateway?

It needs to hack into "who on line" script. I don't see the immediate need for this. Besides one less php script to touch is always a good thing.

[lierduh]

Hi

I started to install the script when my server went down and only some of the tables for the database were created. When I run the settings I get a MYSQL Error.

Can someone tell me what Tables were created so I can delete them manually and reinstall it.

The best way to do when this happens is to delete both nntp_groups and nntp_settings tables. Open the AdminCP interfaces. These tables will be recreated by the AdminCP.

[lierduh]

liederuh you forgot the nntp.sql in the new version ;)

and thx again for your help ..

The Cronjob i do is onlyy the path to gateway in vb cron added?

got another error, maybe you know something about this :D .. i hope i am creating not to much errors and asking you holes in the stomach :D


You are using an older version of the script. Get the new one! If you click the "Install" button on top right of this page. You will receive update/bug fix emails from me for this hack.

Just enter gateway.php without the cron bit into your vB cron job list.

From next version, the default calling of gateway.php will not involve the debugging messages displayed on the screen. If you need to see those messages, then one will need to call the script as .../gateway.php?debug=1.

I doubt many people is using a free news server. Although sometimes you can find misconfiged news servers wide open, it is not guaranteed to work the next day. Subscribe to one of cheapie news server for a couple of bucks/Euro/month is the best way to go. I have seen some need only one time registration with small fee to pay. Or change the host to one who lets you access to a news server.

[lierduh]

This should be the tables in a SQL file for you :).

There is no need for this file with the newer version. If you use this old file, extra empty no use fields will be created!

[Highlander]

Der Hijacker ist COOLWEBSEARCH <BR>Download Hijack&nbsp; This&nbsp; http://www.majorgeeks.com/downloads31.html<BR>Poste dein Hijack This Logfile,in einer der Folgenden ?Expert Forums? <BR>http://board.protecus.de/ <BR>http://www.trojaner-board.de/ <BR><BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; ----- Highlander wrote: -----<BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; also ichd enke nicht dass ihm die group suche weiterhilft :D .. er hat<BR>&nbsp;&nbsp;&nbsp;&nbsp; einen wurm und den entfdernt er am besten mit einer anti wurm kur :D ..<BR>&nbsp;&nbsp;&nbsp;&nbsp; noroton oder andere gute Virenentferner!<BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; -- <BR>&nbsp;&nbsp;&nbsp;&nbsp; Highlander<BR> okai thx .. what about the posting .. i mean exporting to a newsgroup. a user told me that when i post a horroble wide range of unsorted html tags showed up, so that my postings there messed up the whole style .. nearly unreadable with some vb-tags and many html tags.

i disabled html in all my forums and the img code .. should i disable also the vb and smiley code in this forums to export posts on the right way?

here an example:

Der Hijacker ist COOLWEBSEARCH <BR>Download Hijack&nbsp; This&nbsp; http://www.majorgeeks.com/downloads31.html<BR>Poste dein Hijack This Logfile,in einer der Folgenden ?Expert Forums? <BR>http://board.protecus.de/ <BR>http://www.trojaner-board.de/ <BR><BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; ----- Highlander wrote: -----<BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; also ichd enke nicht dass ihm die group suche weiterhilft :D .. er hat<BR>&nbsp;&nbsp;&nbsp;&nbsp; einen wurm und den entfdernt er am besten mit einer anti wurm kur :D ..<BR>&nbsp;&nbsp;&nbsp;&nbsp; noroton oder andere gute Virenentferner!<BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; <BR>&nbsp;&nbsp;&nbsp;&nbsp; -- <BR>&nbsp;&nbsp;&nbsp;&nbsp; Highlander<BR>

[lierduh]

okai thx .. what about the posting .. i mean exporting to a newsgroup. a user told me that when i post a horroble wide range of unsorted html tags showed up, so that my postings there messed up the whole style .. nearly unreadable with some vb-tags and many html tags.


I have no idea what you have done.:) No, html should not show up. Check your post in your post table. If you see html, then the problem is nothing to do with gatway.

[Erwin]

lierduh - I like the way you coded the AdminCP files. :) Very smart.

[Mike11212]

When I run the gateway.php file. It pulls the first 2 post but then it just hangs and does nothing else.

Have I done something wrong?

[webrats]

When I run the gateway.php file. It pulls the first 2 post but then it just hangs and does nothing else.

Have I done something wrong?
ditto how long should i wait?

[lierduh]

When I run the gateway.php file. It pulls the first 2 post but then it just hangs and does nothing else.

Have I done something wrong?

More information will help. What is your lastmsg ID for this group? Have you follow the readme file to find out the last message ID for the group? how many groups do you have? What is displayed on the screen? Is there any errors in your web log?...

[lierduh]

lierduh - I like the way you coded the AdminCP files. :) Very smart.

Thanks Erwin, that is probably the best one can expect from a fellow coder.:)

[BustaCap]

Two questions:

1. How does the mailing list archive look when it's viewed? Does it look just like how the usenet gateway looks? I'm thinking of archiving a yahoo mailing list and was wondering if it would look like how the usenet posts look (ie, threaded posts). If you reply to a mailing list post, will it be sent to everyone else on the mailing list?

2. I tested the usenet gateway function and it works well on my 3.01 board. However, when I was importing the posts from a group (about 1,000 posts), the reported serverload in my vB CP went through the roof. Before importing they were at 18, 19, 22. During the import they went up to 155, 140, 55. I can't imagine my host is gonna appreciate that since I'm using a shared server. Is there some way you could reduce the load on the server during imports?

[Princeton]

excellent hack :D

[lierduh]

Two questions:

1. How does the mailing list archive look when it's viewed? Does it look just like how the usenet gateway looks? I'm thinking of archiving a yahoo mailing list and was wondering if it would look like how the usenet posts look (ie, threaded posts). If you reply to a mailing list post, will it be sent to everyone else on the mailing list?

2. I tested the usenet gateway function and it works well on my 3.01 board. However, when I was importing the posts from a group (about 1,000 posts), the reported serverload in my vB CP went through the roof. Before importing they were at 18, 19, 22. During the import they went up to 155, 140, 55. I can't imagine my host is gonna appreciate that since I'm using a shared server. Is there some way you could reduce the load on the server during imports?

1. It will look just like the Usenet messages, threaded. When you make a post to the mailing list forum, it effectively sends an email to the mailing list as your group setting. If the list accepts posts from anyone, you are on.

One thing I will add into the next version is: right now, the "From: " email address is the same format as the USENET ones, ie. it does not contain the real email address due to spam prevention. There should be an option for the mailing list forums to send real email address. Some mailing lists only accept subscribed member's posts.

2. There are many things the script needs to do when importing a message. I doubt there will be much room to improve. You normally do not experience the high server load, because no one is doing the continuing posts one after another for 1000 times. For a start, a server load of 18 is already too high. I am surprised it did not fall apart.:) You can always only import new messages as per readme.

[Princeton]

anyone has a good USENET list? or, could direct me to one

thanks

[DaveLogic]

Just upgraded from the previous version (which worked fine) and installed perfectly so many thanks....What would be a great improvement would be be this IMO...

1 - A random default avatar for guest NNTP posts - Would tie in with forum based posts better and look better.

2 - Use the same quotation code as in the forum...i.e the tidy box that surrounds quotes normally within forum posts...I don't know how this would transfer back to the newsgroups however....??? Anyway, the Admin cp section really helps...really nice stuff...

[Odoin]

Now keep in mind not every servers handles the same newsgroups... but here is a list I go by...

http://205.215.62.111/groups-list.txt (2mb in size).

[Odoin]

I like the avatar idea... The quotes might be sorda hard because of the different ways the news clients quote messages... maybe a default for the standard ones could be built in?

Mark
Just upgraded from the previous version (which worked fine) and installed perfectly so many thanks....What would be a great improvement would be be this IMO...

1 - A random default avatar for guest NNTP posts - Would tie in with forum based posts better and look better.

2 - Use the same quotation code as in the forum...i.e the tidy box that surrounds quotes normally within forum posts...I don't know how this would transfer back to the newsgroups however....??? Anyway, the Admin cp section really helps...really nice stuff...

[DaveLogic]

I like the avatar idea... The quotes might be sorda hard because of the different ways the news clients quote messages... maybe a default for the standard ones could be built in?

Mark

My Newsgroup stuff gets a tad ignored because all of the NNTP guest post members don't/can't have an avatar which sort of gives the game away...I would imagine that this would better be placed as a "generic" hack as even it it were a static guest avatar it woud be a plausable idea being that boards could use a site logo as default...???

[lierduh]

Just upgraded from the previous version (which worked fine) and installed perfectly so many thanks....What would be a great improvement would be be this IMO...

1 - A random default avatar for guest NNTP posts - Would tie in with forum based posts better and look better.

2 - Use the same quotation code as in the forum...i.e the tidy box that surrounds quotes normally within forum posts...I don't know how this would transfer back to the newsgroups however....??? Anyway, the Admin cp section really helps...really nice stuff...

1. I am not sure it will work, but if someone wants to try it out. Add a user with the user ID of 0. And add avatars etc for the user 0.

2. What really matters is what contents is ended up on Usenet. If the normal usenet users see BB tags etc, they will start to hate your forums. So a text based message will be more acceptable. Even the script gets rid of the BB tags before sending them, there are always users who miss one "[" or "]" and stuff up the formating.:)

[Princeton]

thanks for the list

SUGGESTION on next update
Create a script (ala cron job) to remove certain post(s) (by subject) that contains "porn", "$", "buy", etc, etc...

[KevinM]

I've been playing with getting the email notifications to work with this script. If you find

// send out email notices

in gateway.php and after it put:

exec_send_notification($threadid, $userid, $postid);

It seems to work (well, it did the few times I tested it :speechless: )
Let me know if it works for you.

[Imhotep]

thanks for the list

SUGGESTION on next update
Create a script (ala cron job) to remove certain post(s) (by subject) that contains "porn", "$", "buy", etc, etc...



After a long battle with my host and finding a good NSP, I finally got this hack to work but only through port 23... which is fine with me as long as it works.

I would like to add to what Princeton has suggested. We absolutely need a filter system to screen out not only offensive headers but also attachments. One of my subscribed groups is a binary group. (alt.binaries.3d.lightwave) Lately, I have been deleting virii that come through and plant themselves as simple attachments. The main type has been the .scr variety. I would hate to see one of my members click on one of these attachments before I can find it and del it. Perhaps we can intergrate the attachment file extension permissions from VB into this hack, no?

www.cghaus.com

[KevinM]

The main type has been the .scr variety. I would hate to see one of my members click on one of these attachments before I can find it and del it. Perhaps we can intergrate the attachment file extension permissions from VB into this hack, no?

I have been writing that today (coincidence). I am just testing it, and will post the instructions tomorrow. It checks the extension type aginst what has been allowed in the vb control panel.

[KevinM]

These code changes use vb's attachment permissions (extensions only) before inserting attachments into the database - so no more .scr files :squareeyed:

If the file extension is not set up in the 'Attachment Manager' 'Extensions and Sizes' section of the control panel then the attachment is ignored.

I have tested it on version 2.1.0. All changes are to 'gateway.php'. There are 6 additions or changes to the script to be made to 'gateway.php'.

Find

require_once("global.php");

CHANGE it to:

$specialtemplates = array(
'attachmentcache'
);
require_once("global.php");
$attachtypes = unserialize($datastore['attachmentcache']);

Find:

function process_attachments ($date, $postid, $threadid) {
global $message, $DB_site;

AFTER it put:

global $attachtypes;

Find:

$attachcount = 0;

AFTER it put:

$attaches = 0;

Find:

for ($i = 1; $i <= $message['attachments']; $i++) {

AFTER it put:

// now we check the file extension against the vb settings to check that it exists, and if it does, check it is enabled.
$attachment_name2 = strtolower($message['attachment' . $i]['headers']['filename']);
$extension = file_extension($attachment_name2);

if (!$attachtypes["$extension"] OR !$attachtypes["$extension"]['enabled'])
{
// invalid extension
logging($extension . " extensions are not accepted, as currently set up in the control panel \n");
}
else {

Find:

fclose($handle);
unlink($tempfile);


CHANGE to:

$attaches++;
fclose($handle);
unlink($tempfile);
}

Find:

$DB_site->query("UPDATE " . TABLE_PREFIX . "post
SET attach = attach + " . $attachcount . " WHERE postid = $postid");
$DB_site->query("UPDATE " . TABLE_PREFIX . "thread
SET attach = attach + " . $attachcount . " WHERE threadid = $threadid");
logging("Inserted " . $attachcount . " attachment(s).");
}


CHANGE to:

$DB_site->query("UPDATE " . TABLE_PREFIX . "post
SET attach = attach + " . $attaches . " WHERE postid = $postid");
$DB_site->query("UPDATE " . TABLE_PREFIX . "thread
SET attach = attach + " . $attaches . " WHERE threadid = $threadid");
logging("Inserted " . $attaches . " attachment(s).");

And that is it :nervous:

This script doesn't check against the maximum file size or image dimensions permissions, but it would be fairly easy to add on however I'm not sure of how useful this would be, as people posting to newsgroups would be unaware of your boards limits.

[Imhotep]

AWESOME!!! Thanks Kevin! Your hard work is appreciated. This will go along way towards making this hack a little bit more secure and BB friendly.

[Imhotep]

Hmmm.... it seems I am now getting an error. I'm not sure if this is related to the new attachment perms or not.

Content-type: text/html
X-Powered-By: PHP/4.3.4

<br />
<b>Parse error</b>: parse error in
<b>/home/cghaus2/public_html/forums/gateway.php</b> on line <b>1079</b><br />

I also upgraded to the new version this morning.

[KevinM]

Hmmm.... it seems I am now getting an error. I'm not sure if this is related to the new attachment perms or not.

Maybe I made a mistake writing the instructions. I have attached a version of gateway.php which is working for me with the attachment permissions.

Try this.

[Princeton]

thank you Kevin ... no problems on my end

Imhotep,
ReCheck your code ... you probably made a mistake copying / pasting

[Imhotep]

Maybe I made a mistake writing the instructions. I have attached a version of gateway.php which is working for me with the attachment permissions.

Try this.


Kevin, thanks, this file fixed the problem. I am usually very careful about how I copy/paste, so I am not sure if I made a mistake or not. I suppose if princeton got it working, then I probably fulbbed somewhere along the line.

thanks again

[BustaCap]

In the NNTP Gateway setting, under the "Email address domain" option, I'm not quite sure what to put there. What do you mean by To avoid spams attempting at your server. Please do not use a resolvable domain.

Do I need to change the default value?

Also, for the "Mailing list E-mail" setting, do I put the email address of the mailing list (eg, coolgroups@yahoogroups.com) or is it the email address that will be sending messages to the mailling list (eg, me@mydomain.com). The yahoo mailing list that I want to archive only accepts emails from verified email addressess. Is that going to be a problem?

Finally, two suggestions:

1. For mailing lists, all emails will have a common word or phrase in the subject. For example, the subject of all messages for the "New DVD Releases" mailing list would look something like:

[NewDVD] Spider Man deluxe edition
RE: [NewDVD] Spider Man deluxe edition
etc...

It would be great if you could add a setting in the gateway setup that would allow the user to specify the subject phrase (eg, [NewDVD]) and then strip that from the subject text so that it doensn't show up in the title of every post. Also, even more importantly, have a check box that would allow us to only import message that has the subject phrase. This would allow us to filter out messages that are not actually mailing list messages.

2. Every mailing list message that I get has the same text in the footer. It would be great if there was a setting somehwere where I could specify this footer text (its over 13 lines!) and have it stripped before importing the message.

Thanks, this hack is really useful for me!

[KevinM]

I've been thinking about attempting a 'killfile' feature (a common newsreader feature) which would allow the administrator to set some parameters and stop the downloading of certain posts into the forum.

I propose it works like this:

In the admin control panel, you enter the phrases (surrounded by {} ) which you would not want downloaded (similar to the censorship feature).

As each post is processed, it checks the headers, author and title of a message to see if it matches with the phrases, and if it does, discard it.

Possible uses:
You could stop a specific troll's posts from reaching your boards by putting the posters name into the killfile (although any replies by other usenet users will get through).

You could block messages originating from a specific server (for trolls that keep changing their posting name), although you risk blocking anyone else using the same server (small risk).

You could set up common 'spam' message titles to block.

Downsides:
Performance. Putting dozens of words or phrases into the killfile will slow down the processing of messages while the gateway script is being run.

Pureists. Some people using the newsgroups may consider this a form of moderation in an unmoderated group.

I'd really appreciate others users of this scripts thoughts.

Is this worth pursuing and is it the right approach?

[BustaCap]

Sounds good. Would be great if it worked the other way as well; only posts with a certain word(s) in the title would be posted to the board (eg, see my request #1 in my above post).

[KevinM]

Sounds good. Would be great if it worked the other way as well; only posts with a certain word(s) in the title would be posted to the board (eg, see my request #1 in my above post).

This could work for mailing lists, but not newsgroups. Also, each mailing list would require a different keyword to be matched in the title.

I agree about the footer. Again you would need to accomodate multiple different footers (for each mailing list, or for groups like yahoogroups which regularly change their footers from day to day). Any risk to possible violations of the terms and conditions from the list operators?

[BustaCap]

Yes, I was referring to mailings lists. Ideally, I would like to see a form for every mailing list, which would be required for any "catch phrases".

At the moment none of the posts that I get for my mailings lists are being threaded; they are all only one post deep and each reply shows up as a seperate post. It looks the "RE:" portion of the subject title might be causing problems since it can show up in several different spots in the title. For example, for the original thread, [NewDVD] Spider Man deluxe edition, the replies show up as :

RE: [NewDVD] Spider Man deluxe edition
[NewDVD] RE: Spider Man deluxe edition
RE: [NewDVD] RE: Spider Man deluxe edition

I'm guessing it's just how the various mail programs that people use handle inserting "RE". Looking at several hundred messages, the best way to determine which reply goes with which message would be to look at the title starting from the end of the message and working backwards. You don't usually see the [xxx] or the RE: part at the end of the subject. A quick browse through about 300 messages showed that if I used the last three or four words of the subject line it would have accurately put the replies with the correct original thread.

[scottct1]

I am using the old version of gateway.php (The one that worked with the Beta versions of VB3 where you had to edit the database by hand to add newsgroups)

I am thinking of upgrading to this version. :)

Do I need to do anything to make this version work, such as

Delete all current Newsgroup forums? (To prevent messages in these forums from being sent back out to the Internet)
Do I need to delete any of the old database enteries?

Anything else I need to do to upgrade from the old version?

Thanks for your help and for what appears to be a great hack!

[Odoin]

Not 100% sure but I do not think there has been any DB changes... I went from the edit by hand DB to the new version with no issues.

[scottct1]

Great did you have to empty your existing newsgroup forums? Or was it fine?

I just dont want to send all the messages I have now to be sent back out to usenet. :)

[allan grossman]

I''ve been using this since vb 2.30 - upgraded to vb3 and then 3.01. Have upgraded the script several times and have *never* had to empty existing newsgroup forums. The script just works :)

[Odoin]

Nope... did not have to empty... The major changes are just the way the gateway works and pulls the new messages as well as the admin i face now.

Great did you have to empty your existing newsgroup forums? Or was it fine?

I just dont want to send all the messages I have now to be sent back out to usenet. :)

[KevinM]

Every mailing list message that I get has the same text in the footer. It would be great if there was a setting somehwere where I could specify this footer text (its over 13 lines!) and have it stripped before importing the message.

I am having a look at this now, however I don't have any mailing lists which makes it difficult to test.

The way I have it working is, the administrator enters the characters or phrases that starts the footer message into a setting in the cpanel (not the whole footer, just enough that it is 'unique' and is not likely to form part of a genuine message).

You can enter as many different footers as required.

Every mail message gets checked against these, and if there is a match, strips everything from that point onwards from the message. Usenet messages are ignored.

The only situation where this wouldn't work is where a footer is quoted in someone elses reply, and they then bottom post their message beneath this. From memory of using mailing lists I don't recall this happening often, but I need a guide from you. (It would be ton easier to not have to worry about this :nervous: .)

Let me know if this approach will work for your lists, and will you be able to test the script?

[BustaCap]

KevinM, that is exactly the kind of solution I was hoping for :) The footers in the mailing lists I use alwasy start with the same text and all text after that can be ignored. I'm not too concerned about the quoted footer text myself, so whatever would be easier for you. I would be happy to test it.

Lierduh, are you still supporting this hack? I haven't seen you comment in over a week so I was just wondering.

[KevinM]

What this does:

If you are using this hack to mirror mailing lists you will often find that the mailing list operator adds their own footer to each message which can be quite large.

This function will strip this from the message, prior to insertion into the forum. It can handle multiple different footers (just surround each one with { and }).

I have tested this on a test forum, but I am looking for feedback from a live enviroment.

[BustaCap]

I'll give this a go, thanks!

[BustaCap]

It worked just as I had hoped it would, thanks for taking the time to write that. :D Messages look much less cluttered now.

One issue that I'm still having is that the vast majority of the mailing list posts are not being threaded and are showing up as individual posts. I thought it may have been an issue of the title changing when people reply and it adds an "RE:" to various places in the subject (see my previous post #140), but after looking at it more closely, posts with the exact same subject are showing up as seperate threads:

http://orthotechforum.com/images/mlist.gif

I can't figure out why some (very few) posts are threaded but others with the same subject are not. Admittedly, I know zero about coding, but shouldn't it be pretty straightforward to determine which replies should be threaded based on the subject title? I'm wondering if it might have something to do with the "Use Post Time" setting. The default is set to use the time stamp when the messages are imported, which means that all of the posts at the time of import have the same time stamp. I changed it to use the original post time, thinking that posts with the same title and same timestamp might somehow be throwing off the threading. I'll test this and see how it works. Any idea though why the posts are threading correctly?

[KevinM]

One issue that I'm still having is that the vast majority of the mailing list posts are not being threaded and are showing up as individual posts.

Yes, this has been mentioned before. I remember from the early days of this hack when I used it to mirror a yahoo group, I spent half my time manually merging threads. It drove me mad.

As far as I can tell, the script doesn't rely on matching thread titles at all, but by references in the headers (which works great for usenet, which is built on this standard).

If there isn't a reference or a match then a new thread is started. I should think it is possible to do a secondary check against a thread title before starting a new thread.

I will have a look and see how difficult this would be to implement. I am no expert at php - I just look at other peoples scripts and figure it out (sometimes!).

[BustaCap]

That would be great, let me know how you make out.

[lierduh]

Finally, two suggestions:

1. For mailing lists, all emails will have a common word or phrase in the subject. For example, the subject of all messages for the "New DVD Releases" mailing list would look something like:

[NewDVD] Spider Man deluxe edition
RE: [NewDVD] Spider Man deluxe edition
etc...

It would be great if you could add a setting in the gateway setup that would allow the user to specify the subject phrase (eg, [NewDVD]) and then strip that from the subject text so that it doensn't show up in the title of every post. Also, even more importantly, have a check box that would allow us to only import message that has the subject phrase. This would allow us to filter out messages that are not actually mailing list messages.

2. Every mailing list message that I get has the same text in the footer. It would be great if there was a setting somehwere where I could specify this footer text (its over 13 lines!) and have it stripped before importing the message.

Thanks, this hack is really useful for me!

1. Doesn't mailing lists add prefix to any messages posted to it? The two type of lists I have admined do. they are sympa and mailman.

2. I consider it rude to strip the footers.:) The lists attach them for a purpose, either advertising or rules. They should be presented as they are received in an email mail box. If the list adds 13 lines of footer to the messages, perhaps the subscribers can complain to the list owner to have it shrinked down.

[lierduh]

I can't figure out why some (very few) posts are threaded but others with the same subject are not. Admittedly, I know zero about coding, but shouldn't it be pretty straightforward to determine which replies should be threaded based on the subject title? I'm wondering if it might have something to do with the "Use Post Time" setting. The default is set to use the time stamp when the messages are imported, which means that all of the posts at the time of import have the same time stamp. I changed it to use the original post time, thinking that posts with the same title and same timestamp might somehow be throwing off the threading. I'll test this and see how it works. Any idea though why the posts are threading correctly?

The threading works by checking references in the headers. All the email clients I have used set this field in the header correctly. Although I have not checked, but I am pretty sure that is the requirement set out by the RFC for the email clients. Could you please post the headers of the messages that are not threaded correctly?

[lierduh]

Lierduh, are you still supporting this hack? I haven't seen you comment in over a week so I was just wondering.

I was just out of town for a week.:)

[lierduh]

I am using the old version of gateway.php (The one that worked with the Beta versions of VB3 where you had to edit the database by hand to add newsgroups)

I am thinking of upgrading to this version. :)

Do I need to do anything to make this version work, such as



If anyone is using the (beta) versioin from the early days. There is nothing needs to be done. The AdminCP will do the rest for you when you fire them up the first time.

The AdminCP interfaces are also: Installation script, upgrade script. It is designed in a way not to slow down the normal operation when the version is already up to date.

[BustaCap]

1. Doesn't mailing lists add prefix to any messages posted to it? The two type of lists I have admined do. they are sympa and mailman.

2. I consider it rude to strip the footers.:) The lists attach them for a purpose, either advertising or rules. They should be presented as they are received in an email mail box. If the list adds 13 lines of footer to the messages, perhaps the subscribers can complain to the list owner to have it shrinked down.

1. I'm not sure. I'll post the header though.

2. At the moment I am archiving the group just for my own purposes and the forum is hidden to everyone but myself. If I ever made it public I would likely add the footers back in.

Here are four messages (from a Yahoo group) with the exact same subject line that were not threaded:

To: GAdetection@yahoogroups.com
Message-ID: <ca0tlh+r84f@eGroups.com>
In-Reply-To: <ca0scn+85e3@eGroups.com>
User-Agent: eGroups-EW/0.82
X-Mailer: Yahoo Groups Message Poster
X-eGroups-Remote-IP: 66.218.66.74
From: "User 1" <U1@site.com>
X-Originating-IP: XXX.XX.XX.X
X-Yahoo-Profile: User 1
MIME-Version: 1.0
Mailing-List: list GAdetection@yahoogroups.com; contact GAdetection-owner@yahoogroups.com
Delivered-To: mailing list GAdetection@yahoogroups.com
Precedence: bulk
List-Unsubscribe: <mailto:GAdetection-unsubscribe@yahoogroups.com>
Date: Mon, 07 Jun 2004 05:15:29 -0000
Subject: [GAdetection] Re:Infallible detectives
Reply-To: GAdetection@yahoogroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

To: GAdetection@yahoogroups.com
Message-ID: <ca0tp9+vv2t@eGroups.com>
In-Reply-To: <20040607005707.68431.qmail@web51706.mail.yahoo.com>
User-Agent: eGroups-EW/0.82
X-Mailer: Yahoo Groups Message Poster
X-eGroups-Remote-IP: 66.218.66.68
From: "User 2" <U2@web.net>
X-Originating-IP: XXX.XX.XX.X
X-Yahoo-Profile: User 2
MIME-Version: 1.0
Mailing-List: list GAdetection@yahoogroups.com; contact GAdetection-owner@yahoogroups.com
Delivered-To: mailing list GAdetection@yahoogroups.com
Precedence: bulk
List-Unsubscribe: <mailto:GAdetection-unsubscribe@yahoogroups.com>
Date: Mon, 07 Jun 2004 05:17:29 -0000
Subject: [GAdetection] Re: Infallible detectives
Reply-To: GAdetection@yahoogroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

To: GAdetection@yahoogroups.com
X-Mailer: AOL 3.0 16-bit for Windows sub 41
X-eGroups-Remote-IP: 64.12.137.5
From: User3@aol.com
X-Yahoo-Profile: User3
MIME-Version: 1.0
Mailing-List: list GAdetection@yahoogroups.com; contact GAdetection-owner@yahoogroups.com
Delivered-To: mailing list GAdetection@yahoogroups.com
Precedence: bulk
List-Unsubscribe: <mailto:GAdetection-unsubscribe@yahoogroups.com>
Date: Sun, 6 Jun 2004 18:11:59 EDT
Subject: [GAdetection] Re:Infallible detectives
Reply-To: GAdetection@yahoogroups.com
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

To: GAdetection@yahoogroups.com
Message-ID: <ca056l+dh9f@eGroups.com>
In-Reply-To: <1d6.23335dec.2df4f0af@aol.com>
User-Agent: eGroups-EW/0.82
X-Mailer: Yahoo Groups Message Poster
X-eGroups-Remote-IP: 66.218.66.105
From: "User4" <U4@site.fr>
X-Originating-IP: xxx.xx.xx.x
X-Yahoo-Profile: User4
MIME-Version: 1.0
Mailing-List: list GAdetection@yahoogroups.com; contact GAdetection-owner@yahoogroups.com
Delivered-To: mailing list GAdetection@yahoogroups.com
Precedence: bulk
List-Unsubscribe: <mailto:GAdetection-unsubscribe@yahoogroups.com>
Date: Sun, 06 Jun 2004 22:17:57 -0000
Subject: [GAdetection] Re:Infallible detectives
Reply-To: GAdetection@yahoogroups.com
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit

I notice the 3rd one didn't even have a message ID or a "In reply to". That's not a typo, I wonder if it has something to do with his AOL client. None of the Message-ID's or In-Reply-To match up.... I guess that is the problem :surprised:
If that's true, would it be possible to thread the posts based on the subject title? Then maybe the post order within the thread could be determined by post time.

Welcome back :)

[KevinM]

I have something on my test system on the threading-by-subject which is about 70% complete. It turned out to be easier than I thought.

The way it is working out:

It check against references (as it always has - no change here).
If there is no reference match, it checks if "Re: " appears in the subject title
If it does, it checks for a match by subject title (with the "Re: " removed) and forum and inserts the post.
If not, it creates a new thread (back to the existing script)

If Lierduh finds a way by using the references alone, then that will be a far more efficient way of threading as this will add two more queries to the script.

[BustaCap]

How does it handle multiple "Re:" in the subject?

[KevinM]

Good question. I've got it working fine on my test system now, and it works with multiple "Re: " in the subject.

I will do a final test tomorrow on a clean 'gateway' script, write some instructions and then you can be do some testing on a real mailing list if that's ok. :nervous:

[BustaCap]

Awesome! I'll be happy to test it :)

[trackpads]

Is there anyway to install this so that we can use both the newsgroup and mailing list features? I currently have 2 seperate boards for this and want to merge them both into my VB.

Thanks!

-Jason

[KevinM]

Finally finished it, this got more involved than I originally thought :ermm: , I have tested it on my test forum and it's all working. It needs some real life testing now with someone using mailing lists - thank you BustaCap.

It works pretty much as I described it few posts back:

It searches by references first, if it can't find one or there isn't a match it checks if there is at least one "Re: " in the subject line, which gives us a fair idea that it should belong to an existing thread.

It then will search the subject titles of the existing threads, and if there is a match, select the most recent one and insert it the post into it.

If there is no match, it creates a new thread as normal.

* I have removed the attachment pending a bug removal *

[BustaCap]

I'm going to try this right now :D

[KevinM]

.... waits nervously ....

[KevinM]

Is there anyway to install this so that we can use both the newsgroup and mailing list features? I currently have 2 seperate boards for this and want to merge them both into my VB.

Thanks!

-Jason

Do you mean seperate forums (or sub-forums) within the same board? I'm fairly sure it does this now as standard.

[BustaCap]

It breezed through the first 10 messages, threading them correctly. Then it choked on the 11th message and gave me a SQL error. I pm'd you the error and some further info.

[scottct1]

I ran great for me the first few times, however the messages going out to YAHOOGROUPS are now getting a SQL error.

This is the error I am getting...

Database error in vBulletin 3.0.1:

Invalid SQL: INSERT INTO post
(postid, threadid, title, username, userid, dateline,
pagetext, allowsmilie, showsignature, ipaddress,

iconid, visible, isusenetpost, msgid, ref, parentid)
VALUES (NULL, ,
'[VOOM] Re: Wilt please HELP with accounting dept.',
'prope46', 0,
'1086826687',
'Dear Wilt;
THANK YOU!!! Escalated is an understatement,within an hour or so of
recieving your email your people were calling us apologetic as hell,
and I think we had a call or two the following day and the problem
was resolved!!!

Now gateway.php is running in the backround, and its not stopping. I want to reupload my old gateway.php

How do I stop the old gateway.php from running?

[KevinM]

OK, I'll have a look at this tonight.

[allan grossman]

I ran great for me the first few times, however the messages going out to YAHOOGROUPS are now getting a SQL error.

This is the error I am getting...



Now gateway.php is running in the backround, and its not stopping. I want to reupload my old gateway.php

How do I stop the old gateway.php from running?

If it's a UNIX or Linux box use top to determine the process ID then kill the process with kill -9 process_id - if it's a Windows machine you can use Task Mangler to kill the task.

[trackpads]

When I run the Gateway.php file via telnet I keep getting this error-


Gateway version 2.1.0 13 group(s) gatewayed.

Connecting to server, server says: 200 Randori News http://www.randori.com (Tornado v1.0.5)

Server responded after user name: 381 More Authentication Required

Server responded after password: 281 Authentication Accepted

Info for alt.military at news.randori.com:
211 11542 235125 251459 alt.military

Getting message number 235431:
Segmentation fault

It has run a few times but it fails, is there something I should check?

Thanks again for all the work,

-Jason

[lierduh]

When I run the Gateway.php file via telnet I keep getting this error-




It has run a few times but it fails, is there something I should check?

Thanks again for all the work,

-Jason

How do you run gateway.php? by a lynx session or your php is running as cgi?
If your php is running as cgi, what version? I remember reading somewhere about problems with php running as cgi. The newer version of php fixes that.

Also check your web error log.

[trackpads]

I have the latest PHP and its not running as CGI for sure. I have tried both, via telnet and with the browser. I didnt want to put it into cron until I got the messages downloaded for the first time.

If I run it via shell it will tell me about the segmentation error, if I run it via the browser it just says that its getting message # 231456 and then quits.

Thanks again for all of the help!!!!!!

-Jason

P.s. Great hack!

[lierduh]

I have the latest PHP and its not running as CGI for sure. I have tried both, via telnet and with the browser. I didnt want to put it into cron until I got the messages downloaded for the first time.

If I run it via shell it will tell me about the segmentation error, if I run it via the browser it just says that its getting message # 231456 and then quits.

Thanks again for all of the help!!!!!!

-Jason

P.s. Great hack!

segmentation error is memory related. If you can control php.ini, check the memory_limit setting. The default is 8MB. vB3 often gives error with 8MB. Up it to 16 is a must for vB3. It is also possible the #231456 message has a large attachment.

You can always manually change the lastmsgid in the control panel to be higher than 231456, so that it will bypass this article and only get the aritcles with higher message id.

[trackpads]

Thanks! It was that message I think. I had it skip that one and now it is working.

Thanks!!!!!

-Jason

[BustaCap]

Looking at the header of my outgoing messages, I see that my email address is being scrambled by adding several characters to the end of my name (eg, BustaCap.17laj@site.com). Also, the domain is wrong too (as per the "email address domain in the NNTP Gateway settings). This is great for posts going to newsgroups, but it is preventing me from sending posts to email lists that require a confirmed email address.

Would it be possible to keep the option of scrambling the email address for posts going to newsgroups but allowing for propper email addresses to go out to email lists? If both options aren't possible, perhaps just the ability to have a proper email address go out.

Thanks,
BC

[Marv]

First of all I?ve gotta say, thank you.
The hack is running without any problems and the installation went like a charme. So far, so good..

But now I?ve got some serious problems with other things. I?m living and running my site here in germany and the newsgroups we want to im- and export the postings to/from need correct headers so that outgoing postings to ng?s are available on the newsserver. These are restrictions we simply have to obey.

What am I asking for ?
By now the headers of forum to newsserver-postings are loking like this:

From: Marv <marv.17reqy@usenet.xxx.de>
Newsgroups: de.just.another.newsgroup
Subject: Another posting
Date: Sun, 13 Jun 2004 00:41:09 +0200
Organization: http://www.xxx.de
Lines: 13
Message-ID: <marv.17reqy@usenet.xxx.de>
X-Trace: news.t-online.de nBvEcr5K7cscegZwMIQyfAm1Tzv5/YBpPbMRd52RdGRKaKp4VT
User-Agent: vBulletin USENET gateway
X-Newsreader: vBulletin USENET gateway
X-Originating-IP: 2xx.xxx.xx.xx
Xref: news.t-online.com de.test:578574 To have our postings showing up in our target newsgroup we are forced to complete the headers with the statements:

Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit so that endusers and checkbots read the complete header similiar like this one:
From: Skuld Article Checker <checkbot@ds9.gnuu.de>
Newsgroups: de.just.another.newsgroup
Subject: Re: Another posting
Date: Sat, 12 Jun 2004 15:48:01 GMT
Organization: Goddess Newsfeed
Lines: 44
Sender: <checkbot-owner@ds9.gnuu.de>
Message-ID: <caf88i$rh2$03$1%news.t-online.com@checkbot.fqdn.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit
X-Complaints-To: abuse@ds9.argh.org
Xref: news.t-online.com de.test:426397 Would be very important for us (and 2000+ others on our site) that someone could write this into this hack or could show us, how to fix that problem in another way. Has anyone an idea?

Regards & TIA,
Marv.

[trackpads]

Has anyone seriously given more thought to the Filter idea? I would love to get that on my site since I primarily want the military realted newsgroups but not all the anti-bush spam that ruins them.

Thanks alot for the hard work,

-Jason

[lierduh]

Looking at the header of my outgoing messages, I see that my email address is being scrambled by adding several characters to the end of my name (eg, BustaCap.17laj@site.com). Also, the domain is wrong too (as per the "email address domain in the NNTP Gateway settings). This is great for posts going to newsgroups, but it is preventing me from sending posts to email lists that require a confirmed email address.

Would it be possible to keep the option of scrambling the email address for posts going to newsgroups but allowing for propper email addresses to go out to email lists? If both options aren't possible, perhaps just the ability to have a proper email address go out.

Thanks,
BC

It is simple to fetch the real email address from the user table, but remember every forum user can post yet unlikely everyone of them have subscribed to the list. You naturally do not want other people to post under your email address to the list.

[lierduh]

Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 8bit

Marv.

Find $newthread['postdateline']).

Insert these lines after that:

"\r\nMime-Version: 1.0\r\n" .
"Content-Type: text/plain; charset=ISO-8859-1\r\n" .
"Content-Transfer-Encoding: 8bit" .

should work although not tested. I must say it is quite strange that you need to have these to be able to post to the ng.

[KevinM]

Here are two more mini-hacks which some may find useful. Thanks to BustaCap for testing the threading-by-subject script and to trackpads for testing the killfile filter.

Threading by Subject Title
The threading by subject script works on email lists only, whereby sometimes there are no message references included in the headers, so the only way to match the thread is by the subject title. Now with bugs removed :mad:

Killfile Filter to block posts
The killfile is similar to the equivilent you would find on a newsreader. It allows the admin to block certain messages from downloading if certain words or phrases are met by checking the posters name, subject title, email address or server where they originated. It is a fairly lightweight filter suitable for blocking annoying trolls and needs to used carefully to avoid blocking legitimate posts. Some may find this against the principles of usenet, but it's here if you want it.

You can use either or both of these together and are compatible with the stripping of email footers.

[scottct1]

Does the threading for Mailing lists work correctly now? I installed it the other day and it worked a few times, and then I started getting mysql errors (posted above in this thread)

Also please explain how to set it up to strip document footers. On yahoogroups I would like to keep the ads (I would feel guilty stipping those out) however I would like to remove the instructions on how to unsubscribe to the Yahoogroup that is on the bottom of each message.

Thanks

[KevinM]

Does the threading for Mailing lists work correctly now? I installed it the other day and it worked a few times, and then I started getting mysql errors (posted above in this thread)
Yes, problem solved ;)


Also please explain how to set it up to strip document footers. On yahoogroups I would like to keep the ads (I would feel guilty stipping those out) however I would like to remove the instructions on how to unsubscribe to the Yahoogroup that is on the bottom of each message.

The strip-footers script can be found here (https://vborg.vbsupport.ru/showpost.php?p=519774&postcount=148) You enter the characters or words that start the area you want stripped - just enough for it to be unique, and from that point on the rest of the message will be stripped.

[Marv]

should work although not tested. I must say it is quite strange that you need to have these to be able to post to the ng. Thanks lierduh,
that worked for me.

We?re connecting to a non-public newsgroup on a buiseness-server, so we?re forced to do this modifications in the header. In german newsgroups you?ld find thousands of people yelling at you if you would participate a newsgroup with Outlook or OutlookExpress. They don?t even call those products "newsreader", they call it "data garbage centrifuge" because of the amount of bugs they are producing while trying to convert header-lines in a proper manner).
Most ng-members are using special news-readers like forte agent and others. These software pieces are producing correct headers but can?t read the one?s Outlook is inserting there.

lierduh, one problem remains.
In Germany they use letters like "?" "?" and "?" (and in capitols also of course). Is there anybody out there who could write some code which would convert these special german letters and other signs into ASCII on the server and in the moment gateway.php is sending the mesages out to the newsgroups ? (for outgoing posts to the newsserver & for incoming)

German letters:
? to Ä
? to ä
? to Ö
? to ö
? to Ü
? to ü
? to ß

Signs:
< to <
> to >
? to §
? to ©
? to ®
& to &
? to ·
" to +
  (Space)


We would really appreciate if someone could do this.. Any chance..? :ermm:

[Marv]

Killfile Filter to block posts
Great, Kevin - that?s another thing I was looking for. Thanks ! You read my mind :)

This filter checks 4 elements of each incoming message, and if ANY of them
match it will ignore that message.

The four elements are:

Username
Subject title
Email address
Server address Could you add the body of a message as fifth element ? Some (including me) could find that very useful.

[Marv]

lierduh,

i noticed that the hack?s only fetching posts (sending them to a newsserver) which are answers. postings which are opening up a new thread aren?t included and they never show up in a newsgroup.

Can anyone confirm that for his/her site ?

[BustaCap]

lierduh,

i noticed that the hack?s only fetching posts (sending them to a newsserver) which are answers. postings which are opening up a new thread aren?t included and they never show up in a newsgroup.

Can anyone confirm that for his/her site ? I just tested this and my new thread showed up in the newsgroup.

When I first installed this hack I was using the vB cron for running the gateway, with the gateway.php supposed to be run every hour. However, I found that it was only running several times a day for some reason. When I used this method posts would often not show up. To trouble shoot I manually ran gateway.php immediately after posting and I found that all of those posts would be sent. I now use my site cpanel to run the cron job every 10 minutes and all posts show up. Try posting a new thread and then manually run gateway.php and see if that works.

[Marv]

Try posting a new thread and then manually run gateway.php and see if that works.
Hello BustaCap, thanks for your suggestion - but that?s exactly what I already did before I?d posted my question here. I have no glue why starting posts aren?t send to the newsservers. (It?s definitifly the hack itself - the postings just don?t seem to be in my forum when I run gateway.php..)

And even worse - I can?t run cron jobs in my hosting package :ermm:
Any ideas? (beside upgrading the hosting-package..)

[lierduh]

Hello BustaCap, thanks for your suggestion - but that?s exactly what I already did before I?d posted my question here. I have no glue why starting posts aren?t send to the newsservers. (It?s definitifly the hack itself - the postings just don?t seem to be in my forum when I run gateway.php..)



A bit confused there. As soon as you post a new thread, the thread should be in your forums. That is nothing to do with the gateway. Or unless you meant: you posted a new thread, but the gateway script did not send it to the server, or the server did not accept your new thread post. In any case, you might like to run it manually, capture the screen print out and post here for fault finding purpose.

[trackpads]

Here are two more mini-hacks which some may find useful. Thanks to BustaCap for testing the threading-by-subject script and to trackpads for testing the killfile filter.

Threading by Subject Title
The threading by subject script works on email lists only, whereby sometimes there are no message references included in the headers, so the only way to match the thread is by the subject title. Now with bugs removed :mad:

Killfile Filter to block posts
The killfile is similar to the equivilent you would find on a newsreader. It allows the admin to block certain messages from downloading if certain words or phrases are met by checking the posters name, subject title, email address or server where they originated. It is a fairly lightweight filter suitable for blocking annoying trolls and needs to used carefully to avoid blocking legitimate posts. Some may find this against the principles of usenet, but it's here if you want it.

You can use either or both of these together and are compatible with the stripping of email footers.
GREAT work!!!!!!!

[trackpads]

I have a problem and I am sure I am doing something wrong. I have many different mailing lists that go to my list email box. I have been trying to set up the script to take each list and put it into seperate forums. For example-

List A to Forum A
List B to Forum B etc etc

The problem is that no matter what I put in the settings in the newsgroup/mlist settings it still tells me that it sucessfully connected to the pop3 server and retrieved 0 messages. I also went in and looked are there over 60 messages in the inbox of that account.

Thanks again for the help,

-Jason

[BustaCap]

Are you sure that your email client isn't set to delete the messages from the server once it downloads them?

[Marv]

Or unless you meant: you posted a new thread, but the gateway script did not send it to the server, or the server did not accept your new thread post. In any case, you might like to run it manually, capture the screen print out and post here for fault finding purpose. good morning, lierduh.

sorry - that?s what I should have done with the last post already.

Directly after submitting a new thread to the forums I manually run gateway.php without clicking anywhere else and it says:
Connecting to server, server says: 200 The server welcomes infong64.xxx.de (2xx.xxx.xxx.xxx). Authorization required for reading and posting.

Server responded after user name: 381 PASS required

Server responded after password: 281 Authentication accepted. (UID=238408)

Info for here.goes.the.newsgroup.name at news.xxx.de:
211 18230 450417 469608 here.goes.the.newsgroup.name nothing more...that?s it.
If someones posting now a new answer to that post gateway.php works as it should, but is only sending the answer to the newsserver - not the starting post. so it?s a bit confusing for the newsgroup when answers appear but the starting post is missing..
And of course - the post shows up in my forums but gateway.php just doesn?t seem to recognize that.

If it?s not to impudent, lierduh, would you mind to have a look in my forums by yourself? In that case I would send you the logins for .htaccess and the forum via PM. (erm...and before you ask - the problem remains with and without .htaccess enabled ;) )
Guess it?s easier to make a troubleshooting when you?re on the deck and can see the script in action. Just a suggestion.

And by the way.. - I really appreciate the work you?ve done to this hack. Thanks for the given support - must be hard to handle all the little problems that endusers like me are coming up with..:nervous:

[trackpads]

Are you sure that your email client isn't set to delete the messages from the server once it downloads them?
No, I was using the webmail and I just rechecked, they are still on there.

Thanks again!

-Jason

[Dioxin]

how can i deactivate the attachment import from the script?

does it work, if i comment the following line??:
if ($message['attachments']) {
// process_attachments($date, $postid, $threadid);
}

[BustaCap]

One thing I will add into the next version is: right now, the "From: " email address is the same format as the USENET ones, ie. it does not contain the real email address due to spam prevention. There should be an option for the mailing list forums to send real email address. Some mailing lists only accept subscribed member's posts.
This would be very helpful. At a minimum it would allow at least me to post to the group via the forum.

It is simple to fetch the real email address from the user table, but remember every forum user can post yet unlikely everyone of them have subscribed to the list. You naturally do not want other people to post under your email address to the list.
The way I would have this particular forum setup would be to first deny posting permission to all board members and then give permission only to those members who are already members of the email group I am mirroring. However, I would only do this if the messages look like they are coming from the sender and not from me. For this type of scenario I wouldn't care if my group members posted to the group using my email address as long as the individual posts showed up with their name in the From field.

[trackpads]

For some reason, I set up about 15 groups for 15 different mailing lists but the program imports them into the wrong groups. For example out of the 15 it imported them all into just 2 of the groups.

Any idea what I am doing wrong?

Thanks again for the help!

-Jason

[lierduh]

For some reason, I set up about 15 groups for 15 different mailing lists but the program imports them into the wrong groups. For example out of the 15 it imported them all into just 2 of the groups.

Any idea what I am doing wrong?

Thanks again for the help!

-Jason

Are you using the same Pop3 mail box? The current version requires a separate email for each mailing list.

[lierduh]

This would be very helpful. At a minimum it would allow at least me to post to the group via the forum.


The way I would have this particular forum setup would be to first deny posting permission to all board members and then give permission only to those members who are already members of the email group I am mirroring. However, I would only do this if the messages look like they are coming from the sender and not from me. For this type of scenario I wouldn't care if my group members posted to the group using my email address as long as the individual posts showed up with their name in the From field.

There is no clear solution to this. For the lists that need confirmed email address, the individual users need to subscribe to the list. Once they are subscribed, there is very little benefit for them to post through the forum instead of just shoot an email directly to the list. It will also look odd for the rest of the list members to see an "email" sent to the list is actually coming from a forum.

1)Open list, anyone can post to it, even someone is not subscribed. This is the best way to use this script.

2)Closed list. There isn't a good clear solution. Even if you offer your master email address as the posting email address with other user's forum user name. It will not be a good picture to be honest. It is kind of cheating, and unrespectful to the list owner, since the list is a close list. I really need to think again about how to handle this.:) One thing a forum ower does not want to have is to have the list members hating your forums. The same goes for newsgroups.

[lierduh]

how can i deactivate the attachment import from the script?

does it work, if i comment the following line??:
if ($message['attachments']) {
// process_attachments($date, $postid, $threadid);
}

As you have found out. That is correct.:)

[lierduh]

good morning, lierduh.

sorry - that?s what I should have done with the last post already.

Directly after submitting a new thread to the forums I manually run gateway.php without clicking anywhere else and it says:
nothing more...that?s it.
If someones posting now a new answer to that post gateway.php works as it should, but is only sending the answer to the newsserver - not the starting post. so it?s a bit confusing for the newsgroup when answers appear but the starting post is missing..
And of course - the post shows up in my forums but gateway.php just doesn?t seem to recognize that.

If it?s not to impudent, lierduh, would you mind to have a look in my forums by yourself? In that case I would send you the logins for .htaccess and the forum via PM. (erm...and before you ask - the problem remains with and without .htaccess enabled ;) )
Guess it?s easier to make a troubleshooting when you?re on the deck and can see the script in action. Just a suggestion.

And by the way.. - I really appreciate the work you?ve done to this hack. Thanks for the given support - must be hard to handle all the little problems that endusers like me are coming up with..:nervous:

Can you access to your MySQL table directly, perhaps using something like phpmyadmin?

Find out which postid has the problem of not being sent. Check the "post" table, and see if the "isusenetpost" is 1. When someone posts a new message, isusenetpost is 0 as the default. The script changes it to 1 after posting it.

[trackpads]

Are you using the same Pop3 mail box? The current version requires a separate email for each mailing list.
Yup. same box. Dang :-)

[BustaCap]

Once they are subscribed, there is very little benefit for them to post through the forum instead of just shoot an email directly to the list.
Actually, just the opposite. By posting through the forum you are letting your posts be indexed and searchable by user name, otherwise the posts are by "guest". I use that feature quite often on the rest of my forum. Also, the biggest benefit in my opinion is that it's much easier to follow threads in a forum with threaded posts as compared to email lists where posts are received in your mailbox. If you don't read the list every day, it's a real pain trying to follow topics in your inbox vs. threaded posts on your forum.

However, for some reason I'll never understand, there will always be people who only want to participate via email. Thus the need for both the forum and email list and not just one or the other.



2)Closed list. There isn't a good clear solution. Even if you offer your master email address as the posting email address with other user's forum user name. It will not be a good picture to be honest. It is kind of cheating, and unrespectful to the list owner, since the list is a close list. I really need to think again about how to handle this.:) One thing a forum ower does not want to have is to have the list members hating your forums. The same goes for newsgroups.
It sounds like you are assuming that I'm not the list master. As it happens I'm not ;). But my point is that you are looking at this from only one point of view, which is putting the forum owner in a dishonest light. Of course, I would never do this without a discussion with the list owner as well as the list users. I mentioned before that only lists users would have posting privileges, making the "closed" list not an issue. It would only be done with everyone?s approval. However, when I suggest it I would like to be able to demonstrate how it would work, but I can't do that without a working system :)

I'm quite sure that if it worked like how I hope it will that everyone would be agreeable to it since it would benefit both the list and the forum; increased exposure for both groups and imo, a much better user interface with the forum.

[Marv]

Can you access to your MySQL table directly, perhaps using something like phpmyadmin? hi lierduh, I can :)

Find out which postid has the problem of not being sent. Check the "post" table, and see if the "isusenetpost" is 1. When someone posts a new message, isusenetpost is 0 as the default. The script changes it to 1 after posting it. It?s interesting - I had a look in the tables as you suggested and some of them still had the value isusenetpost is 0. But that was yesterday! I said it doesn?t post new threads - but after I had that little peek into the tables...and damn,..now it works like a charme :paranoid: What the heck?s happening there.....?? May be my database is alive and made from flesh and blood ? But than it would be a female for sure :)

* Marv takes full cover.. :knockedout:

Whatever, the problem seems to be solved. Thanks, lierduh.

[lierduh]

Actually, just the opposite. By posting through the forum you are letting your posts be indexed and searchable by user name, otherwise the posts are by "guest". I use that feature quite often on the rest of my forum. Also, the biggest benefit in my opinion is that it's much easier to follow threads in a forum with threaded posts as compared to email lists where posts are received in your mailbox. If you don't read the list every day, it's a real pain trying to follow topics in your inbox vs. threaded posts on your forum.



So the benefit only comes down to that particular user's username is used instead of a 'Guest post' in the forums? (oh, also the post count is boosted by one?:)). When a list member (who is also your forum member) posts to the list by sending an email, he or she will receive all the messages in the email. A filter would need to be activated in his/her email clients to get rid of all the messages coming from the mailing list. So that he/she can read the same messages in your forum with some of the reading benefits that a forum provides (may be)? An email client usually does threading too.:)

There are quite a few benefits by using email to participate in a mailing list instead of using a forum interface. eg.

1) No delay of sending the message
2) Alerted straight away when a reply is available
3) Knowing the format sent will be the format seen in the email
...


However, for some reason I'll never understand, there will always be people who only want to participate via email. Thus the need for both the forum and email list and not just one or the other.


I can fully understand someone who wants to use the proper way to participate in a mailing list. I can however understand people who do not trust a forum to send the message.:)



It sounds like you are assuming that I'm not the list master. As it happens I'm not ;). But my point is that you are looking at this from only one point of view, which is putting the forum owner in a dishonest light. Of course, I would never do this without a discussion with the list owner as well as the list users. I mentioned before that only lists users would have posting privileges, making the "closed" list not an issue. It would only be done with everyone?s approval. However, when I suggest it I would like to be able to demonstrate how it would work, but I can't do that without a working system :)

I'm quite sure that if it worked like how I hope it will that everyone would be agreeable to it since it would benefit both the list and the forum; increased exposure for both groups and imo, a much better user interface with the forum.

My point of view comes from providing the functionality that are logical to general users. Or providing options for different users to make them logical.:)

I guess what you want to set up is (let me know if it is not):

* Let only handful of forum members to post in the mailing list forum, perhaps by setting forum permission.
* They need to subscribe to the list using the same email address as the email address they use for the forums

I think finding the methodology is the hard part.

Looks like the mailing list part of the function has taken over the usenet part. It can only be:

1) The usenet part just works without the need to post a message
2) There are plenty demand for the mailing list function (should I say list archive function at this stage)

[BustaCap]

So the benefit only comes down to that particular user's username is used instead of a 'Guest post' in the forums? (oh, also the post count is boosted by one?:)). Mainly it allows you to search by username. For example, show me all posts by Capt. Kirk with Klingon in the title. I use this type of search all the time when I remember who posted the thread and the subject, but I can't seem to find it by a general search. I also use the "find all posts by this user" quite often.

When a list member (who is also your forum member) posts to the list by sending an email, he or she will receive all the messages in the email. A filter would need to be activated in his/her email clients to get rid of all the messages coming from the mailing list.
Well, I would suspect that those who choose to participate via the forum would no long use the email list since it would be redundant.

When I originally was looking at forum software there was one product (I forget which one) that allowed users to particpate via the forum or via email. So, if someone posted a message on the forum they would receive an email that they could then respond to. That response would then show up back in the forum. Neither one would even be aware the other wasn't posting messages in the same way. This allows the best of both worlds, those who want to particpate via the forum can and those who choose to only particpate via the mailing list can also do so. The hard part is making it transparant to the user. When I think how it would work I guess that's what I'm shooting for :) Those who are happy to post via email could continue to do so and those who want to view and post messages on the forum could choose that option. The end result is that both groups are happy and both groups have increased exposure.

I guess what you want to set up is (let me know if it is not):

* Let only handful of forum members to post in the mailing list forum, perhaps by setting forum permission. Yes. The way I would set it up would be to deny everyone posting privlages to that forum, and then I would create a usergroup with posting privlages, and anyone that was a member of the listgroup who requested it, I would move them to that usergroup.


* They need to subscribe to the list using the same email address as the email address they use for the forums I think so although I'm not sure. My *guess* would be that for it to work, my email address would first be accepted, and then the listbot would look at the From field in the header (or maybe the reply to email address?) and then use that to show who the message is from.

I am likely the only one here who is interested in this function :D, so I don't want to take up too much of your time making something that I'm likely the only one to use. I was just hoping it might be somthing quick and simple, like unscrambling the outoing email address and perhaps inserting the posters name into the "From" field. Regardless, I am very happy with how it works now and I appreciate the work you have done on the hack :)

[fustanella]

What is it?
It's great, that's what is is. :D

I am delighted with this; thank you! It was painless to install, worked both inbound and out the first time without fiddling, and adds a handy new service for my members.

I would like to find a way to automate the gateway.php calls under WinNT 4, though. I understand the concept of 'cron' but don't see how this maps directly over, other than through "Scheduled Tasks" - specifically, how I can run the PHP script either in a browser window that closes when done, or without the need for a browser at all.

(Anyone who has insights on my related question of importing text files as vB3 messages is welcome to chime in as well, over at https://vborg.vbsupport.ru/showthread.php?t=66250 - thanks.)

[jilly]

i can find the name, like alt.paranormal - how do i know what it's nntp is?

[fustanella]

i can find the name, like alt.paranormal - how do i know what it's nntp is?NNTP is the protocol, like HTTP is for web pages. You'll find newsgroups on your ISP's news-server, often news.(ispname).com. Check with your ISP for specifics.

If your ISP doesn't carry newsgroups or the specific one you want, there are subscription services available, and maybe even a free German news-server still available upon request.

[trackpads]

Is there a way to write a conditional in Postbit LEgact so that if it is a USENET or MList post that we can disable the entire postbit or just make it display something like "USENET"? That would make it a lot nicer when you have some really long names that are taken from the usenet.

Also, is there a way to disable the killfile for mlists only? The killfile is an AWEOME addition and has really, really helped me on my site.

Thanks again for all the work on this!!

-Jason

[AWS]

It's great, that's what is is. :D

I am delighted with this; thank you! It was painless to install, worked both inbound and out the first time without fiddling, and adds a handy new service for my members.

I would like to find a way to automate the gateway.php calls under WinNT 4, though. I understand the concept of 'cron' but don't see how this maps directly over, other than through "Scheduled Tasks" - specifically, how I can run the PHP script either in a browser window that closes when done, or without the need for a browser at all.

(Anyone who has insights on my related question of importing text files as vB3 messages is welcome to chime in as well, over at https://vborg.vbsupport.ru/showthread.php?t=66250 - thanks.)
On Windows create a scheduled task.
Create a new task. When the box pops up browse to php.exe. The full path should show for the exe.
You should see something like
C:\Inetpub\php\php.exe
or whatever the path to php.exe is. At the end of that add the path to gateway.php. When done you should have something like
C:\Inetpub\php\php.exe C:\Inetpub\wwwroot\gateway.php
Now go to advanced and set it to run every 15 minutes for 23 hours and 59 minutes. Click OK and when the password box comes up put in your administartor password.
You now should have a scheduled task called php.exe. Open scheduled task folder and right click the task and run it. A DOS box should popup as it runs. If the box pops up and disapears right away then try to run the task command from a dos prompt. By doing this you'll see what the problem is. More than likely bad path.

[fustanella]

C:\Inetpub\php\php.exe C:\Inetpub\wwwroot\gateway.phpThank you. I'm familiar with scheduled tasks, and prefer to try in a command window before creating a task so errors can be seen. Good thing, as I got 'em. :)

Turck MMCache - can not create shared memory area
in unknown on line 0 - this occurs both in a graphical warning requiring clicking on the pretty button, and in the text in the command window. I'm running the same script that works through my browser.

I searched my machine and checked my vB install to make sure I didn't have various copies of PHP floating about, and I have only the one in my c:\php4\php dir. It's PHP 4.3.3 if that matters.

[Princeton]

the more I use this hack the more I like it!

... missing htmlspecialchars($subject) :rolleyes:

[Cold Steel]

After getting thousands of messages from usenet, I'm starting to run into errors.

This is the latest:


Getting message number 300606:
'Re: Bush's war -- against Richard Clarke' from Congenital Kano. Thread found by References.
Getting message number 300607:
'Reversing Pledge, Bush Lets OPEC Bilk Americans' from Paul Kekai Manansala. New thread.
Getting message number 300608:
<html><head><title>asiansINC.com Database Error</title><style type="text/css"><!--.error { font: 11px tahoma, verdana, arial, sans-serif; }--></style></head>
<body></table></td></tr></table></form>
<blockquote><p class="error">&nbsp;</p><p class="error"><b>There seems to have been a slight problem with the asiansINC.com database.</b><br />
Please try again by pressing the <a href="javascript:window.location=window.location;">refresh</a> button in your browser.</p><p class="error">An E-Mail has been dispatched to our <a href="mailto:">Technical Staff</a>, who you can also contact if the problem persists.</p><p class="error">We apologise for any inconvenience.</p><form><textarea class="error" rows="15" cols="100" wrap="off">Database error in vBulletin 3.0.1:

Invalid SQL: SELECT post.postid FROM thread AS thread, post AS post
WHERE thread.threadid = post.threadid AND
(post.msgid = '&lt;For-Tom_Cerco's-benefit-A772E2.23264904042004@phswest.com&gt;') AND
thread.forumid = 72
ORDER BY post.dateline DESC LIMIT 1
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 's-benefit-A772E2.23264904042004@phswest.com&gt;') AND


mysql error number: 1064

Date: Sunday 20th of June 2004 01:44:04 AM
Script: http://forums.asiansinc.com/gateway.php
Referer:
Username: Webmaster
IP Address: **.**.**.***
</textarea></form></blockquote>

</body></html>


Any ideas?

[Cold Steel]

Also, I signed up for an account at Giganews for $7.99/month for 2GB.

Then I found out about Teranews.com - 50MB per day for only a $3.95 setup fee. This is slightly less than 2GB a month, but their paid acccount for $7.95 gives me 250MB a day (~7.3 GB month).

Is there really such a huge difference in the quality of the connection? I don't think I'll need that much bandwidth. I'm considering dropping giganews and signing up with a free account at teranews. Thoughts?

[Cold Steel]

If it's a UNIX or Linux box use top to determine the process ID then kill the process with kill -9 process_id - if it's a Windows machine you can use Task Mangler to kill the task.

How do I determine which one is gateway.php?

[lierduh]

After getting thousands of messages from usenet, I'm starting to run into errors.

This is the latest:


(post.msgid = '<For-Tom_Cerco's-benefit-A772E2.23264904042004@phswest.com>') AND


Any ideas?

The single quote in the message ID is the cause of the problem. Message ID should not contain quotes charactor. You can either bypass this message by editing the lastmsg in the control panel, or do the following:

Around line 338 find:

(post.msgid = '$par_msgid') AND

Change it to:

(post.msgid = '" . addslashes($par_msgid) . "') AND

[KevinM]

Also, is there a way to disable the killfile for mlists only? The killfile is an AWEOME addition and has really, really helped me on my site.



Yes you can. Find this:else if ($killfileresult = checkkillfile($message))

and change it to to:else if ($killfileresult = checkkillfile($message) AND $grouptype == 'news') This will work on newsgroups only.

To work on mailing lists only:
else if ($killfileresult = checkkillfile($message) AND $grouptype != 'news')

I haven't tried it, but it should work.

[KevinM]

Could you add the body of a message as fifth element ? Some (including me) could find that very useful.

If you find this:if ((stristr(trim($message['subject']), $killphrase)) OR
(stristr(trim($message['from']), $killphrase)) OR
(stristr(trim($message['path']), $killphrase)))

and change it to this:
if ((stristr(trim($message['subject']), $killphrase)) OR
(stristr(trim($message['from']), $killphrase)) OR
(stristr(trim($message['path']), $killphrase)) OR
(stristr(trim($message['text']), $killphrase)))

This will now search the main text of the message as well.

[KevinM]

This is a quick one to get the thread preview to work with this hack.

Find:
logging("'$subject' from ". $from_name .
". New thread.\r\n");

And below it put:
// enable thread preview
$DB_site->query("UPDATE " . TABLE_PREFIX . " thread
SET firstpostid = $postid
WHERE threadid = $threadid");

If you have thread preview enabled in the adminCP then all new threads from now on will show a preview when you hover over them.

[Cold Steel]

The single quote in the message ID is the cause of the problem. Message ID should not contain quotes charactor. You can either bypass this message by editing the lastmsg in the control panel, or do the following:

Around line 338 find:

(post.msgid = '$par_msgid') AND

Change it to:

(post.msgid = '" . addslashes($par_msgid) . "') AND


Thanks. :)

Made the change, but now I'm getting this error:


Getting message number 300622:
'Re: When did "oriental" become a bad word?' from nttran@webtv.net. Thread found by References.
Getting message number 300623:
'Re: Japanese loves American fashion.' from Charley Brown and Almighty Snoopy. Thread found by References.
Getting message number 300624:
'Re: Ref: Hiroshima and Nagasaki Justified?' from Mike1. Thread found by References.
<html><head><title>asiansINC.com Database Error</title><style type="text/css"><!--.error { font: 11px tahoma, verdana, arial, sans-serif; }--></style></head>
<body></table></td></tr></table></form>
<blockquote><p class="error">&nbsp;</p><p class="error"><b>There seems to have been a slight problem with the asiansINC.com database.</b><br />
Please try again by pressing the <a href="javascript:window.location=window.location;">refresh</a> button in your browser.</p><p class="error">An E-Mail has been dispatched to our <a href="mailto:">Technical Staff</a>, who you can also contact if the problem persists.</p><p class="error">We apologise for any inconvenience.</p><form><textarea class="error" rows="15" cols="100" wrap="off">Database error in vBulletin 3.0.1:

Invalid SQL: INSERT INTO post
(postid, threadid, title, username, userid, dateline,
pagetext, allowsmilie, showsignature, ipaddress, iconid,
visible, isusenetpost, msgid, ref, parentid)
VALUES (NULL, 33342,
'Re: Ref: Hiroshima and Nagasaki Justified?',
'Mike1', 0,
'1087752217',
'(alfred montestruc) wrote:

&gt;....societies have....


Ambiguous-Collective fallacy. (It\'s a popular one.)

Lemme help ya: http://home.mn.rr.com/meadowbrookhome/z/FALLACYS.HTM


No charge.

--

Reply to mike1@@@usfamily.net sans two @@, or your reply won\'t reach me.

Anarchy: The political manifestation of Liberty.
Capitalism: The economic manifestation of Liberty.
\&quot;An election is nothing more than an advance auction of stolen goods.\&quot;
-- Ambrose Bierce, \&quot;The Devil\'s Dictionary\&quot;
\&quot;Democracy: Nine Nazis and one Jew voting in the 1932 German elections.\&quot;
', 1, 0,
'For-Gilly\'s-benefit@usfamily.net', 0, 1, 1,
'&lt;For-Gilly's-benefit-AE567C.01160607042004@phswest.com&gt;',
'&lt;5dda60h3nm2n3as4k80kemmaaf6pb2v8kh@4ax.com&gt; &lt;c2c06a17.0403311712.1ded4c6c@posting.google.com &gt; &lt;ht9p60p9o42esuufj2mqghisklr4os2q21@4ax.com&gt; &lt;c2c06a17.0404031725.609a165a@posting.google.com &gt; &lt;hstu60htq5qcemv309jel8f9neu9g10v64@4ax.com&gt; &lt;c2c06a17.0404041024.7a514d28@posting.google.com &gt; &lt;For-Tom_Cerco's-benefit-A772E2.23264904042004@phswest.com&gt; &lt;c2c06a17.0404050502.26a4556f@posting.google.com &gt; &lt;For-Tom_Cerco's-benefit-358608.22421205042004@phswest.com&gt; &lt;c2c06a17.0404061456.69a8fa88@posting.google.com&gt;' ,
0 + 0)
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 's-benefit-AE567C.01160607042004@phswest.com&gt;',
'&lt;5dda60h3n

mysql error number: 1064

Date: Sunday 20th of June 2004 12:23:37 PM
Script: http://forums.asiansinc.com/gateway.php
Referer:
Username: Webmaster
IP Address: **.**.**.***
</textarea></form></blockquote>

</body></html>

[Cold Steel]

How do I determine which one is gateway.php?

Is there a way to get it to run more than every 30 minutes? I keep on getting this message:

Another instance of gateway.php is running, try again in 30 minutes if the script was crashed.

when running it manually. Would the same thing happen if I set it to cron?

[Cold Steel]

After disabling the one newsgroup above that is still giving me mysql errors, I have gateway.php set to run every 10 minutes via cron.

However, I am getting tons of doubleposts in those newsgroups. I think it must be because of gateway.php, because Agent and Google don't show the double posts. I don't think they're being threaded properly, either. I'd greatly appreciate it if someone could take a look for me:

alt.asian-movies
http://forums.asiansinc.com/forumdisplay.php?f=74
http://groups.google.com/groups?hl=en&lr=&ie=UTF-8&q=alt.asian-movies&meta=

alt.food.asian
http://forums.asiansinc.com/forumdisplay.php?f=73
http://groups.google.com/groups?q=alt.food.asian&ie=UTF-8&hl=en

[lierduh]

Thanks. :)

Made the change, but now I'm getting this error:
[/code]

Around line 392, find:

'{$message['message-id']}',
'{$message['references']}',

Change them to:

'" . addslashes($message['message-id']) . "',
'" . addslashes($message['references']) . "',

[Cold Steel]

Woohoo, working like a charm again.

Positive Karma for you if this place had it. :)

[Qwest]

Mine was working fine, then all of a sudden it started getting stuck on "Getting message number 11653:".

It goes that far then says done. Of course when I try to refresh it I get the 30 minute warning. No new posts are picked up even after I've added new news groups.

Please Help!

[Cold Steel]

You may want to do these fixes:

https://vborg.vbsupport.ru/showpost.php?p=524533&postcount=217
https://vborg.vbsupport.ru/showpost.php?p=524755&postcount=224

I've had no problems after I made those changes.

[Cold Steel]

How come some posts go through, but some don't?

This one didn't go through - http://forums.asiansinc.com/showpost.php?p=475604&postcount=2

But these four did - http://forums.asiansinc.com/search.php?searchid=20571

[trackpads]

Yes you can. Find this:else if ($killfileresult = checkkillfile($message))

and change it to to:else if ($killfileresult = checkkillfile($message) AND $grouptype == 'news') This will work on newsgroups only.

To work on mailing lists only:
else if ($killfileresult = checkkillfile($message) AND $grouptype != 'news')

I haven't tried it, but it should work.
Thanks!!!!!

[Cold Steel]

Is there anyway to set it so vB signatures ARE passed into the newsgroups?

[KevinM]

Is there anyway to set it so vB signatures ARE passed into the newsgroups?

It's an option in the control panel (nntp gateway settings). Set the 'send no signature' to 0.

Keep in mind that most groups have rules on the size of the sigs, mostly 2 lines or 4 lines and with VB3 your users could have much, much bigger sigs.

[Princeton]

Does anyone have the "mailing-list footer stripper" working (released by KevinM)?

If I understand correctly you are suppose to add the first line of the footer between { } and it should strip the remaining lines -- is this correct?

If it is - than I must be missing something. :(

[Princeton]

Request
An option to NOT send posts (created by forum members) to newsgroup.

eg:
Send posts created by members to newsgroup? Yes/No
(The best place to put this would be in the gateway settings.)

[Cold Steel]

How come some posts go through, but some don't?

This one didn't go through - http://forums.asiansinc.com/showpost.php?p=475604&postcount=2

But these four did - http://forums.asiansinc.com/search.php?searchid=20571
I don't think posts from members with longer usernames or spaces in their usernames are going through. Any idea why?

[Qwest]

Important,

This hack is great but it's driving my post count on my site INSANE!!

Is there anyway to run this hack but have the posts NOT counted towards your BOARD'S total posts?

NOTE: I said BOARD total not USER total. Everywhere I ask this people tell me about teh option in the admincp to not count posts towards USER totals.

[lierduh]

I don't think posts from members with longer usernames or spaces in their usernames are going through. Any idea why?

Space in user name is not a problem. Length should not be too.

Please try to capture the screen when you run it manually.

[lierduh]

Important,


This hack is great but it's driving my post count on my site INSANE!!

Is there anyway to run this hack but have the posts NOT counted towards your BOARD'S total posts?


NOTE: I said BOARD total not USER total. Everywhere I ask this people tell me about teh option in the admincp to not count posts towards USER totals.

I think you may have to disable the thread/post counts in the usenet forums (by altering code).

[Cold Steel]

Space in user name is not a problem. Length should not be too.

Please try to capture the screen when you run it manually.
It's attached. The usernames of "Webmaster" and "teapot" have no problems being posted to usenet -- I've checked through Agent.

However, the usernames "The Invisible Man" and "Three Gun Fish" haven't gotten through.

[lierduh]

It's attached. The usernames of "Webmaster" and "teapot" have no problems being posted to usenet -- I've checked through Agent.

However, the usernames "The Invisible Man" and "Three Gun Fish" haven't gotten through.

I can't see the post by Webmaster or teapot being sent too. Are you talking about an old post which was not sent? you need to test on a freshly posted message.

[Cold Steel]

Ok, I made a post http://forums.asiansinc.com/showthread.php?p=476054#post476054 but for some reason it gave me a 500 error when running gateway.php.

So I disabled signatures and tried again, same result.

See attached.

[lierduh]

Ok, I made a post http://forums.asiansinc.com/showthread.php?p=476054#post476054 but for some reason it gave me a 500 error when running gateway.php.

So I disabled signatures and tried again, same result.

See attached.

The error must be your original problem. You may need to do a manual post following the instruction in the first page, ie. ssh, telnet etc. Basically your news server does not know the posting command sent by gateway script. 1) Are you allowed to post? 2) Have you changed the script, especially nntp.php?

[Cold Steel]

I am going to try the SSH instruction on the front now.

1) I should be allowed to post, since some posts have gone through before through gateway.php.

2) The only modifications I made were the ones you suggested to gateway.php.

If you have the time, would you mind getting on some sort of instant messaging client?

[Cold Steel]

Hmm. Both telnet and Agent tell me that I'm not allowed to post...


Very odd.

[fustanella]

On Windows create a scheduled task. Just to clarify, did you mean a task within the Windows operating system, or within the task manager built into vB3? I took you to mean the former, and when I tried it I couldn't get it to work; just now, though, I stumbled across the latter and it appears to be doing the trick each hour.

[Cold Steel]

Forget it - I signed for a free account with Teranews which is letting me post.

[Qwest]

Just to clarify, did you mean a task within the Windows operating system, or within the task manager built into vB3? I took you to mean the former, and when I tried it I couldn't get it to work; just now, though, I stumbled across the latter and it appears to be doing the trick each hour.

I find it better to use the one on teh Operating system if you can. The one in VB is based on visitors to your site. If you site has lulls in traffic your task won't be run until someone visits.

[Qwest]

Everyone I found out how to make the posts not count towards your total board post counts.

Note: this hack is totally the work of Wayne Luke. I'm just copy/pasting

Should be easy...

Add a smallint field on the database record such as "is_usenet" and set it to '1' for the forums that are.

Then in index.php find the following code (it is near the bottom):

if (is_array($forumcache))
{
foreach ($forumcache AS $forum)
{
$totalthreads += $forum['threadcount'];
$totalposts += $forum['replycount'];
}
}


And change it to:

if (is_array($forumcache))
{
foreach ($forumcache AS $forum)
{
if (!$forum['is_usenet']) {
$totalthreads += $forum['threadcount'];
$totalposts += $forum['replycount'];
}
}
}


My only addition is that you have to go into your AdminCP and update post counts for the changes to take affect. Try running gateway.php as well.

[kkiely]

Hi, could everyone who runs a board with USENET access reply as to
1. what news account you are using
2. how much it costs
3. how many GB you get/month
4. how reliaable it is

I have a few questions

1. What is your favorite news account?

2. If someone wants to run 2 different web sites using VBulleting with the USENET hack - can they use the same news feed and share the bandwidth?

3. Do most webmasters here find that the cost of the news account is worth the extra traffic, etc?

4. Can you make it so that anyone who posts to Usenet through your board has a signature in their posts that read something like

______________________________
"This was posted at www.ABC.com"

[Cold Steel]

1. I was using Giganews (http://www.giganews.com/customer/gn142452) (the link is my referral link). But for some reason, they stopped letting me post.

So I went to Teranews.com. They let me post once, and then none of them were getting through.

2. Giganews is $7.99 a month. Teranews is only a $3.95 setup fee.

3. Giganews is 2GB a month. Teranews is 50MB a day.

4. Both unreliable if you ask me, although I would think Giganews is more reliable than Teranews.

1. No favorite yet.

2. Yes.

3. Just started. Don't know yet.

4. Don't know.

I'm starting to think this is more trouble than it's worth.

Help!!!

[KevinM]

If I understand correctly you are suppose to add the first line of the footer between { } and it should strip the remaining lines -- is this correct?
:(

There are a couple of things to check first (I'm sure you have, but just to be sure...) as these are insensitive to any tolerance.

Double check that this bit
//strip the footer from emails only
if ($grouptype != 'news')
{
$message['text'] = stripfooter($message['text']);
}has been entered in the right place in gateway.php.

Lastly, check that the new Varname you entered in the controlpanel is called exactly "emailfooter" (without quotes).

Try to keep what you put between the {} as short as possible, but enough that it won't occur naturally in a normal message. What is the phrase you are trying to match? Are there any unusual characters that may cause the script to hiccup?