NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

[Princeton]

thanks kevin,
I did a clean install and everything is working as it should.

However the "preview" hack doesn't work. It's not fetching $postid.
Taken from partial error report
SET firstpostid =

It would be great to have an option to modify past posts that have "unwanted footers".

Hey, no one has responded to a post I created a few days ago ... in regards to $subject. It needs to be wrapped in htmlspecialchars() - - some titles may have html in it.

thanks again

[Cold Steel]

I figured out why giganews suspended my posting privileges.

Someone sent in a complaint about one of the posts that one my users made.

:(

[Princeton]

I figured out why giganews suspended my posting privileges.

Someone sent in a complaint about one of the posts that one my users made.

:(that's why it would be good to have an option to disable upload (to newsgroup/mlist) of postings made within the forum

[KevinM]

However the "preview" hack doesn't work.

?5 says it does :nervous:

[Princeton]

well ... it doesn't work for me. :ermm:

I'll look again ... it was working before.

[john_rsd]

I have just installed this mod and I am having trouble getting it to run.

First i tried NNTP and that failed.

So then i tried to collect from a pop3 mail account and I got this error when running gateway.php for the first time.

I am sure I did not make an error as the installation is pretty straight forward.

Gateway version 2.1.0 1 group(s) gatewayed. Could not connect to NNTP-serverNot connectedNot connected


There seems to have been a slight problem with the Forums database.
Please try again by pressing the refresh button in your browser.

An E-Mail has been dispatched to our Technical Staff, who you can also contact if the problem persists.

We apologise for any inconvenience.

If ran from scheduled task manager in vb3

Database error in vBulletin 3.0.1:

Invalid SQL: UPDATE nntp_settings
SET value =
WHERE varname = 'last_postid'
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 3

mysql error number: 1064

Date: Wednesday 23rd of June 2004 04:17:56 PM
Script: http://www.proteluser.com/bbs3/admincp/cronadmin.php?do=runcron&cronid=13
Referer: http://www.proteluser.com/bbs3/admincp/index.php
Username: Site_Admin
IP Address: 217.43.199.18

Otherwise the forums work OK

Any help apprciated.

[john_rsd]

Sorry, it was my own fault, email works now.

But I have another few questions on some things I would like to do.

The mail list that i am backing up requires registration to post and the backup i will make on the forum should be read only. i don't want to spam the list with test messages to try this one out so i thought i would ask.

If I simply change the forum permissions to prevent usergroups from being able to post/reply for that forum, should that be sufficient to stop forum users from using the email gateway to post messages to the mail list server?

Also when messages are delivered they sometimes are prefixed "RE:...." or "FW:...." Can these characters be stripped?
If I could strip things like FW: and RE: I could forward old messages to test threading etc (which does not seem to work).
Each post has a fixed name enclosed with suqare brackets always, like [listname]

Any help appreciated

[BustaCap]

Sorry, it was my own fault, email works now.


If I simply change the forum permissions to prevent usergroups from being able to post/reply for that forum, should that be sufficient to stop forum users from using the email gateway to post messages to the mail list server? Yes, that will work.

Also when messages are delivered they sometimes are prefixed "RE:...." or "FW:...." Can these characters be stripped?
If I could strip things like FW: and RE: I could forward old messages to test threading etc (which does not seem to work). Use KevinM's hack to thread properly and that will get rid of the "RE's".


Each post has a fixed name enclosed with suqare brackets always, like [listname] You can get rid of those by using the Replacement Variable Manager. Go to your Admin CP, select Styles and Templates, then Replacement Variable Manager. For the style you want this to apply to, select add new replacement variable, then enter the text you want removed (eg, [yourlistname]), and leave the replacement blank. That will do the trick. Note, the text isn't actually stripped, it's just hidden from view.

[Cold Steel]

that's why it would be good to have an option to disable upload (to newsgroup/mlist) of postings made within the forum
What is the point of having newsgroups in the forums, then? Just another content source?

[john_rsd]

Cheers Dave

I applied Kevins mod, but it seems erratic. As the mail list is not public and the posts spread reasonably evenly across the day I do not think its a server issue.

Screenshot is below.

A couple of posts were threaded correctly, then some were not, then another 2, then some were not.

Perhaps related to time received? I think the threaded mod should filter by subject no problem regardless of when it is received, if i read the functions correct, or is the filtering done after messsages are batch downloaded or per message. The mail server from which it collects the messages is the same as the site is hosted on, so no delays there.

you can see from the list that one subject was threaded OK (after Kevins mod) then another, but the posts after that were not, even although subject lines were identical.

Any ideas?

[BustaCap]

That's what my posts looked like before I applied Kevin's hack. Double check your gateway.php file and make sure you followed the directions exactly. Also, run gateway.php manually, copy and paste what the screen log shows, and post it here.

[Princeton]

What is the point of having newsgroups in the forums, then? Just another content source?
Some of the information in mlist is very valuable; however, if you or a member does something wrong you can be removed/banned. I would like the option to prevent posting to mlist for this very reason. A forum member will always be able to post to the forum - it just won't be sent to the public mlist.

ARCHIVE - for information only; some of the content found in newsgroup/mlist can answer posted questions on forum (similar threads)

[KevinM]

I'll look again ... it was working before.

If you still can't get it working, post the 10 lines from gateway.php just before this mini-hack and I'll have a look.

[KevinM]

I applied Kevins mod, but it seems erratic. As the mail list is not public and the posts spread reasonably evenly across the day I do not think its a server issue.


From your screenshot, that is exactly the problem that this mini-hack fixes. It picks these up by their subject title and threads correctly where the reference in the email header is missing.

Like BustaCap say, check the installation in gateway.php, and if you still have problems post the output here.

[Princeton]

I got it working ... thank you. :up:

[Princeton]

Kevin,
I never get this error when I call the file directly. However, I do occasionally get this error when Schedule Task runs the file.
Invalid SQL: INSERT INTO post
(postid, threadid, title, username, userid, dateline,
pagetext, allowsmilie, showsignature, ipaddress,

iconid, visible, isusenetpost, msgid, ref, parentid)
VALUES (NULL, ,
'',
'', 0,
'',
'', 1, 0,
'', 0, 1, 1,
'',
'',
+ 0)
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '
'',
'', 0,
'',
'', 1, 0,
'', 0, 1, 1
(I direct this to you (kevin) because it only happens when I use your mini hack.)

[john_rsd]

Kevin

Today it was working and I have not changed the gateway.php file.

Output from running from inside AdminCP

Gateway version 2.1.0 1 group(s) gatewayed. Successfully connected to the 'mail.rsdtv.com' pop3 server Retrieving 6 E-mails from the 'mail.rsdtv.com' pop3 server Getting message number 1: '[dxp] OT: Differential Stripline Impedance' from Jason Morgan. New thread. Getting message number 2: 'Re: [dxp] show footprint on schematic' from Don Lucas. Thread found by Subject Title. Getting message number 3: 'Re: [dxp] [dxp]: Junction Printing Error' from Don Lucas. Thread found by Subject Title. Getting message number 4: 'Re: [dxp] PCB manufacturing via internet?' from Don Lucas. Thread found by Subject Title. Getting message number 5: 'Re: [dxp] PCB manufacturing via internet?' from Richard Pender. Thread found by Subject Title. Getting message number 6: 'Re: [dxp] PCB manufacturing via internet?' from Don Lucas. Thread found by Subject Title.

I set this up as a scheduled task in the AdminCP to run every 15 minutes but it seems to have no effect, I have to push it manually to run. If I tried to run it directly from the browser I got an error that it was already running so I have set the check in AdminCP to 1 now.

However it seems the ones before failed to match (title '2004 simulator ops') see attachment

[KevinM]

However it seems the ones before failed to match (title '2004 simulator ops') see attachment

It looks to be working correctly. The thread that you mention looks like a duplicate post (it has no "Re: " in the title), so the correct action is to start a new thread.

It will only search for a match by subject title under the following circumstances:

1. There are no references in the header that match an exisiting thread

AND

2. There is at least one "Re: " in the subject title, which gives an indication that there should be a parent thread for this post.

The output from gateway.php which you have shown also indicates that it is working correctly.

[KevinM]

(I direct this to you (kevin) because it only happens when I use your mini hack.)

It's hard to tell from the sql query what is causing it to fail. If it ever happens when you are running gateway manually, make a copy & post the output which was running at the time.

Are you running the 'threading by subject' hack? There was an error in my first release which caused the $parentid not to generate, which would produce a error like the one you have. It was corrected with this version shortly afterwards: https://vborg.vbsupport.ru/showpost.php?p=522470&postcount=181

[john_rsd]

Kevin

Could it be that some users have just removed the subject prefix (or their mail clients have not added one?) . I have not used any other mail clients except Eudora or Outlook and these both addd it.

So in order for the threading function to be initiated it checks the first chars of the subject feild for matches on 'RE' ? Then parses an subject index for a match?

Guess its not case sensitive?

BTW, thanks for this really cool mod :)

[KevinM]

Kevin
So in order for the threading function to be initiated it checks the first chars of the subject feild for matches on 'RE' ? Then parses an subject index for a match?


The primary method of matching threads is by a reference number which is inthe headers of newsgroup posts and emails. This was written by Lierduh and is highly effective. All newsgroups thread pretty much perfectly just by this method.

Some mail groups (my personal experience is with Yahoo groups) do not reliably put the reference number in the header of the email, therefore I wrote a secondary check (case insensitive!) which checks again before starting a new thread. The "Re: " can appear anywhere in the subject title.

If you really, really have problems, you can find in the script where it searches for "Re: " and change it to "", and it will search every post that comes through for a subject match. This WILL slow things down, and will also match a thread from (say) 3 years ago if it has the same title. This is completely untested (and not recommended) as the basic threading will pick up most of them.

BTW, thanks for this really cool mod
Lierduh cooked the Sunday roast, I just made some gravy :cheeky:

[john_rsd]

Thanks Kevin

I undestand a little better now, so ill dig deeper.

the source of the list is a private server using Lyris, its not yahoo or anything like that.

I might try using"" as this list server only has 50 or so posts a day so is never busy enough to slow the server down.

Also if using "" it should only d/l message that are in the pop3 box so it might not cause a flood.

The roast and the gravy make a fine combo mate, sweet :)

[john_rsd]

Oh nuts

Just seen the glitch again

This time i ran the script directly

Gateway version 2.1.0 1 group(s) gatewayed.
Successfully connected to the 'mail.proteluser.com' pop3 server
Retrieving 4 E-mails from the 'mail.proteluser.com' pop3 server
Getting message number 1:
'Re: [dxp] OT: Differential Stripline Impedance' from Ian Wilson. New thread.

But this was a reply to an earlier one and has Re in it

Header from first messsage/thread

Return-Path: <bounce-dxp-3338@forums.altium.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 97607 invoked from network); 24 Jun 2004 17:25:39 -0000
Received: from unknown (HELO 203.10.94.132) (203.213.65.162)
by host109.ipowerweb.com with SMTP; 24 Jun 2004 17:25:39 -0000
Content-Class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-MimeOLE: Produced By Microsoft Exchange V6.0.6249.0
Subject: [dxp] OT: Differential Stripline Impedance
Date: Thu, 24 Jun 2004 18:18:34 +0100
Message-ID: <LYRIS-3338-45721-2004.06.25-03.13.39--john#rsdtv.com@forums.altium.com>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: OT: Differential Stripline Impedance
Thread-Index: AcRaEGUknFNjWsP/Qper0ZkkCMrRLA==
From: "Jason Morgan" <jason.morgan@citel.com>
To: "DXP Technical Forum" <dxp@forums.altium.com>
List-Unsubscribe: <mailto:leave-dxp-3338F@forums.altium.com>
Reply-To: "DXP Technical Forum" <dxp@forums.altium.com>

Header from reply messsage

Return-Path: <bounce-dxp-3338@forums.altium.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 69724 invoked from network); 24 Jun 2004 23:30:20 -0000
Received: from unknown (HELO 203.10.94.132) (203.213.65.162)
by host109.ipowerweb.com with SMTP; 24 Jun 2004 23:30:20 -0000
Message-Id: <LYRIS-3338-45752-2004.06.25-09.17.27--john#rsdtv.com@forums.altium.com>
X-Sender: i.wilson@considered.com.au@mail.considered.com.au@ 127.0.0.1
X-Mailer: QUALCOMM Windows Eudora Version 5.2.0.9
Date: Fri, 25 Jun 2004 09:25:28 +1000
To: "DXP Technical Forum" <dxp@forums.altium.com>
From: Ian Wilson <i.wilson@considered.com.au>
Subject: Re: [dxp] OT: Differential Stripline Impedance
In-Reply-To: <LYRIS-395-45721-2004.06.25-03.13.39--i.wilson#considered.c
om.au@forums.altium.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
List-Unsubscribe: <mailto:leave-dxp-3338F@forums.altium.com>
Reply-To: "DXP Technical Forum" <dxp@forums.altium.com>

Is this because the Message ID is different? Or is the filter being confused between ID's in the reply and only seeing the first one (different) and not the second one (match)

John

[vprp]

When I run gateway.php, I get the message:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

What's the problem?

I am trying to do this for the newsgroup rec.sport.golf

[Princeton]

When I run gateway.php, I get the message:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

What's the problem?

I am trying to do this for the newsgroup rec.sport.golf you are not allowed to access newsgroup via script or you have settings wrong (username/password if required).

I remommend you do a test first.
Use-------
- msnews.microsoft.com as SERVER
- microsoft.public.security as NEWSGROUP

NOTE: add 58700 as LAST MESSAGE or check the XREF # in header of last message found in usergroup.

[vprp]

I tried that microsoft newsgroup and i got the same message. should i try to reinstall?

[Princeton]

I still get the message.
And ... yes, I am using the code posted on that POST.

Would you care to see the file?
I've made some "visual/html" modifcations to the file ... however, everything is clearly marked where the changes have been made (your mini-hacks , etc,)

[Princeton]

I tried that microsoft newsgroup and i got the same message. should i try to reinstall?yes, start from scratch ... make sure you read the instructions carefully

[john_rsd]

Anyone else using email delivery method and found problems with the script crashing?

I have been sending most things I could think of at the script and it has crashed twice, big time.

The first time was when a message was forwarded to the lists, the forward contained multiple forwarded messages and no subject. In this case the first message was part entered into the database (with no subject filled in) but nothing on the rest.

The second time was when a message was sent to the lists with no subject, perhaps it was still recovering from the first crash.

I might go back and try the nntp feed instead of email.

[Imhotep]

:ermm: My forums stopped grabbing new Usenet messages on 6/21. I'm trying to finger out exactly where the log jam has occured. My cron job is still going off like clockwork.

This is the entire content of my cron results minus the header garbage.

Gateway version 2.1.0 4 group(s) gatewayed.

Connecting to server, server says: 200 News.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com

Info for comp.graphics.apps.lightwave at news-40.giganews.com:
211 12589 237200 249788 comp.graphics.apps.lightwave

Getting message number 249690:


Any ideas as to what is happening? It looks like it only hits one group... then stops. ? :ermm:

[AWS]

:ermm: My forums stopped grabbing new Usenet messages on 6/21. I'm trying to finger out exactly where the log jam has occured. My cron job is still going off like clockwork.

This is the entire content of my cron results minus the header garbage.

Gateway version 2.1.0 4 group(s) gatewayed.

Connecting to server, server says: 200 News.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com

Info for comp.graphics.apps.lightwave at news-40.giganews.com:
211 12589 237200 249788 comp.graphics.apps.lightwave

Getting message number 249690:


Any ideas as to what is happening? It looks like it only hits one group... then stops. ? :ermm:
Getting message number 249690:
The message is not there and the script thinks it has reached the end. Go to newsgroup settings and change the id number and run the script again.

[Imhotep]

awesome! thank you very much

[kmike]

Getting message number 249690:

The message is not there and the script thinks it has reached the end. Go to newsgroup settings and change the id number and run the script again.

While you posted good workaround for this, the root of this problem is actually completely different. We encountered the same problem at about the same time, and the real cause was PHP crashing on the huge spam messages crossposted to several newsgroups. Here's an example (5000+ lines!):

http://groups.google.com/groups?dq=&start=50&hl=en&lr=&ie=UTF-8&group=rec.models.rc.air&selm=62c72772.0406210145.2f251ad5%40posting.google .com

The quick fix is to limit amount of lines we get from the news server per message.
After the line 128 of includes/nntp.php, which reads:
$post = '';
add this line:

$lc = 0;

Then find this code fragment starting on line 134:

if ($line == ".") {
break;
} else {
$post .= $line ."\n";
}
Change it to:

if ($line == ".") {
break;
} else if ($lc++ < 300) {
$post .= $line ."\n";
}

Change 300 here to the amount of lines from news posting you want to allow to get into forum's message.

Enjoy :)

[Princeton]

A word (title or message) that long will break a fixed width forum. It will even make a liquid layout appear "bad".

The fetch_word_wrapped_string function does not work on long words with periods, commas, special characters. This is a bug that has been reported.

Until there's a fix ... you should use the fetch_trimmed_title on titles (showthread/forumdisplay). fetch_trimmed_title function works with the $vboptions[wordwrap] settings.

If you do not want to use the fetch_trimmed_title function...
I use this on my cms: (mainly for right/left column):
function gtp_strip_title($str, $start, $end) {
if (strlen($str) > $start){
$str = substr($str, 0, $end)."...";
}
return $str;
}

I suggest having an option to remove (not insert in db) messages with long titles.
All messages with long words (titles) are 100% spam.

[kkiely]

I am a newbie. With this script I keep hearing people saying to "run a cron job"

1. What is "cron"? I assume it is some type of scheduling task system.
2. What will it do with this script? does it auto-import the posts?

[Odoin]

A cron is where you go downtown and play the shell game and cron people out of their money...

Just kidding... many providers that offer a Control Panel have some way to setup a cron job. You are right what a cron is... It runs a script or program and a set time.

In this case you want it to run the gateway.php script to pick up and send new NNTP messages. You could do this your self by pointing your browser to the gateway.php file... however if you forget or are gone for a few days the new posts will not be updated. Another factor is on large NNTP groups it takes a lot more time if ran once a day rather then every hour.

[chopperp1]

Hello to All
I'm trying to get this hack working on company's web server. Everything was pretty much straight forward till I got to the point of running gateway.php. That's what I get:
"...pop3 gateway

Gateway version 2.1.0 1 group(s) gatewayed. Bad User Name!Retrieving Protocol E-mails from the 'dl360mail3.yamner.com' pop3 server ..."

I tripple checked all settings and everything looks good. User account for pop3 is correct including password. Web server talks to mail via local subnet (no filtering there). Can't figure out why this thing can't log in.
any clues? :ermm:

[lierduh]

Hello to All
I'm trying to get this hack working on company's web server. Everything was pretty much straight forward till I got to the point of running gateway.php. That's what I get:
"...pop3 gateway

Gateway version 2.1.0 1 group(s) gatewayed. Bad User Name!Retrieving Protocol E-mails from the 'dl360mail3.yamner.com' pop3 server ..."

I tripple checked all settings and everything looks good. User account for pop3 is correct including password. Web server talks to mail via local subnet (no filtering there). Can't figure out why this thing can't log in.
any clues? :ermm:

The best way is to log into your web server and do a telnet from there:

#telent dl360mail3.yamner.com 110
#USER your-username
#PASS your-password
#STAT

I suspect your pop3 server does not like the pop3 access from a local subnet. You gave a public domain name, yet accessing through a local private net. Either routing or pop3 server's "allowed access" can cause your problem.

[john_rsd]

Ive been testing this for a few days now.

I have found that the script has crashed twice causing the following error

Database error in vBulletin 3.0.1:

Invalid SQL: INSERT INTO attachment SET dateline = '1088526838',
filename = '',
filedata = 'GIF89a?\0,\0?\0

<<snipped here>>
thumbnail = '',
filesize = + 0,
postid = 202
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '+ 0,
postid = 202' at line 5

mysql error number: 1064

Date: Tuesday 29th of June 2004 09:33:59 AM
Script: http://proteluser.com/bbs3/gateway.php
Referer:
Username: Site_Admin

But the script seems to have went so far then went pear shaped, at this point threading by title was completely non-effective and the 22 messages were not threaded at all despite having subject headers matching exactly. See screenshot attached.

There also seems an issue with handling possible EOL characters as i got a message like this inserted to the forums. Notice the '=' inserted at the end of a line before a return.

I've just been able to use DXP for a complete job and feel that I can now c=
omment on a few things that I've found.

Of course some of this may have a new 'DXP' way of doing things that I don'=
t know of yet.

In 99Se I could 'RMB' cut, copy or paste when editing many parameters such =
as drawing number in the schematic editor, I can't do this anymore and inst=
ead must use the more cumbersome ctrl C / ctrl V method.

Usually when editing text I want to change the text itself - in 99SE, doubl=
e clicking on a text string opened an editor panel with the 'text' selected=
.. This was very good for text editing as all I needed to do was to double c=
lick, type in the new value and then enter. Now I double click and then hav=
e to find and select the 'value' to change.


It should have looked like this, at least this is how outlook displays it in the mail box.

I've just been able to use DXP for a complete job and feel that I can now comment on a few things that I've found.

Of course some of this may have a new 'DXP' way of doing things that I don't know of yet.

In 99Se I could 'RMB' cut, copy or paste when editing many parameters such as drawing number in the schematic editor, I can't do this anymore and instead must use the more cumbersome ctrl C / ctrl V method.

Usually when editing text I want to change the text itself - in 99SE, double clicking on a text string opened an editor panel with the 'text' selected. This was very good for text editing as all I needed to do was to double click, type in the new value and then enter. Now I double click and then have to find and select the 'value' to change.

This message was sent as plain text.

This was the second message inserted in that batch. The first message is uneffected and was threaded correctly.

Is there any compatability issues with mysql or php versions ???

Any help appreciated

[john_rsd]

After looking at the messages retreived from the pop3 mailbox on a different account to the one used for gatweway.php i noticed that there was a few messages originally in HTML and with gif as attachments.

Could the above error be an issue with trying to insert the attachment?

Attachments are usually frowned upon on mailing lists which is why I failed to notice it when making my last post.

Is there a quick fix to simply drop attachments completely?

[lierduh]

After looking at the messages retreived from the pop3 mailbox on a different account to the one used for gatweway.php i noticed that there was a few messages originally in HTML and with gif as attachments.

Could the above error be an issue with trying to insert the attachment?

Attachments are usually frowned upon on mailing lists which is why I failed to notice it when making my last post.

Is there a quick fix to simply drop attachments completely?

Are you using the original gateway.php or you have installed Kevin's mini-hacks? Not easy to find the problem if we are not taling about the same script. Kevin has re-written the attachment routine which I will include with next version.

[alesis404]

Hello, in the read me you mention to change this in the "nav" template if you would like usenet posts from not showing up in new postings

Find:
<td class="vbmenu_control"><a
href="search.php?$session[sessionurl]do=getnew">$vbphrase[new_posts]</a></td>

Change it to:

<td class="vbmenu_control"><a
href="search.php?$session[sessionurl]do=getnew&exclude=22,23,24,25,26,27">$vbphrase[new_posts]</a></td>

I have searched my *nav* template and the closet match looks like this

<td class="vbmenu_control"><a href="search.php?$session[sessionurl]do=getnew" accesskey="2">$vbphrase[new_posts_nav]</a></td>

*Perhaps my templates are newer than when this was released? Can someone please confirm if the code for editing "nav" template has indeed changed ,thanks.

[john_rsd]

Are you using the original gateway.php or you have installed Kevin's mini-hacks? Not easy to find the problem if we are not taling about the same script. Kevin has re-written the attachment routine which I will include with next version.

Sorry for missing information .

I used the original gateway.php from the first post.

I added Kevins mod for thread by subject. The forum is a test one so does not have users accessing at same time as script runs. Script was run directly from browser, the forums only content is the messages imported from the mailing list. Max 20 messages per day, max 10-15 messges per batch.

[lierduh]

Hello, in the read me you mention to change this in the "nav" template if you would like usenet posts from not showing up in new postings



<td class="vbmenu_control"><a href="search.php?$session[sessionurl]do=getnew" accesskey="2">$vbphrase[new_posts_nav]</a></td>

*Perhaps my templates are newer than when this was released? Can someone please confirm if the code for editing "nav" template has indeed changed ,thanks.

This is the right one. It used to be "new_posts", the latest vB3 has changed to "new_posts_nav".

[alesis404]

Thank's lierduh!

[chopperp1]

Thanks for quick reply.
I already tried telnet from webserver- no problems. I used FQDN because we host our internal dns servers, anyway I get same error when I specify by IP.
__________________________________________________ _________________
+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.co
m) ready.
+OK
+OK User successfully logged on.
+OK 21 32830
__________________________________________________ __________________

Is there anything else I should look for?
Thx

[lierduh]

Sorry for missing information .

I used the original gateway.php from the first post.

I added Kevins mod for thread by subject. The forum is a test one so does not have users accessing at same time as script runs. Script was run directly from browser, the forums only content is the messages imported from the mailing list. Max 20 messages per day, max 10-15 messges per batch.

I have merged KevinM's attachment routine into the gateway.php. Please try out.

You now need to run

http://forums.yourdomain.com/gateway.php?debug=1

Without the debug=1, the script won't output anything on the screen.

[lierduh]

Thanks for quick reply.
I already tried telnet from webserver- no problems. I used FQDN because we host our internal dns servers, anyway I get same error when I specify by IP.
__________________________________________________ _________________
+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.co
m) ready.
+OK
+OK User successfully logged on.
+OK 21 32830
__________________________________________________ __________________

Is there anything else I should look for?
Thx

Please include the exact letters you typed during the telnet session and the response the server returned.

[chopperp1]

Please include the exact letters you typed during the telnet session and the response the server returned.
+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.co
m) ready.
+OK
USER <domain>\<user>
PASS <password>
+OK User successfully logged on.
+OK 21 32830

I also tried forward slash or user@domain.com - didn't work

[lierduh]

+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.co
m) ready.
+OK
USER <domain>\<user>
PASS <password>
+OK User successfully logged on.
+OK 21 32830

I also tried forward slash or user@domain.com - didn't work

How long is <domain>\<user> ?

The user name field is limited to 32 characters. If you need longer, you can modify the database to increase the char length.

If you can access the DB directly, eg. using phpmyadmin, check what exactly is stored in the DB for the user name.

Looks like it is either the length or the "\" causing the issue.

[chopperp1]

How long is <domain>\<user> ?

The user name field is limited to 32 characters. If you need longer, you can modify the database to increase the char length.

If you can access the DB directly, eg. using phpmyadmin, check what exactly is stored in the DB for the user name.

Looks like it is either the length or the "\" causing the issue.


domain as you noticed is "yamner' and user name is 5 characters. I just checked "nntp_groups" table in mysql and I don't see any issues there (server lenght=64; username=32; password=32) all values are correct . :ermm:
But I noticed one thing, when I go to admincp>Execute SQL Query I get the message "You are not authorized to execute SQL queries"- I'm logged in as a superadmin. Do you think this might be my problem? If yes, what file should I modify?
Thx

[john_rsd]

I have merged KevinM's attachment routine into the gateway.php. Please try out.

You now need to run

http://forums.yourdomain.com/gateway.php?debug=1

Without the debug=1, the script won't output anything on the screen.

Cool :)

Ill try it, is this just the original php file + Kevins mod or
an updated gateway.php + Kevins mod?

Ill run it later and see if I can recreate it.

[KevinM]

Cool :)

Ill try it, is this just the original php file + Kevins mod or
an updated gateway.php + Kevins mod?



I updated how part of the script that handles the attachments works (this is not related to any previous mini-hacks I have released). There are 3 minor improvements:
1. The script will now handle attachments as 'save as files' if that is how your version of vb is set-up.
2. It checks attachment permissions against the forum (allow attachments or not) and extension types (as set up in control panel).
3. On some flavours of php installations the making of thumbnails was problematic. It now follows vb's standard way of thumbnail creation.

There may be other improvements made by Lierduh ;)

[vprp]

Add a smallint field on the database record such as "is_usenet" and set it to '1' for the forums that are.

How do you add a smallint field?

[alesis404]

Can someone please tell me how to properly stop attachments being made with posts, I get constant errors sent to my email everytime gateway.php run's , I tried to comment out a small piece in gateway.php and this stopped the attachments and the posts so I must have did something wrong. I re-edited and am getting attachment errors again.

[john_rsd]

I updated how part of the script that handles the attachments works (this is not related to any previous mini-hacks I have released). There are 3 minor improvements:
1. The script will now handle attachments as 'save as files' if that is how your version of vb is set-up.
2. It checks attachment permissions against the forum (allow attachments or not) and extension types (as set up in control panel).
3. On some flavours of php installations the making of thumbnails was problematic. It now follows vb's standard way of thumbnail creation.

There may be other improvements made by Lierduh ;)

Kevin

So is the mini hack for thread by subject enhancement is now integrated or will it need updated again for this version of gateway.php?

[john_rsd]

Its Ok Kevin, I answered my own question (just tried it)

If i add the thread by subject mini hack I get

Parse error: parse error in /home/protelus/public_html/bbs3/gateway.php on line 401

Without it its fine.

[KevinM]

Its Ok Kevin, I answered my own question (just tried it)

If i add the thread by subject mini hack I get

Parse error: parse error in /home/protelus/public_html/bbs3/gateway.php on line 401

Without it its fine.

I will update it when Lierduh makes an 'official' release.

[lierduh]

domain as you noticed is "yamner' and user name is 5 characters. I just checked "nntp_groups" table in mysql and I don't see any issues there (server lenght=64; username=32; password=32) all values are correct . :ermm:
But I noticed one thing, when I go to admincp>Execute SQL Query I get the message "You are not authorized to execute SQL queries"- I'm logged in as a superadmin. Do you think this might be my problem? If yes, what file should I modify?
Thx

With pop.php file, please find:

$this->_write("USER $this->username");

and insert the following line before that:

echo "=========USER Name: " . $this->username . " ==========";

This will show the user name entered by the script. I want to confirm if the right username is used by the script at this stage.

Also with gateway.php, find:

$pop_connect = $pop3->connect();

and add the following line after this:

logging($pop3->showlog());

This will show the entire pop3 connection log. No fix at this stage, just need to find out the reason behind this first.

[lierduh]

Can someone please tell me how to properly stop attachments being made with posts, I get constant errors sent to my email everytime gateway.php run's , I tried to comment out a small piece in gateway.php and this stopped the attachments and the posts so I must have did something wrong. I re-edited and am getting attachment errors again.

Have you tried the script I posted in last page?

[john_rsd]

I will update it when Lierduh makes an 'official' release.

Cool :cool:

Thanks

[alesis404]

Have you tried the script I posted in last page?

Lierduh are you referring to post #297 of this thread? If so do i just upload the new gateway.php and this will stop the attachments or will stop the errors?

[john_rsd]

lierduh

I am still having trouble with the new gateway.php script with file attachments.

How aware is the script about which type of attachment it is?

The forum has no file upload permissions. So I tested this by emailing the account which is used for gateway with a big file and it crashed (I suspect timeouts)

So i send it a very small file to test this theory, it was a small zip file only 36kB and instead of dropping the attachment as it should, it inserted it as text below the final characters of the message, see screenshot atatched.

Anything you need me to do for you to trap what is going on?

[chopperp1]

With pop.php file, please find:

$this->_write("USER $this->username");

and insert the following line before that:

echo "=========USER Name: " . $this->username . " ==========";

This will show the user name entered by the script. I want to confirm if the right username is used by the script at this stage.

Also with gateway.php, find:

$pop_connect = $pop3->connect();

and add the following line after this:

logging($pop3->showlog());

This will show the entire pop3 connection log. No fix at this stage, just need to find out the reason behind this first.

Sorry for late response, I was away for few days.
Here's what I got after adding those lines
"...pop3 gateway

Gateway version 2.1.0 1 group(s) gatewayed. =========USER Name: yamner\******==========+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.com) ready. USER yamner\****** -ERR Protocol error. Bad User Name!Retrieving Protocol E-mails from the '209.178.209.17' pop3 server ..."

I had to hide a user name, but real log shows correct one. Also, as you can see, this time I used external IP for pop server - different isp than board is hosting on...

[trackpads]

What is the maximum characters that we can put in the Killfile exclude field? I think I ran out already, is there a way to expand it?

Thanks again!

-Jason

[kall]

Whoooops.

When upgrading to 3.0.2, don't forget that the script will repost all of the threads already in your selected forum.

*starts composing apology emails*

[KevinM]

What is the maximum characters that we can put in the Killfile exclude field? I think I ran out already, is there a way to expand it?

It's limited to 128 characters (which is what the original scripts default settings are), and, like yourself I found it too small.

Run the following sql query and you can have up to 64000 characters (give or take!)

ALTER TABLE `nntp_settings` CHANGE `value` `value` MEDIUMTEXT DEFAULT NULL ;


It's possible that in future upgrades of gateway.php the script may revert back to the original settings (as it auto-installs), so if there are any future gateway updates that you install you will have to run this query again after installing (and keep a copy of what's in the killfile beforehand as this may be erased).

[kall]

And here's what one nice person told me:


News has a well-defined protocol (NNTP - RFC977 etc) and if you have a program which retrieves and/or send articles then it should retain the very important "Message-ID" header which each article contains. Then if it accidentally tried to post an old article it would get rejected because of the conflicting Message-ID (unique for each article). Otherwise it would generate a unique Message-ID for a new article and present it with the "IHAVE" nntp command.


*sighs*

[AWS]

Whoooops.

When upgrading to 3.0.2, don't forget that the script will repost all of the threads already in your selected forum.

*starts composing apology emails*
Are you saying when you upgraded to 3.0.2 the gateway script sent all your posts to usenet from the usenet forums?

How can this happen?
You have to run gateway.php to upload new posts and it has the message id already set so duplicates shouldn't be uploaded. The only way I can see this happening is if the upgrade to 3.0.2 made all posts look as new posts or somehow the upgrade lost all id numbers in the db.

[kall]

The only way I can see this happening is if the upgrade to 3.0.2 made all posts look as new posts or somehow the upgrade lost all id numbers in the db.

*nods*

That's what happened, because I ended up doing the upgrade by installing afresh and importing all of the posts etc using Impex..obviously losing the columns that denoted the threads as being 'old'.

[AWS]

*nods*

That's what happened, because I ended up doing the upgrade by installing afresh and importing all of the posts etc using Impex..obviously losing the columns that denoted the threads as being 'old'.
OOPs. I sure wouldn't want to be on the receiving end of the mail you are going to receive.

[kall]

*nods*

I think I got most of it last night....a lot of it Spam itself, ironically. :)

Posting a public apology in the newsgroup and on my homepage is about all I can do tho unfortunately.

[Cluster One]

I installed the script, tried running gateway.php and it keeps asking my newserver for more permission, it gets my username, but for some reason will not get my password. I've checked the SQL database and the password is there, but when I check in the admincp panel, its not displayed.

Am I doing something wrong? I followed the installation instructions completely.

[Cluster One]

Okay I fixed my own problem, apparently an SQL error the field PASSWORD was in capitals and i altered it to lowercase now it accepts my password and is now retrieving messages as we speak.

[lierduh]

+OK Microsoft Exchange 2000 POP3 server version 6.0.6249.0 (dl360mail3.yamner.co
m) ready.
+OK
USER <domain>\<user>
PASS <password>
+OK User successfully logged on.
+OK 21 32830

I also tried forward slash or user@domain.com - didn't work

I noticed you have an extra +OK there before the USER command. Do you need to type a Return?

Also try this:

In pop.php around line 108, find:

fwrite($this->pop_connect, $message . "\n");

Change the "\n" to "\r\n"

[trackpads]

Has anyone figured a way to completely stop attachments from being imported? I just want my regular users to be able to post but not pull in any from the newsgroups at all.

Thanks!

-Jason

[kmike]

Has anyone figured a way to completely stop attachments from being imported? I just want my regular users to be able to post but not pull in any from the newsgroups at all.

Thanks!

-Jason
Just comment out "process_attachments()" call, or add "return;" just after "function process_attachments(...) {" line.

[trackpads]

Just comment out "process_attachments()" call, or add "return;" just after "function process_attachments(...) {" line.
Thanks but can you tell me which line and what exactly to do? I found three different areas that I think you mean but I am not sure.

Thanks again for the help!

-Jason

[chopperp1]

I noticed you have an extra +OK there before the USER command. Do you need to type a Return?

Also try this:

In pop.php around line 108, find:

fwrite($this->pop_connect, $message . "\n");

Change the "\n" to "\r\n"

GREAT it works!
One more question, How can I make all messages pulled from pop to appear as posts in the thread, not as threads in the forum?
Thanks,

[fourflush]

I implemented the gateway on my forums and it is working well, except that many messages are not threaded. There was a fix posted earlier, but I believe that it can only be used for mailing lists. Can it also be used to correct the problem for USENET? You can see it here: http://www.theaudioforum.com

Thanks!!

[BustaCap]

I haven't had any problems since I've been using the hack, but today I added a new newsgroup and got this error:

Database error in vBulletin 3.0.1:

Invalid SQL: SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
post.title AS posttitle, thread.title AS threadtitle
FROM post AS post LEFT JOIN thread AS thread
ON (post.threadid = thread.threadid)
WHERE post.isusenetpost = 0 AND
post.postid > 61294 AND
thread.forumid = 34
mysql error: Got error 134 from table handler

mysql error number: 1030

Date: Sunday 11th of July 2004 01:50:23 PM
Script: http://www.mysite.com/forum/gateway.php (http://www.lagwtf.com/forum/gateway.php)
Referer:
Username: Unregistered
IP Address: xx.xx.xx.xx
After I got this error, the other two forums that are importing newsgroups stopped updating. Also, for some reason I couldn't connect to my admincp (even though I could for another site running on the same server). And just now, I can't even connect to the forum. I get this error message:

Database error in vBulletin :

Link-ID == false, connect failed
mysql error:

mysql error number: 0

Date: Sunday 11th of July 2004 04:11:01 PM
Script: http://mysite.com/forum/ (http://lagwtf.com/forum/)
Referer:
IP Address: xx.xx.xx.x
I had been running gateway.php via a cronjob in my cpanel, but I was worried that gateway.php might have been a runaway job so I deleted it. However, I don't think that actually shut it down.

Any ideas what could be causing this?

[Princeton]

Still waiting for an update. :)

[lierduh]

I haven't had any problems since I've been using the hack, but today I added a new newsgroup and got this error:


Any ideas what could be causing this?

One of your table is corrupted. The reason could be a lot of things. One thing the gateway can do is pushing the server to the limit when importing large amount of data, ie. the initial importing for a newsgroup. That should not cause your database to corrupt though.

http://www.google.com.au/search?sourceid=navclient&ie=UTF-8&oe=UTF-8&q=Got+error+134+from+table+handler

[rinkrat]

I'm getting prompted to download gateway.php when I try and open it in the browser. Is this what it's supposed to do?

[rinkrat]

I have this script installed with a news feed but when I run go to mysite.com/gateway.php why does it have a pop up that asks me to down the file with the list of posts that it is importing?

Also, can someone cut and paste me their cron script that they use to update it? I cant get mine to work


This is what I'm getting. Now what do we do next?

[lierduh]

This is what I'm getting. Now what do we do next?

Please refer to the third post on the first page.

[john_rsd]

lierduh

Did you have any ideas on this one?

https://vborg.vbsupport.ru/showpost.php?p=529039&postcount=315

[lierduh]

lierduh

Did you have any ideas on this one?

https://vborg.vbsupport.ru/showpost.php?p=529039&postcount=315

The attachment was not decoded.

If you can post the sample email, I will have a look. I need the email in raw format, you can delete the middle of the attachment. I mainly need to analyse the mime containers.

[john_rsd]

The attachment was not decoded.

If you can post the sample email, I will have a look. I need the email in raw format, you can delete the middle of the attachment. I mainly need to analyse the mime containers.

Would it be better if I resend the email directly to you with attachment?
The file is only very small.

I can go to outbox and resend and just change destination address, that way you can get 100% same as I send.

The email path was from my home PC, directly to the mail server for the domain used (same server) and gatweway.php will collect it locally. Or i can place an email address from you as a forward and you can get the output directly from the server.

The emails for the address used are not held/stored at the server

If it helps the file was packed with winrar, but I use .zip extension as more people understand zip (can be opened in WinXp/zip compatible).

[allan grossman]

lierduh, you've done it again ;)

I installed the latest version and it's got all the options I need - so I don't have to hack gateway.php to get things the way I want them. Works just great.

I especially appreciate the 'no altering vB files' option for the control panel. I've decided with vB3 I'm not installing features that alter vB files - well, I did add a oneliner for photopost integration but that's all ;)

Being able to run nntp_settings and nntp_groups helps me out a lot.

Thanks again -

[lierduh]

Would it be better if I resend the email directly to you with attachment?
The file is only very small.

If it helps the file was packed with winrar, but I use .zip extension as more people understand zip (can be opened in WinXp/zip compatible).

Please post to the forum instead of emailing me. I only need the header, the beginning and the ending of the attachment. It does not matter what file extension you have. As long as they are correctly MIME coded, it should be decoded properly. What email program do you use?

Which version of the script are you using? did you hack any part of the script before you had the attachment inserted as normal post?

[john_rsd]

Please post to the forum instead of emailing me. I only need the header, the beginning and the ending of the attachment. It does not matter what file extension you have. As long as they are correctly MIME coded, it should be decoded properly. What email program do you use?

Which version of the script are you using? did you hack any part of the script before you had the attachment inserted as normal post?

OK, Ill see if I can do that shortly.

The Email was sent from Outlook 2003 (11.5608.5703)

I used the last version of the script that you posted which you advised.

I did not hack any part of the script.

The forum has posting of attachments disabled.

Ill go off and send the email to myself now to get the headers :nervous:

[john_rsd]

Message headers as received

Return-Path: <john@rsdtv.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 72330 invoked from network); 14 Jul 2004 19:43:10 -0000
Received: from unknown (HELO johnhomepc) (81.154.91.66)
by host109.ipowerweb.com with SMTP; 14 Jul 2004 19:43:10 -0000
Reply-To: <john@rsdtv.com>
From: "John A. Ross [RSDTV]" <john@rsdtv.com>
To: <john@rsdtv.com>
Subject: File for PP data
Date: Wed, 14 Jul 2004 20:43:09 +0100
Organization:
MIME-Version: 1.0
Content-Type: multipart/mixed;
boundary="----=_NextPart_000_000F_01C469E3.2DABC120"
X-Mailer: Microsoft Office Outlook, Build 11.0.5510
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
Thread-Index: AcRhY0z/k+oCtlpYSxG0N5CkzFFbWA==

The script that is running is this one

https://vborg.vbsupport.ru/showpost.php?p=527881&postcount=297

[john_rsd]

Dear lierduh

I am having some more troubles, i guess it might be associated with the mime decoding and the handling of line wraps. Possibly you have some ideas already because of the attachment issue.

Example

Message at my inbox

> Thank St?phane, a very good article. I'm getting there (an all
> nighter on an urgent job helped...), DXP2004 is getting better by the
> minute!
>
> Regards
>

Message in forum

>=20
>=20
> Thank St=E9phane, a very good article. I'm getting there (an=20
> all nighter
> on an urgent job helped...), DXP2004 is getting better by the minute!
>=20
> Regards
>=20

Headers

Return-Path: <bounce-dxp-xxxx@forums.altium.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 25816 invoked from network); 14 Jul 2004 23:04:10 -0000
Received: from unknown (HELO 203.10.94.132) (203.213.65.162)
by host109.ipowerweb.com with SMTP; 14 Jul 2004 23:04:10 -0000
From: "xxxx xxxx" <xxxx.xxxx@xxxxx.xxx>
To: "DXP Technical Forum" <dxp@forums.altium.com>
Subject: Re: [dxp] Editing component pads and holes
Date: Thu, 15 Jul 2004 11:06:52 +1200
Message-ID: <LYRIS-3338-48461-2004.07.15-08.55.49--john#rsdtv.com@forums.altium.com>
MIME-Version: 1.0
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
X-Priority: 3 (Normal)
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook, Build 10.0.2616
In-Reply-To: <LYRIS-4669-48457-2004.07.15-08.48.41--xxxxx.xxxx#xxxx.xxx@forums.altium.com>
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1441
Importance: Normal
List-Unsubscribe: <mailto:leave-dxp-xxxx@forums.altium.com>
Reply-To: "DXP Technical Forum" <dxp@forums.altium.com>

This is now a regular occurance with the test forums. Every other post looks like this.

The isue seems to be LF/CR and also bad handling of charset because of the missing "?"

Do you want me to replace the last version of gateway.php you supplied with the original (from first post) and see if the issue is still present?

Any help appreciated

[john_rsd]

This is very strange now

I just did another update

What was sent


Subject : Re: [PEDA] SOT-23 pinout

Pins 2&3 reversed???

This is my default:

3
1 2

I think I've seen 1&2 reversed occasionally.

The only time I use something else is with SOT-23 diode & LEDs, where I use A&K & have an appropriate modified footprint.


----- Original Message -----
From: xxxxxxxxxx
To: xxxxxxxxxx Sent: Tuesday, July 13, 2004 11:05 PM
Subject: [PEDA] SOT-23 pinout


When you organise your footprints, what pinout do you use for SOT-23's?

Do you organise the pins...
-like shown in the attachment?
-or with pins 2&3 reversed?

The bigger question really is: Where do you derive your pin numbering
guidance from?

I find that some datasheets just show the pin names against the actual
pins (& cutout the middleman of *numbers*) thus avoiding the issue
altogether. This doesn't help those of us who want to be able to use a
common footprint across multiple designs.

One answer is "define your own company standard", but if I chose
against an industry standard, that'd be a bit silly wouldn't it.

Find local movie times and trailers on Yahoo! Movies.
http://au.movies.yahoo.com



What i got was this a subject and no message body


Subject : Re: [PEDA] SOT-23 pinout



Headers

Return-Path: <error@techservinc.com>
Delivered-To: rsdtvcom-john@rsdtv.com
Received: (qmail 63335 invoked from network); 14 Jul 2004 23:43:59 -0000
Received: from unknown (HELO mail.techservinc.com) (67.115.134.89)
by host109.ipowerweb.com with SMTP; 14 Jul 2004 23:43:59 -0000
Received: from techservinc.com ([192.168.0.1])
by "DOLIST Server Expert 3.4 B2157" <forums@techservinc.com>
with SMTP id <D0000877213.OUT>; Wed, 14 Jul 2004 16:41:54 -0700
Envelope-to: proteledaforum@techservinc.com
Delivery-date: Wed, 14 Jul 2004 18:41:19 -0500
Received: from [209.226.175.10] (helo=tomts4-srv.bellnexxia.net)
by esc18.midphase.com with esmtp (Exim 4.34)
id 1BktNH-0004r8-Kp
for proteledaforum@techservinc.com; Wed, 14 Jul 2004 18:41:19 -0500
Received: from VergentBertha ([67.71.75.76]) by tomts4-srv.bellnexxia.net
(InterMail vM.5.01.05.32 201-253-122-126-132-20030307) with ESMTP
id <20040714234114.BVMY14414.tomts4-srv.bellnexxia.net@VergentBertha>
for <proteledaforum@techservinc.com>;
Wed, 14 Jul 2004 19:41:14 -0400
Message-ID: <001201c469fc$07338af0$6401a8c0@VergentBertha>
From: "Brian Guralnick" <xxxxxxxxx@xxxxxxx.xx>
To: "Protel EDA Forum" <proteledaforum@techservinc.com>
References: <20040714030517.54866.qmail@web50902.mail.yahoo.com>
Date: Wed, 14 Jul 2004 19:41:03 -0400
Organization: Vergent Technology Inc.
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="----=_NextPart_000_000F_01C469DA.7FB1EB10"
X-Priority: 3
X-MSMail-Priority: Normal
X-Mailer: Microsoft Outlook Express 6.00.2800.1409
X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1409
List-Server: DOLIST Server Expert 3.4 B2157
List-Help: <mailto:forums@techservinc.com?body=help%20proteled aforum>
List-Unsubscribe: <mailto:forums@techservinc.com?body=leave%20protele daforum>
List-Subscribe: <mailto:forums@techservinc.com?body=join%20proteled aforum>
List-Post: <mailto:proteledaforum@techservinc.com>
List-Owner: <mailto:forumadministrator@techservinc.com>
List-Archive: <mailto:forums@techservinc.com?body=search%20protel edaforum%20date%202004-07-14>
List-Info: Forum for discussion of Protel EDA, related software, & Design
X-Tracking: #proteledaforum@techservinc.com#
X-DOLIST-Unsubscribe: To leave, send 'leave proteledaforum' to 'forums@techservinc.com'
Subject: Re: [PEDA] SOT-23 pinout
Resent-From: proteledaforum@techservinc.com
Errors-To: error@techservinc.com
Return-Path: error@techservinc.com
Reply-To: "Protel EDA Forum" <proteledaforum@techservinc.com>
X-DOListmember: #john@rsdtv.com#

[cyc]

Hi

Great hack, nice and easy to setup!

I have one problem though, in several groups I get an error on the odd message that looks like this:


Invalid SQL: SELECT post.postid FROM thread AS thread, post AS post
WHERE thread.threadid = post.threadid AND
(post.msgid = '&lt;For-Gilly's-benefit-797E25.00512909042004@news.usfamily.net&gt;') AND
thread.forumid = 170
ORDER BY post.dateline DESC LIMIT 1
mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 's-benefit-797E25.00512909042004@news.usfamily.net&gt;') AND


mysql error number: 1064

Date: Monday 19th of July 2004 12:12:51 PM


I've tried using the filter thats posted a few pages back but it doesnt seem to help, does anyone have any ideas?

cheers

[lierduh]

Hi

Great hack, nice and easy to setup!

I have one problem though, in several groups I get an error on the odd message that looks like this:

mysql error: You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near 's-benefit-797E25.00512909042004@news.usfamily.net&gt;') AND


I've tried using the filter thats posted a few pages back but it doesnt seem to help, does anyone have any ideas?

cheers

The problem is the message-id of the message you are trying to import does not comply to the standard. It contains a quote in the message-id. I have a fix for the same problem, if you go back, you should be able to find my post.

Or otherwise, just up the lastmsg and skip this message.

[lierduh]

Dear lierduh
This is now a regular occurance with the test forums. Every other post looks like this.

The isue seems to be LF/CR and also bad handling of charset because of the missing "?"

Do you want me to replace the last version of gateway.php you supplied with the original (from first post) and see if the issue is still present?

Any help appreciated

The >=20 is to do with Characterset/language. Many people suffer this problem, it can almost all go back to Outlook Express generated messages. According to the header, the message should be a pure text message. I myself pretty confused about the problem. Even vB team mentioned this problem makes their support harder.

What language do the senders use usually?

In the next version (I need a good block of time like a free weekend to work on it), I will have an option to disable the importing of attachments.

[cyc]

The problem is the message-id of the message you are trying to import does not comply to the standard. It contains a quote in the message-id. I have a fix for the same problem, if you go back, you should be able to find my post.

Or otherwise, just up the lastmsg and skip this message.

Thanks mate!

[trackpads]

Is there a way to send the signature and a forum tagline (with the thread links) to the usenet but not the profile information? I REALLY dont like my profile or my users going directly to the usenet.

Thanks again for all the help!

-Jason

[john_rsd]

The >=20 is to do with Characterset/language. Many people suffer this problem, it can almost all go back to Outlook Express generated messages. According to the header, the message should be a pure text message. I myself pretty confused about the problem. Even vB team mentioned this problem makes their support harder.

What language do the senders use usually?

In the next version (I need a good block of time like a free weekend to work on it), I will have an option to disable the importing of attachments.

Most users will post messages in English but I am not sure what the default regional settings would be in windows.

I think I seen some threads at vb forums and I think it was over multiple line breaks.

When the 'reply to forum' feature was available on these forums i never seen this happen, but you could never tell what was replied to by email and what was posted to the forums and it also used named pipes instead of collecting from the mailbox. Perhaps an idea for an option for users who have server which support named pipes and skip the mailbox stage completely (also no need for crons).

vB use a similar system for their support tickets, reply by email, not sure if they see the same issues but I have just looked at the tickets i have submitted and they seem OK also.

This is indeed a great addition to vb, any help you can offer to fix these small things is great.

[john_rsd]

This was one of the thread i was thinking of, but not the exact one, ill keep looking

http://www.vbulletin.com/forum/showthread.php?t=25810&highlight=line+breaks

[lierduh]

vB use a similar system for their support tickets, reply by email, not sure if they see the same issues but I have just looked at the tickets i have submitted and they seem OK also.

This is indeed a great addition to vb, any help you can offer to fix these small things is great.

I am just wondering if the pop3 server has to do with this. Do you know what kind of pop3 server you are using? You can "telnet <server address> 110" and find out.

[Taco John]

Got this error:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

What am I doing wrong? I'm trying to connect to alt.sports.football.pro.denver-broncos

[rinkrat]

All of the sudden my names of every usenet message has gone blank. It was working before. Where should I look for the fis for this?

[Bro_Joey_Gowdy]

clicks install

[john_rsd]

I am just wondering if the pop3 server has to do with this. Do you know what kind of pop3 server you are using? You can "telnet <server address> 110" and find out.

I just send you login and account by PM ;)

[dalecom]

I have installed and tried to run gateway.php for the first time and I get this error :

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

Any ideas ??

tia

[lierduh]

I just send you login and account by PM ;)

Have been quite busy with things. Will have a look very soon.

[lierduh]

All of the sudden my names of every usenet message has gone blank. It was working before. Where should I look for the fis for this?

Have you installed other hacks recently?

[lierduh]

Got this error:

Gateway version 2.1.0 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected


What am I doing wrong? I'm trying to connect to alt.sports.football.pro.denver-broncos

As the message says, the gateway can not connect to the server. You might like to read the third post in this thread for debugging purpose.

[Cold Steel]

Gateway.php is regularly causing the server to have loads of 30+ when it's called.

I'm running 2.1.1 of this script, and 3.0.1 vBulletin. I have three newsgroups. http://forums.asiansinc.com/forumdisplay.php?f=71

Are there any settings or any changes that I can make to prevent the high server loads?

[lierduh]

I just send you login and account by PM ;)

The password did not work for me when I tried telnet into the server.

-ERR authorization failed

[lierduh]

Gateway.php is regularly causing the server to have loads of 30+ when it's called.

I'm running 2.1.1 of this script, and 3.0.1 vBulletin. I have three newsgroups. http://forums.asiansinc.com/forumdisplay.php?f=71

Are there any settings or any changes that I can make to prevent the high server loads?

Basically the script continuesly imports the messages. So the load could be higher than the usual user posts. It needs to build the search index etc. and other server intensive tasks.

For slower servers, you can slow down the script by insert this line

sleep (3);

Just before

} //for each message loop

(should be around line 518)

That will pause 3 seconds after each message is retrieved.

Also how often do you run the script? If you are using the vB built-in cron scheduler, normall it only allows you to run one task each hour. You can create two or more of the same scheduled tasks running in different times of the hour.

[trackpads]

Basically the script continuesly imports the messages. So the load could be higher than the usual user posts. It needs to build the search index etc. and other server intensive tasks.

For slower servers, you can slow down the script by insert this line

sleep (3);

Just before

} //for each message loop

(should be around line 518)

That will pause 3 seconds after each message is retrieved.

Also how often do you run the script? If you are using the vB built-in cron scheduler, normall it only allows you to run one task each hour. You can create two or more of the same scheduled tasks running in different times of the hour.
Hey again!

Is there a way to make the script skip to the next message in a newsgroup if it fails on the last one?

For example, sometimes the script will stop working because of an error importing a single message and I wont realize it for a few days.

It might be better to just have it automatically skip to the next message or whatnot to prevent this?

Thanks again for all the great work!

-Jason

[Cold Steel]

Basically the script continuesly imports the messages. So the load could be higher than the usual user posts. It needs to build the search index etc. and other server intensive tasks.

For slower servers, you can slow down the script by insert this line

sleep (3);

Just before

} //for each message loop

(should be around line 518)

That will pause 3 seconds after each message is retrieved.

Also how often do you run the script? If you are using the vB built-in cron scheduler, normall it only allows you to run one task each hour. You can create two or more of the same scheduled tasks running in different times of the hour.
Thanks. I don't think the server is that slow, but I've made the change anyway.

I had the script running every ten minutes via cPanel cron (couldn't figure out how to work vB cron), but I've cut it back to every half hour.

Thanks again!

[lierduh]

Hey again!

Is there a way to make the script skip to the next message in a newsgroup if it fails on the last one?

For example, sometimes the script will stop working because of an error importing a single message and I wont realize it for a few days.

It might be better to just have it automatically skip to the next message or whatnot to prevent this?

Thanks again for all the great work!

-Jason

This is a good idea. Will include this in the next version.:)

[Sanjiyan]

Can I ask what is the prefix part of the mailing list options (in admincp) are for?

[kkiely]

anyone know what this error is


Database error in vBulletin :

Link-ID == false, connect failed
mysql error:

mysql error number: 0

Date: Friday 16th of July 2004 03:06:00 PM
Script: http://
Referer:
IP Address:


i get these occasionally but my board seems to be working fine

---------
this was a different error i got on a board after i installed the hack but didnt completely test it

Database error in vBulletin :

Invalid SQL:
SELECT title, data FROM datastore
WHERE title IN ('options', 'cron', 'forumcache', 'usergroupcache', 'stylecache', 'smiliecache', 'bbcodecache', 'attachmentcache')

mysql error: Lost connection to MySQL server during query

mysql error number: 2013

Date: Friday 16th of July 2004 12:12:24 PM
Script: http://www.yourgov.org/newreply.php?do=newreply&noquote=1&p=37
Referer: http://www.yourgov.org/showthread.php?t=24
IP Address: 65.213.77.129

[john_rsd]

The password did not work for me when I tried telnet into the server.

-ERR authorization failed

Sorry, I made small error on username :o

Details fixed and send by PM

[trackpads]

This is a good idea. Will include this in the next version.:)


Thanks!! You have one hell of an addon going here!

Here are two more conundrums for you-

1. For some reason I am having an attachment problem. Here is an example post-

http://www.trackpads.com/forum/showthread.php?postid=776189#post776189

Everything looks fine, the pics are there but when I click on one I just get a blank page. I looked in the admin panel and it has the right sizes for each picture but I cannot see them?

2. Another idea for the script. Would it be possible to add the username of the USENET poster to the killfile? There are quite a few I would like to stop from getting through. The problem is that VB wont let me prune them based on their usernames.

Thanks again!!!!!

-Jason

[Rob Eh]

Is there a way to specify the time zone used in the control panel for this?

It uses the server time and not the time set by vB in the control panel.

Thanks,

[xb_]

Warning: array_merge() [function.array-merge]: Argument #2 is not an array in <path-to-vbb>/includes/init.php on line 517
Warning: Invalid argument supplied for foreach() in <path-to-vbb>/includes/sessions.php on line 270

vbb 3.0.3 on apache 2.0.50 + php5 has that effect when opening gateway.php.
Also, ever since last version of this hack, and running it on vbb 3.0.1 + apache 2.0.49 + php 4.3.8 really messed up the attachements in a mailinglist setup.

/me clicks uninstall (it was fun while it worked)

[lierduh]

I have uploaded a new NNTP gateway version. V2.2.0 is now available.

Some of the improvements include:

Provided option of not importing attachments
Provided option of pause between messages
Provided option of not sending messages to USENET
Provided mechanism to skip importing a message if it was crashed on it
Provided "Default Footer" option
Provided strip_footer function for mailing lists
(written originally by KevinM)
Provided Killfile functions with Header/Body/AND/OR settings
Provided ability to thread messages by subject for mailing lists
Provided ability to use a single POP3 account for different mailing lists
(use prefix)
Merged KevinM's attachment routine, save thumbnail to hard drive
as vB3.0.3
Official support to have the script run by Scheduled Tasks
Many bug fixes

To upgrade, you only need to open the AdminCP NNTP Settings page. The
upgrade will be carried out automatically in the background.

To run the script manually, you must turn on the debug mode now:

http://forums.yourdomain.com/gateway.php?debug=1

Enjoy...

[lierduh]

Warning: array_merge() [function.array-merge]: Argument #2 is not an array in <path-to-vB>/includes/init.php on line 517
Warning: Invalid argument supplied for foreach() in <path-to-vB>/includes/sessions.php on line 270

vB 3.0.3 on apache 2.0.50 + php5 has that effect when opening gateway.php.
Also, ever since last version of this hack, and running it on vB 3.0.1 + apache 2.0.49 + php 4.3.8 really messed up the attachements in a mailinglist setup.

* xb_ clicks uninstall (it was fun while it worked)

You have a very wierd problem there. sessions.php is not involved with the gateway.php, how can you have error message associated with it?

[lierduh]

Is there a way to specify the time zone used in the control panel for this?

It uses the server time and not the time set by vB in the control panel.

Thanks,

Please use the new version.

[lierduh]

Thanks!! You have one hell of an addon going here!

Here are two more conundrums for you-

1. For some reason I am having an attachment problem. Here is an example post-

http://www.trackpads.com/forum/showthread.php?postid=776189#post776189

Everything looks fine, the pics are there but when I click on one I just get a blank page. I looked in the admin panel and it has the right sizes for each picture but I cannot see them?

2. Another idea for the script. Would it be possible to add the username of the USENET poster to the killfile? There are quite a few I would like to stop from getting through. The problem is that VB wont let me prune them based on their usernames.

Thanks again!!!!!

-Jason

Both problems have been addressed in the new version.

vB3.0.3 changed the way the thumbnail is saved, hence the problem.

[lierduh]

Sorry, I made small error on username :o

Details fixed and send by PM

Usually MS server uses the full email address as the user name.

Could you 'put' some emails in the box. I will see if I can find anything. It is empty now.

[Reeve of shinra]

It also allows a forums user to send messages to the mailing list by the forum posting interface.


I was thinking of using this with a yahoo group but its
moderated so if my members sent replies via this hack, it would have to use thier email address. Is that possible? Thanks!

[fonzerelli_79]

great script

got it working no problem

i got thousands of posts downloaded into my forums using gateway.php

problem is most of them have no content - is this because im using a free server and ive reached my limit. If my limit was reached would posts still be downloaded but would they just be blank?

[lierduh]

I was thinking of using this with a yahoo group but its
moderated so if my members sent replies via this hack, it would have to use thier email address. Is that possible? Thanks!

Will need to hack the code. It may work for moderated group, but there are problems associated with groups that need subscription.

[lierduh]

great script

got it working no problem

i got thousands of posts downloaded into my forums using gateway.php

problem is most of them have no content - is this because im using a free server and ive reached my limit. If my limit was reached would posts still be downloaded but would they just be blank?

Need more information. What is the message look like using a normal news reader? What message does it display when you run the debug mode?

[trackpads]

Way to go Lierduh! Excellent!

Is there a way to make the Killfile for the USENET posts only? I had this set up under the old Killfile so that I could take all the political rantings out of the usenet posts but not out of the information mailing lists etc.

Thanks again for all the great work!!

-Jason

[fonzerelli_79]

Need more information. What is the message look like using a normal news reader? What message does it display when you run the debug mode?
no actual message appears - an hour glass appears as if gateway is working

if you load the page again it says "Another instance of gateway.php is running, try again in 30 minutes if the script was crashed." so it should be working ok

i really think its down to me using a free usenet server as the first few hundred were fine with all content etc

[fonzerelli_79]

i checked this morning - all new posts are fine and seem to be working ok - but there are thousands of threads with titles but no actual content -

im not too bothered why this is because they are older threads - though at the same time i really should find out why it did it incase it happens again

[fonzerelli_79]

it seems to be still happening -some posts have content and some dont

is there a way to stop posts with no content being posted?

thanks :)

[rinkrat]

I've been wanting to do this for years and I finally can!! Thanks for this hack :)

[Marv]

To upgrade, you only need to open the AdminCP NNTP Settings page. The
upgrade will be carried out automatically in the background.
lierduh, do we have to upload (and overwrite) all of the present files ?
( noticed a missing update note in gateway.php. it still says "define('VERSION', '2.1.0');" )

regards,
Marv.

[lierduh]

Way to go Lierduh! Excellent!

Is there a way to make the Killfile for the USENET posts only? I had this set up under the old Killfile so that I could take all the political rantings out of the usenet posts but not out of the information mailing lists etc.

Thanks again for all the great work!!

-Jason

Please remember the Killfile looks throught the entire header when the 'H:' is used. So you can easily include a unique news post string. eg.
{[B:your search string] AND [H:news.group.name]}

[lierduh]

lierduh, do we have to upload (and overwrite) all of the present files ?
( noticed a missing update note in gateway.php. it still says "define('VERSION', '2.1.0');" )

regards,
Marv.

Looks like you do not have the new files. To confirm, I have just downloaded the file and it indeed has 2.2.0 Version gateway.php included.

Yes, you need to replace all the existing files. All the files have been changed since last version.

[Marv]

Looks like you do not have the new files. To confirm, I have just downloaded the file and it indeed has 2.2.0 Version gateway.php included.

Yes, you need to replace all the existing files. All the files have been changed since last version.you?re so damn right. extracted the zip into the wrong DIR..
thanks for guiding the blind, lierduh.

[Rob Eh]

it seems to be still happening -some posts have content and some dont

is there a way to stop posts with no content being posted?
All of mine since the upgrade are that way. Titles and no content.

[Rob Eh]

I over wrote all of the files in the new zip file again. The overwritten files were all dated the same time as when I did the upgrade. I ran the gateway.php file manually with debug after I back dated the last post # on one of my groups by 10. All of those 10 posts worked fine. I'm going to take it back a few again and wait until it runs from the vB scheduled tasks. I will report my findings.

[lierduh]

All of mine since the upgrade are that way. Titles and no content.

Well, the only way I can fix things is by knowing more details. What is the post should look like to start with. How about perhaps a link in Google Group.

The debug message is vital too.

[Rob Eh]

From running gateway from the scheduled tasks, all of the posts are coming back as blank.

The new upgrade does not seem to support being run in this fashion. :(

Suggestions?

[Rob Eh]

Well, the only way I can fix things is by knowing more details. What is the post should look like to start with. How about perhaps a link in Google Group.

The debug message is vital too.

Here is one of the threads that I haven't deleted the new blank messages yet. The last three messages in the thread should be the same as the first three.
http://forums.fu2.ca/showthread.php?t=3686

I will run it again with the debug message and post it shortly.

[lierduh]

From running gateway from the scheduled tasks, all of the posts are coming back as blank.

The new upgrade does not seem to support being run in this fashion. :(

Suggestions?

Thanks, will investigate.

[Rob Eh]

Here is the debug message from running it manually
Gateway version 2.2.0 2 group(s) gatewayed.

Connecting to server, server says: 200 News.GigaNews.Com
Server responded after user name: 381 more authentication required
Server responded after password: 281 News.GigaNews.Com
Info for alt.sport.air-guns at news-60.giganews.com:
211 2852 16876 19727 alt.sport.air-guns


Connecting to server, server says: 200 News.GigaNews.Com
Server responded after user name: 381 more authentication required
Server responded after password: 281 News.GigaNews.Com
Info for rec.sport.paintball at news-60.giganews.com:
211 34660 523609 558268 rec.sport.paintball

Getting message number 558259:
'Re: Looking for a guy named Preacher from Cali....' from Kenner Costen, MCSE, MCSA, MCP. Thread found by References.
Getting message number 558260:
'Re: The Next Reality Show' from Destiny Publications. Thread found by References.
Getting message number 558261:
'FS: Custom Cocker Package - Cheap!' from Russell Twiss. New thread.
Getting message number 558262:
'Re: Boomstick' from Space Cadet. Thread found by References.
Getting message number 558263:
'Re: OT: Patches' from FL PBPress. New thread.
Getting message number 558264:
'Re: OT: Patches' from ayar-15. Thread found by References.
Getting message number 558265:
'Viper vs Vector ??' from Graham. New thread.
Getting message number 558266:
'Re: Viper vs Vector ??' from Insane Ranter. Thread found by References.
Getting message number 558267:
'Re: Viper vs Vector ??' from Graham. Thread found by References.
Getting message number 558268:
'Re: Viper vs Vector ??' from Insane Ranter. Thread found by References.
These all posted porperly. It's just when it's ran from the scheduled task. In fact, the job can't be completeing as it is not showing up in the logfile.

I don't have access to crontab.

[Marv]

lierduh,
after upgrading the files I was wondering wyh nothing happend. So I manually run the gateway.php and received this:

Invalid SQL-Query:
SELECT post.postid FROM vb3thread AS thread, vb3post AS post
WHERE thread.threadid = post.threadid AND
(post.msgid = '<c0b6mb$h3d$1@newsreader2.net.com>') AND
thread.forumid = 100
ORDER BY post.dateline DESC
mysql error: Lost connection to MySQL server during query

mysql error number: 2013

I?m not very familiar with the sql database. vBulletin has a prefix "vb3" in the database - is that part of the problem ? At least I don?t really understand what this error-msg is trying to tell me... :ermm:

[lierduh]

Here is the debug message from running it manually
[PHP]Gateway version 2.2.0 2 group(s) gatewayed.



Ok, download the v.2.2.1. This should fix the Scheduled Tasks problem. You only need to replace the gateway.php.

https://vborg.vbsupport.ru/attachment.php?attachmentid=19995

[lierduh]

lierduh,
after upgrading the files I was wondering wyh nothing happend. So I manually run the gateway.php and received this:



I?m not very familiar with the sql database. vBulletin has a prefix "vb3" in the database - is that part of the problem ? At least I don?t really understand what this error-msg is trying to tell me... :ermm:

It means the gateway.php can not connect to the server for that query.
Please try again, and let me know if it persists.

[Rob Eh]

lierduh, that cured the problem with blank posts.

Thanks! :up:

[john_rsd]

Dear lierduh


The software retrieves messages from a pop3 account and import them into the forums threaded.

>snip<

Jul 31 2004 (release version 2.2.0)
[list]
Provided ability to thread messages by subject for mailing lists


I have 2.2.0 but thread by subject does not seem to work. Very strange

[xb_]

You have a very wierd problem there. sessions.php is not involved with the gateway.php, how can you have error message associated with it?

heh. i'm not sure, my only guess would be that vbulletin has reorganized the including of files in 3.0.3. However because of your reply i decided to give it another go with the newer version. Which didn't give me a session error, but it did give me a prefix error
Invalid SQL: UPDATE forums_ thread
SET firstpostid = 3214
WHERE threadid = 1858
mysql error: Table 'www.forums_' doesn't exist

mysql error number: 1146

Date: Tuesday 03rd of August 2004 12:08:12 PM
Script: <masked>/gateway.php
Referer:
Username: Unregistered That seems to be a problem around line nr 1050; one space too much. After i've removed that space it started importing.

Still i was left with two messages:Warning: array_merge() [function.array-merge]: Argument #1 is not an array in <somepath>/htdocs/global.php on line 334

Warning: Invalid argument supplied for foreach() in <somepath>/htdocs/includes/functions.php on line 2223
Both seem to be related with a certain template array.
It appears to work alright tho, besides those messages. yet i'm crossing my fingers for attachements.

[john_rsd]

http://proteluser.com/bbs3/showthread.php?t=2074

I sent 2 messages to check thread by subject which failed.

I then sent a reply to see of the RE part was needed in the subject.

The last message i sent was corrupted and instead of looking like this as it was in the first post

At a gathering of science reporters several years ago, Dr. George
Lundberg, former editor of the Journal of the American Medical
Association, said that the Internet "is the most important advance in
medical communications since the printing press." If Dr. Lundberg is
correct, perhaps one of the Web's most useful tools for advancing
medical communications is e-mail.

It now looked like this

> At a gathering of science reporters several years ago, Dr.=20
> George Lundberg, former editor of the Journal of the American=20
> Medical Association, said that the Internet "is the most=20
> important advance in medical communications since the=20
> printing press." If Dr. Lundberg is correct, perhaps one of=20
> the Web's most useful tools for advancing medical=20
> communications is e-mail.
>=20

We looked at this before but now i can replicate it as it was me who sent the message and replied to it. All messages were sent from outlook2003 in plain text format.

[trackpads]

Please remember the Killfile looks throught the entire header when the 'H:' is used. So you can easily include a unique news post string. eg.
{[B:your search string] AND [H:news.group.name]}
I could but the problem is that I have about 60 usenet forums where I want the killfile but never need it in the Mlists. I would have to add a kill word for each of the 60 usenet forums to get the same affect.

Thanks again!

-Jason

[fonzerelli_79]

the old gateway.php file looks to have fixed it - now all i have to deal with is the fact ive got thousands of threads with no content!!!

[Rob Eh]

Prune those threads, reset your last article # and let it start again.

[fonzerelli_79]

does anyone know if there is a way to mass prune threads in a specific forum where the posts are blank - doesnt seem to be this option in the mass prune forum tools admin page

thanks :)

[lierduh]

I could but the problem is that I have about 60 usenet forums where I want the killfile but never need it in the Mlists. I would have to add a kill word for each of the 60 usenet forums to get the same affect.

Thanks again!

-Jason

You can surely find the common words amongst the news groups where the mailing lists do not possesss. eg. the news server? look for things in the Path: and Xref...

[lierduh]

Dear lierduh



I have 2.2.0 but thread by subject does not seem to work. Very strange

Have you set the option in AdminCP to 1? A message must have "Re: " in the beginning of the subject before it can be threaded.

[lierduh]

heh. i'm not sure, my only guess would be that vbulletin has reorganized the including of files in 3.0.3. However because of your reply i decided to give it another go with the newer version. Which didn't give me a session error, but it did give me a prefix error
That seems to be a problem around line nr 1050; one space too much. After i've removed that space it started importing.

Still i was left with two messages:
Both seem to be related with a certain template array.
It appears to work alright tho, besides those messages. yet i'm crossing my fingers for attachements.

Thanks for that, there was an extra space in that line.

Usually Warnings do not affect php scripts. Strange I am not getting any warnings, even in the error log. Have you had other hacks done involves templates?

[lierduh]

http://proteluser.com/bbs3/showthread.php?t=2074

It now looked like this
We looked at this before but now i can replicate it as it was me who sent the message and replied to it. All messages were sent from outlook2003 in plain text format.

I have found the reason for this:


A third thing that can cause "messy" email is the fact that some older mail programs do not gracefully handle the advanced character sets that let us use "special characters" such as bullet marks, foreign letters, and other non-standard characters.
Eudora and many other email programs use something called "quoted-printable" encoding to send these special characters, which requires that the receiving email program support an email standard called MIME. If the email program of your recipient is not "MIME-compliant," then they might see text that looks like this:

See ++++=20 >See ++++ run. Run, ++++, run.=20 See Jane dance.=20 >Dance,.=20 Jane, dance.=20

The solution is to turn off "quoted-printable" encoding for messages that are going to people with older mail programs.

In Eudora, there is a button in the compose mail window marked "QP." Usually it's toggled on. Press it to turn it off. It will stay off until you turn it on again.

Outlook and Outlook Express do not use quoted-printable encoding by default. You can make sure it's not on in Outlook at Tools>Options>Mail Format>Settings. The "Encode text as" item lets you choose none, quoted-printable or base64 encoding. In Outlook Express, the same choices can be found at Tools>Options>Send>Plain Text Settings (under Mail Sending Format).



extracted from:
http://www.techsoup.org/print/printpage.cfm?ArticleID=180&type=article

Will come up with a solution for this for this project. I need you to send more reply emails to the test account with this feature.

[trackpads]

For some reason I opened it today and I am getting this -

<br />
<b>Warning</b>: array_merge() [<a href='function.array-merge'>function.array-me
rge</a>]: Argument #1 is not an array in <b>/home/trackpad/public_html/forum/glo
bal.php</b> on line <b>335</b><br />
<br />
<b>Warning</b>: in_array() [<a href='function.in-array'>function.in-array</a>]:
Wrong datatype for second argument in <b>/home/trackpad/public_html/forum/inclu
des/functions.php</b> on line <b>2172</b><br />
<br />
<b>Warning</b>: Invalid argument supplied for foreach() in <b>/home/trackpad/pu
blic_html/forum/includes/functions.php</b> on line <b>2177</b><br />

[xb_]

Usually Warnings do not affect php scripts. Strange I am not getting any warnings, even in the error log. Have you had other hacks done involves templates?
I have one hack involving templates, which is auto add template to all styles or something similair. However these kind of warnings might not be so hack (specific) related. I'm having two or three more hacks pending with similair warnings. These warnings started to occur after upgrading to php5 and upgrading to vB 3.0.3. I'm not sure (yet) which upgrade exactly implemented this.

For some reason I opened it today and I am getting this This is exactly what i'm describing. Keep reading ;)

Gonna read some of vB.com about this, if they mention it.

they did: http://www.vbulletin.com/forum/bugs.php?do=view&bugid=1588 It is a result of php5. I'm using the bugreport as an example to resolve the warnings throughout vB about array_merge. And it seems to fix things.

[john_rsd]

Have you set the option in AdminCP to 1? A message must have "Re: " in the beginning of the subject before it can be threaded.

lierduh

I now installed 2.2.1 also

Where is the option in the AdminCP?

I found the attachment option and set to 0 as i do not want attachments, the delay between messages option (set to 1 to see what happens) but nothing for threading?

[john_rsd]

I have found the reason for this:


A third thing that can cause "messy" email is the fact that some older mail programs do not gracefully handle the advanced character sets that let us use "special characters" such as bullet marks, foreign letters, and other non-standard characters.
Eudora and many other email programs use something called "quoted-printable" encoding to send these special characters, which requires that the receiving email program support an email standard called MIME. If the email program of your recipient is not "MIME-compliant," then they might see text that looks like this:

See ++++=20 >See ++++ run. Run, ++++, run.=20 See Jane dance.=20 >Dance,.=20 Jane, dance.=20

The solution is to turn off "quoted-printable" encoding for messages that are going to people with older mail programs.

In Eudora, there is a button in the compose mail window marked "QP." Usually it's toggled on. Press it to turn it off. It will stay off until you turn it on again.

Outlook and Outlook Express do not use quoted-printable encoding by default. You can make sure it's not on in Outlook at Tools>Options>Mail Format>Settings. The "Encode text as" item lets you choose none, quoted-printable or base64 encoding. In Outlook Express, the same choices can be found at Tools>Options>Send>Plain Text Settings (under Mail Sending Format).



extracted from:
http://www.techsoup.org/print/printpage.cfm?ArticleID=180&type=article

.

Ah, good, at least once a problem is defined a solution can be looked for.

I did not find the options mentioned in the article in Outlook 2003 version or Outlook express 6



Will come up with a solution for this for this project. I need you to send more reply emails to the test account with this feature.

lierduh

You want me to send more junk email :D ??

Seems strange requests these days when people usually do not want any more emails!

I am joking, no probs, i can send some reply

BTW I did not turn on any 'feature' just turned off all "HTML" and "rich text" options in Outlook 2003.

Excellent work..

[john_rsd]

lierduh

I now installed 2.2.1 also

Where is the option in the AdminCP?

I found the attachment option and set to 0 as i do not want attachments, the delay between messages option (set to 1 to see what happens) but nothing for threading?


Sorry

Found it

It was not in the nntp settings within the vb admincp. I ran the file externally and it was OK.

Now all is well, must have been a cache issue or something.

[john_rsd]

lierduh

Still something up with thread by subject

http://www.proteluser.com/bbs3/forumdisplay.php?f=2

See subject

Re: [LF] AW: [LF] Tin Whiskers,Tin Finish and Accelerated Stress Testing

This was not threaded with the other posts even with 'Re' in front. Its common for mail lists to have a prefix like [name] so could 2 instances like this cause a problem?

AW is a common prefix also for reply.

This was post i expected to see it threaded with and it had already done this with one previous post.

[LF] AW: [LF] Tin Whiskers, Tin Finish and Accelerated Stress Testing

Same with the thread [LF] LF with RF which had 2 replies and was missed

This mostly worked 8 out of 10 times before with original gateway.php and Kevins mod.

http://proteluser.com/bbs3/forumdisplay.php?f=3&page=52&sort=lastpost&order=&pp=20&daysprune=-1

What you think?

[cyc]

Hi

Has anyone noticed some posts created within in the forum are not threading properly when view with a newsreader or via google groups etc.. I have always disabled the post reply button in usenet sections, so users are forced to quote properly.

An example of what i'm talking about is

View in threaded moded within forum;

Post 1
| Post 2
| Post 3
| Post 4


Viewed using newsreader;

Post 1
| Post 2
| Post 3 (reply quoting 2)
| Post 4 (reply quoting 3)

I'n this case all posts to the thread originated from our forum and where posted within 25 minutes of each other.

[lierduh]

Hi

I'n this case all posts to the thread originated from our forum and where posted within 25 minutes of each other.

If the user clicks the Big Reply button, the posts will be threaded to the top message. If the Quote button is pushed, the post will be threaded under the one which one is replying.

I have some advanced gateway tweaking guide which automatically hides the big Reply button for USENET forums. I will post it somewhere later. The method needs to change vB php code, hence the advanced title.

[cyc]

I have some advanced gateway tweaking guide which automatically hides the big Reply button for USENET forums. I will post it somewhere later. The method needs to change vB php code, hence the advanced title.

Sounds good, I got around this by creating another style for usenet forums and commenting out the big replyto button. I just find some post don't thread properly :devious: dunno whats causing it.

[fonzerelli_79]

is anyone pruning their usenetgroup forum

my group has pulled about 160,000 posts so far and it keeps increasing - im fairly pleased with this in one respect in that the search engines will have plenty of pages to keep them busy which should hopefully increase traffic

however, the more and more posts there are the larger my database will be which means that backing up will be a pain just now but if i leave it as it is for a few months id have to back up a forum of millions of posts - not good

whats everyone else doing about this?

[john_rsd]

Slightly different request

Does the thread by subject query exist somehwre as a seperate item that can be run on existing forums?

What I mean is, does a query exist to run on the currently non-threaded posts in the forum to thread them?

Would be nice to tidy up the posts currently existing while threading by subject was not available. Manually doing it would be a terrible task ....

John

[Sanjiyan]

I am using 2.2.1 but I have a question, is there away for it to when it posts to the forums, to use the date it was posted to usenet and not the time it was posted to the forums?

I prefer to have the posts (time/date stamp) to be relative to the time/date it was posted originally to usenet, and not the time/date the script posted it to the website (forums)

[Princeton]

fantastic job on the upgrade!!!:up:

[NietzscheKid]

Hello.

I am very frustrated. I'd so love to install this mod but it does not work for me.

I have done what you recommended in post #3 about adding a new value to the registry key. But that did not change anything either. I still get this error:

Gateway version 2.2.1 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

I also do not have access to ssh or telnet so I cannot test it there either.

So, I am hoping that perhaps you have any info or tipps for this or is my case hopeless?

I do not run vb in debug mode. Is that the problem?

Thanks for any advice/help. :)

[lierduh]

Hello.

Gateway version 2.2.1 1 group(s) gatewayed.
Could not connect to NNTP-serverNot connectedNot connected

I also do not have access to ssh or telnet so I cannot test it there either.

Thanks for any advice/help. :)

There is nothing one could help you.:)

It basically means the script can not connect to the server. Either you entered the server wrong, or your web server does not allow you to connect on port 119. Perhaps email your host will get more information.

[lierduh]

I am using 2.2.1 but I have a question, is there away for it to when it posts to the forums, to use the date it was posted to usenet and not the time it was posted to the forums?

I prefer to have the posts (time/date stamp) to be relative to the time/date it was posted originally to usenet, and not the time/date the script posted it to the website (forums)

Too many options starting to become a problem.:)

If you go through the settings control panel, you will find the setting for this.

[lierduh]

lierduh

This was not threaded with the other posts even with 'Re' in front. Its common for mail lists to have a prefix like [name] so could 2 instances like this cause a problem?

AW is a common prefix also for reply.

What you think?

I just checked your forums, the messages seem to be threaded together probably. Are you sure those two messages were not imported before you turned on the option?

If you see AW (what language?) as a common reply prefix, you you simply add AW: to the existing re: fw: line in gateway.php. That will be easy to hack.

[NietzscheKid]

Hello.

It is working with the example you posted in your readme file.

However, I have tried, unsuccessfully, to install this newsgroup.

Group: alt.music.pearl-jam

server: group.google.com

This does not work for some reason.

Is there something wrong with these two entries? :ermm:

Again, sorry for these questions.

[lierduh]

Sounds good, I got around this by creating another style for usenet forums and commenting out the big replyto button. I just find some post don't thread properly :devious: dunno whats causing it.

I have uploaded the "Advanced_gateway_hacking_guide.txt".
The guide only contains the method to hide the big Reply button.

Go to the first page to download.

[lierduh]

Hello.

Group: alt.music.pearl-jam

server: group.google.com

Again, sorry for these questions.

group.google.com does not even resolve DNS.

You need to find out what is the news server your host lets you use, or pay for one. They are very cheap anyway, do a search in this thread for ones to use.

[trackpads]

AKkkkk, now I am getting this error-


<br />
<b>Warning</b>: array_merge() [<a href='function.array-merge'>function.array-merge</a>]: Argument #1 is not an array in <b>/home/trackpad/public_html/forum/global.php</b> on line <b>335</b><br />
<br />
<b>Warning</b>: in_array() [<a href='function.in-array'>function.in-array</a>]: Wrong datatype for second argument in <b>/home/trackpad/public_html/forum/includes/functions.php</b> on line <b>2172</b><br />
<br />
<b>Warning</b>: Invalid argument supplied for foreach() in <b>/home/trackpad/public_html/forum/includes/functions.php</b> on line <b>2177</b><br />

[NietzscheKid]

group.google.com does not even resolve DNS.

You need to find out what is the news server your host lets you use, or pay for one. They are very cheap anyway, do a search in this thread for ones to use.

I don't understand. Does this hack allow me to use alt.music.pearl-jam from google but i need to find another server or what do you mean? Does anyone know or use a google newsgroup? If there's any kind soul who can give me a tipp on which news server supports google newsgroups like e.g. alt.music.xx, please give this poor woman some hints. I'd be forever in your debt. :)

Thank you :)

PS: I'd also gladly pay for a news server. The only thing that I really need to know is whether I could then use google's alt.music.xxx newsgroup to display in my forums?
I've seen google's newsgroup mentioned in this thread once and in the old hack thread also. So I assume this must be working?

[Princeton]

It's been a while since I used this ... please verify.
If is_running is set to 1 it WILL CHECK if another instance is running?

[john_rsd]

I just checked your forums, the messages seem to be threaded together probably. Are you sure those two messages were not imported before you turned on the option?

If you see AW (what language?) as a common reply prefix, you you simply add AW: to the existing re: fw: line in gateway.php. That will be easy to hack.

lierduh

No, they were delivered to the mailbox but were not collected until after I upgraded.

But I did have the cache issue at that time as I could not see the option in the admincp to turn threading on. So most likely it was actually turned off. Seems to work now as you seen.

Regarding the parsing for the "Re:" I saw once instance of non threading

http://proteluser.com/bbs3/forumdisplay.php?f=8

Subject [Elecraft] KX1 Random Wire Antenna recommendations

Post http://proteluser.com/bbs3/showthread.php?t=2262 has had the "Re:" placed after the mailing list name between the []'s, I cannot even guess how that happened but i suspect it was down to the authors email client.

I guess the Re: has to be the first characters in the string to work?

[xb_]

lierduh,

Could you implement something in the next version that will log the actual ip into the post as poster ip instead of the email addy (in a mailing list setup) ?

I'm also having alot of work merging replies to excisting threads, for some reason alot of the emails aren't glued together in the correct thread. I would apreciate it if it can be improved.

regards

[lierduh]

I don't understand. Does this hack allow me to use alt.music.pearl-jam from google but i need to find another server or what do you mean? Does anyone know or use a google newsgroup? If there's any kind soul who can give me a tipp on which news server supports google newsgroups like e.g. alt.music.xx, please give this poor woman some hints. I'd be forever in your debt. :)

Thank you :)

PS: I'd also gladly pay for a news server. The only thing that I really need to know is whether I could then use google's alt.music.xxx newsgroup to display in my forums?
I've seen google's newsgroup mentioned in this thread once and in the old hack thread also. So I assume this must be working?

Google does not own USENET. Google only archive them, that it, no more, no less.

What is USENET?
http://www.faqs.org/faqs/usenet/what-is/part1/

or a short version

http://whatis.techtarget.com/definition/0,289893,sid9_gci213262,00.html

[lierduh]

AKkkkk, now I am getting this error-



Perhaps reading a couple of pages back can help.

[lierduh]

lierduh
Post http://proteluser.com/bbs3/showthread.php?t=2262 has had the "Re:" placed after the mailing list name between the []'s, I cannot even guess how that happened but i suspect it was down to the authors email client.

I guess the Re: has to be the first characters in the string to work?

It means the post was not sent as a follow up.(possibly no reference in the header. The mailing list does not know it is a reply, so made it a new post.

[lierduh]

lierduh,
I'm also having alot of work merging replies to excisting threads, for some reason alot of the emails aren't glued together in the correct thread. I would apreciate it if it can be improved.

regards

If you can list the examples and the original message's subject. I will find the reason.

Basically the thread by subject works by:

1) The message needs to have Re: in the beginning of the subject before the script even starts to worry about finding a thread.
2) "Re: asdf" will match thread's title with
asdf
Re: asdf

"Re: Re: asdf" will match thread's title with
asdf
Re: asdf
Re: Re: asdf

"Re: Re: Re: asdf" will match

Re: asdf
Re: Re: asdf
Re: Re: asdf
Re: Re: Re: asdf
(not match with asdf)

It will find the oldest thread, and will use the newest post as the parent message id (only used when the forum reader uses non-inline display mode).

[lierduh]

It's been a while since I used this ... please verify.
If is_running is set to 1 it WILL CHECK if another instance is running?

Should be left at 0.

[Princeton]

Should be left at 0.
thank you lierduh - I did look into the files and found it. :)

Default = 0. Set to 0 : Check if another instance of gateway.php is running. Set to 1 : Do not check.

[movielad]

I'm using the NNTP gateway script to provide users access to our internal news server. As such, I'm creating user accounts under vBulletin which matches the LHS of their email address (i.e. anything before the @ sign). When posting to the news server, I'm getting a copy of the message ID as the From: header. I want to be able to post with a legitimate email address.

For example:

xxxxxxx.rrrrr@domain.name

(where xxxxxxxx is the normal LHS email name and rrrrr is the message ID)

whereas I'd like it to just be:

xxxxxxx@domain.name

Is there an easy way of doing this/changing the current gateway.php file to handle this?

Regards,

MArtyn

[allan grossman]

Hey, lierduh -

Is there a reason why the nntp_groups table is ISAM instead of MyISAM? I know if you convert the table to MyISAM the script fails - and since it's usually a tiny table it probably wouldn't benefit much from optimizing, but having to deselect it when optimizing the forum database might be something to look at.

cheers -

allan

[john_rsd]

lierduh

Is there a limitation on the subject line size?

I received some messages (all via the same mailserver) as follows

Original message http://proteluser.com/bbs3/showpost.php?p=2663&postcount=1

Original subject line [dxp] A rule for allowing two components on top of each other?

Then I seen a reply which was not threaded

Reply http://proteluser.com/bbs3/showpost.php?p=2717&postcount=1

Subject line Re: [dxp] A rule for allowing two components on top of eachother?

Notice the space is missing between the last 2 words "each other" is now "eachother". Hence the reason for none threading.

But the reply as i see it in my mailbox, from the same email server (the same message), is actually correct and does read Re: [dxp] A rule for allowing two components on top of each other?

So the originating message was actually correct, it was not edited by the user. But i did notice that the subject line was wrapped to a new line between these words in the actual header. Perhaps related to the issues with the strange end of line characters in some posts?

Any information appreciated.

I forward a copy of the message headers for this non threaded reply to the mailbox i set up for you (same mail server again)

[lierduh]

Hey, lierduh -

Is there a reason why the nntp_groups table is ISAM instead of MyISAM? I know if you convert the table to MyISAM the script fails - and since it's usually a tiny table it probably wouldn't benefit much from optimizing, but having to deselect it when optimizing the forum database might be something to look at.

cheers -

allan

allan, you are using the very original table created by Gilby's original hack. I remember in that SQL, the tables were created as ISAM.

If someone started new with the vB3 hack, the tables will be created using MyISAM table type.

I can't see the reason why MyISAM will cause the script to fail. Both of my nntp tables are MyISAM.

[xb_]

If you can list the examples and the original message's subject. I will find the reason. I've set you up an account, details are in pm. There are only a few examples at this moment, since most of them i had already merged manually.

ok by now there are plenty examples :D

[Marv]

Hi lierduh,

if possible, could you tell me what I have to do to make gateway.php pause between connecting to different ng?s ?
I guess it?s the same function like this:

if ($nntp_settings['pause_seconds'])
{
sleep ($nntp_settings['pause_seconds']);
}

Should only be another table-name and selectable value. Some newsservers are blocking the script in case the gateway is way to fast for an old and slow newsserver. (Err-Msg: "Too many connections. Your IP is temporarly blocked.")

TIA,
Marv.

[dethfire]

alot of my imports have RE: in the title, how can I go through and delete that part from all my thread titles?

[allan grossman]

allan, you are using the very original table created by Gilby's original hack. I remember in that SQL, the tables were created as ISAM.

If someone started new with the vB3 hack, the tables will be created using MyISAM table type.

I can't see the reason why MyISAM will cause the script to fail. Both of my nntp tables are MyISAM.

Thanks, lierduh - I'll convert the table. I guess I have been running this thing for awhile ;)

edit: Converted the table to MyISAM. All is well - thanks, lierduh :)

[john_rsd]

lierduh

Thanks for the updated mime.php file, i have uploaded it now. The forums should fill up a bit now.

Excellent work my friend!

[john_rsd]

Error messages

After isntalling the new mime.php i now get this error when importing some messages

Warning: Unknown modifier 'b' in
/home/protelus/public_html/bbs3/gateway.php on line 346 />

Warning: Unknown modifier '>' in
/home/protelus/public_html/bbs3/gateway.php on line 346 />

Warning: Unknown modifier 'a' in
/home/protelus/public_html/bbs3/gateway.php on line 346 />

You can see the whole message here
http://proteluser.com/bbs3/showthread.php?t=2400

Never seen this error before, ill investigate more

Any information appreciated

[lierduh]

Error messages

After isntalling the new mime.php i now get this error when importing some messages



You can see the whole message here
http://proteluser.com/bbs3/showthread.php?t=2400

Never seen this error before, ill investigate more

Any information appreciated

Edit gateway.php, around line 333, replace the whole block with the following:


if (!$message['text'] and $message['html']){
$pattern = array(
"/\n/",
"/<br([^>]*)>/siU",
"/<[\/]*(div|p)([^>]*)>/siU",
"/<b>(.*)<\/b>/siU",
"/<i>(.*)<\/i>/siU",
"/<a[^>]*href=([^ >]*)>(.*)<\/a>/siU"
);
$replace = array(
" ",
"\n",
"\n\n",
"\\1",
"\\1",
"\\2 (\\1)"
);

$message['text'] = strip_tags(preg_replace($pattern, $replace, $message['html']));
}

[lierduh]

I've set you up an account, details are in pm. There are only a few examples at this moment, since most of them i had already merged manually.

ok by now there are plenty examples :D

Actually I need the original message headers.

[Marv]

Hi lierduh,

if possible, could you tell me what I have to do to make gateway.php pause between connecting to different ng?s ? Some newsservers are blocking the script in case the gateway is way to fast for an old and slow newsserver. (Err-Msg: "Too many connections. Your IP is temporarly blocked.")
Could someone help me with that problem ? I can run the gateway only once a day connecting to 9 newsgroups. That means I receive ~50 posts from the first newsgroup and while they are ported over to the db my Server-IP becomes blocked :ermm:
I guess the script works perfect as it should. I only must connect to a newsserver that seems to accept only 5 request per second. 5 connections to newsgroups are working fine - everything above becomes blocked. I guess that?s because the script doesn?t loop through each ng like the script imports the single posts. Instead it gateways all ng?s with only one request which means multiple connections at one time. (True or false ?)

Is there a way that gateway.php could pause between two ng?s for ? seconds ?

Regards,
Marv.

[Blackbeard]

I have also just installed this hack, seems i have got to 2 posts in the forum and the debug as stopped
refreshed and got this

Another instance of gateway.php is running, try again in 30 minutes if the script was crashed.

please help.

[Blackbeard]

Now all the post started appiering and as a test i used the microsoft one in the readme file, after 1000's of post appear on my forum i selected gateway to disable, but posts keep posting on the forum, why? when you start it dont it stop?

[john_rsd]

Edit gateway.php, around line 333, replace the whole block with the following:


Thank you, off to try now.

[lierduh]

Could someone help me with that problem ? I can run the gateway only once a day connecting to 9 newsgroups. That means I receive ~50 posts from the first newsgroup and while they are ported over to the db my Server-IP becomes blocked :ermm:
I guess the script works perfect as it should. I only must connect to a newsserver that seems to accept only 5 request per second. 5 connections to newsgroups are working fine - everything above becomes blocked. I guess that?s because the script doesn?t loop through each ng like the script imports the single posts. Instead it gateways all ng?s with only one request which means multiple connections at one time. (True or false ?)

Is there a way that gateway.php could pause between two ng?s for ? seconds ?

Regards,
Marv.

A quick and dirty way is to add:

sleep (5);

under this line:

while ($group = $DB_site->fetch_array($get_groups)){

Change 5 to other seconds you want. The script queries the server one by one, it does not create more than one connection at a time. It behaves quite like a normal news reader.

[Blackbeard]

When all these post from newsgroup, if a member replies to the post should it post back to newsgroup, iam i ment to do anything more than basic installation, as i have tested reply, didnt get any error but viewed newsgroup and no reply posted.

am i ment to put anything in prefix, username etc to post back etc or do i have to alter any settings in gateway settings??

[Marv]

A quick and dirty way is to add:

sleep (5);

under this line:

while ($group = $DB_site->fetch_array($get_groups)){

Change 5 to other seconds you want.
That does the trick. Thank you, lierduh http://www.my-smileys.de/smileys1/thumbup.gif

The script queries the server one by one, it does not create more than one connection at a time. It behaves quite like a normal news reader.
I see - I tested the gateway this morning with a commercial newsserver and the script worked like a charme (out of the box). It seems the problem was in fact the free newserver I used until now. But modificated with sleep(3); even the free one works fine.

[xb_]

Actually I need the original message headers.

Ok. I have attached one example for now, as i didn't used the client that allowed me to view/save raw messages with ease. I used outlook, while outlook express seems better for the job. When i exported from outlook to outlook express, it didn't export all of the original headers. (which i had to add manualy again) :|

Anyway knowing this, the next example should be easyer.

[Blackbeard]

When all these post from newsgroup, if a member replies to the post should it post back to newsgroup, iam i ment to do anything more than basic installation, as i have tested reply, didnt get any error but viewed newsgroup and no reply posted.

am i ment to put anything in prefix, username etc to post back etc or do i have to alter any settings in gateway settings??

any chance someone can help why can i not post back to newsgroups, in settings in the email bit, iam ment to add the email like webmaster@learnall.com or mail.learnall.com which one is it. Or is it something else.

please help i have clicked installed

[lierduh]

Ok. I have attached one example for now, as i didn't used the client that allowed me to view/save raw messages with ease. I used outlook, while outlook express seems better for the job. When i exported from outlook to outlook express, it didn't export all of the original headers. (which i had to add manualy again) :|

Anyway knowing this, the next example should be easyer.

There should be no reason for those posts not to be threaded.

Don't tell me you have not turned on "Find Thread by Subject" within AdminCP.

[lierduh]

any chance someone can help why can i not post back to newsgroups, in settings in the email bit, iam ment to add the email like webmaster@learnall.com or mail.learnall.com which one is it. Or is it something else.

please help i have clicked installed

A few things might help other people helping you.

Your current AdminCP setting
Reading Readme file
Post debug info

[xb_]

There should be no reason for those posts not to be threaded.

Don't tell me you have not turned on "Find Thread by Subject" within AdminCP.

hm i didn't indeed :| sorry for the trouble.. and offcourse thanks for the support.
As a sidenote, imho this setting should be enabled automagicly when using mailinglist archive.

[Marv]

Hi lierduh,

are you working an the next release of this hack yet ? if so I would really appreciate if you ( or someone else ) could make this hack producing correct email adresses in the header ( i.e. From: My Name <myemail@domain.com>) .
Many people in the newsgroups which we?re importing are asking for that.

Is there a simple way of including the email-adress from the db to the header information, so that others only have to hit the reply button to drop the author a line?

Regards,
Marv.

[john_rsd]

lierduh

Did you ever consider making the code for the sort by reference/thread a stand alone query that could be run on an existing forum?

Mainly for forums which have a lot of non-threaded entries due to not having correct settings, or where the thread by subject was not used with the original hack.

[Xer]

could anyone can tell me how to make gateway.php runs automatically

i did put the cronjob under VB admincp but it dont works i set [30 * * * * ./includes/cron/gateway.php] and i checked the cronjob log , cant see this job is running

if i put in [30 * * * * ./includes/cron/gateway.php?dubug=1] vb says

Warning: main(./includes/cron/gateway.php?debug=1): failed to open stream: No such file or directory in /home/myabz/domains/myabz.info/public_html/dream/forums/admincp/cronadmin.php on line 56

Fatal error: main(): Failed opening required './includes/cron/gateway.php?debug=1' (include_path='.:/usr/local/lib/php') in /home/myabz/domains/myabz.info/public_html/dream/forums/admincp/cronadmin.php on line 56



thanks :disappointed:

[Marv]

could anyone can tell me how to make gateway.php runs automatically

i did put the cronjob under VB admincp but it dont works i set [30 * * * * ./includes/cron/gateway.php] and i checked the cronjob log , cant see this job is running

if i put in [30 * * * * ./includes/cron/gateway.php?dubug=1] vb says



thanks :disappointed:
You pointed the cron to the wrong DIR. Gateway.php should be located in {forumsroot}/gateway.php . In the vb-cronjob field write: ./gateway.php .

That?s all.

[allan grossman]

Hi lierduh,

are you working an the next release of this hack yet ? if so I would really appreciate if you ( or someone else ) could make this hack producing correct email adresses in the header ( i.e. From: My Name <myemail@domain.com>) .
Many people in the newsgroups which we?re importing are asking for that.

Is there a simple way of including the email-adress from the db to the header information, so that others only have to hit the reply button to drop the author a line?

Regards,
Marv.

I'm gonna politely disagree here unless there's a way to disable the function. Zillions of spammers mine Usenet for email addresses - I'd prefer the addresses remain munged up.

Me? I use abuse@localhost as a return address. I don't know if it does any good but it makes me feel better ;)

[Marv]

I'm gonna politely disagree here unless there's a way to disable the function. Zillions of spammers mine Usenet for email addresses - I'd prefer the addresses remain munged up. Hi Allan,

that?s the reason why I never release me email-adress to the usenet. I use a nospam@myhost.com adress, which spammers can reply to but their emails become deleted automaticly.

But I think - for me and my forums - it wouldn?t be a useful feature. In the past days more and more people which are using clients for posting/reading are flaming why the heck we dont obey the usenet rules which include that every post has to have a correct and replyable emailadress which belongs to the original poster. (For details have a look into RFC 2822 (http://www.faqs.org/rfcs/rfc2822.html)) .
By now some of them say that in case we don?t change the behavior of this script they?ll drop abuse complaints to the network administration and our newsgroup-provider for closing our accounts. And as it looks so far they will obey the rules and disable our account. Even if the newsgroup-provider would reject their request - the network admins won?t.
So,..for us it?s not only a useful feature - it?s a needed one :ermm:

If this could be implemented into this hack I would really appreciate that. Otherwise we`ld have to disable "Sending postings to newsgroups" and would have a read-only forum that makes no sense for anyone.

[john_rsd]

I do not think cron jobs under vb are true crons.

I know the scheduled task manager within vb cannot execute by itself without users actually visiting the site, my assumption is no actual timer routines actually exist and the tasks rely on user hits to the site in order to trigger the code execution (psuedo scheduled) to check time and compare against tasks around same time or they do not execute.

The vb cron jobs may well be similar.

I tried both and they failed to execute, I had to set one going on the server itself.

http://www.vbulletin.com/forum/showthread.php?t=108503

[Marv]

I do not think cron jobs under vb are true crons.

I know the scheduled task manager within vb cannot execute by itself without users actually visiting the site, my assumption is no actual timer routines actually exist and the tasks rely on user hits to the site in order to trigger the code execution (psuedo scheduled) to check time and compare against tasks around same time or they do not execute.

John, the vbcron isn?t a cronjob based on the server time. You?ll find $cronimage in several templates. Every time this template is called $cronimage triggers the vbcronjob. So you?re right - the vbcron is based on hits.

[kmike]

Observant Usenet user emailed me that some replies going to Usenet from our gateway didn't have "References" header and "Re: " in the subject. I investigated this, and it appears that current code is indeed buggy.

Current code first chooses new forum threads and then new posts to send to news server, but SQL query responsible for new threads gets it all wrong.
It gets not only new threads' posts themselves, but also posts to the same thread by the thread author, if the thread was started from the forum. It means that all subsequent forum posts to the thread by the thread author won't get "References" header and also "Re: " in the subject, and therefore won't be threaded in the newsgroups.

The fix is to use special "firstpostid" field in the thread table for determining the first post in the thread.

Replace this code:

$get_newthreads=$DB_site->query("SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle
FROM " . TABLE_PREFIX . "post as post LEFT JOIN " .
TABLE_PREFIX . "thread as thread ON (
thread.threadid = post.threadid
AND post.userid = thread.postuserid)
WHERE post.isusenetpost = 0 AND
post.postid > {$nntp_settings['last_postid']} AND
thread.forumid = {$group['forum']}");

with this:

$get_newthreads=$DB_site->query("SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle
FROM " . TABLE_PREFIX . "thread as thread LEFT JOIN " .
TABLE_PREFIX . "post as post ON
thread.firstpostid = post.postid
WHERE post.isusenetpost = 0 AND
post.postid > {$nntp_settings['last_postid']} AND
thread.forumid = {$group['forum']}");

[PokerFinder]

Yes, I would like to know how to keep the posts to be at a certain amount. I want the total to be 20,000 so any new posts that come in, should be replaced with old posts....clean and add so i dont lose my isp account or website stops. thanks for your help in advance!

is anyone pruning their usenetgroup forum

my group has pulled about 160,000 posts so far and it keeps increasing - im fairly pleased with this in one respect in that the search engines will have plenty of pages to keep them busy which should hopefully increase traffic

however, the more and more posts there are the larger my database will be which means that backing up will be a pain just now but if i leave it as it is for a few months id have to back up a forum of millions of posts - not good

whats everyone else doing about this?

[PokerFinder]

Hi,

I have made changes per your sugestions...thanks...now...I can't get my posts to show up on usenet. it simply stays on my forum but i can get new posts fine.

i did several tests sending and out of 9, only 1 post went through...my port 119 isnt blocked and i have used a nntp gateway before (mynewsgroups :) but switched to this wonderful hack. please help!

ps. am i suppose to have the gateway or nntp/newsgroup settings in vbulletin option in admin cp area? i saw them when i installed it but then it disappeared :ermm:

Observant Usenet user emailed me that some replies going to Usenet from our gateway didn't have "References" header and "Re: " in the subject. I investigated this, and it appears that current code is indeed buggy.

Current code first chooses new forum threads and then new posts to send to news server, but SQL query responsible for new threads gets it all wrong.
It gets not only new threads' posts themselves, but also posts to the same thread by the thread author, if the thread was started from the forum. It means that all subsequent forum posts to the thread by the thread author won't get "References" header and also "Re: " in the subject, and therefore won't be threaded in the newsgroups.

The fix is to use special "firstpostid" field in the thread table for determining the first post in the thread.

Replace this code:

$get_newthreads=$DB_site->query("SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle
FROM " . TABLE_PREFIX . "post as post LEFT JOIN " .
TABLE_PREFIX . "thread as thread ON (
thread.threadid = post.threadid
AND post.userid = thread.postuserid)
WHERE post.isusenetpost = 0 AND
post.postid > {$nntp_settings['last_postid']} AND
thread.forumid = {$group['forum']}");

with this:

$get_newthreads=$DB_site->query("SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle
FROM " . TABLE_PREFIX . "thread as thread LEFT JOIN " .
TABLE_PREFIX . "post as post ON
thread.firstpostid = post.postid
WHERE post.isusenetpost = 0 AND
post.postid > {$nntp_settings['last_postid']} AND
thread.forumid = {$group['forum']}");

[Blackbeard]

iam using a hack of vb.org that shows

There have been # threads and # posts since your last visit!

is there anyway i can exclude my gatewayposts from this count

[Blackbeard]

Is there anyway this can be added as and extra page in the forum and doesnt effect any form of post counts and forum totals.

The reason i ask is its 50/50 on my forum in keeping it, as some of themembers dont like to see all these posts made by none members,but would like to keep it.

Is there anyway it could be made to stand alone on new vb page.

[dayblue]

Is there anyway to get this to import an .mbox file or anything from a Yahoo group, or am I barking up the wrong tree?

Thanks.

[The Prince]

Hi all,

I setup this hack and it half works. I can import messages from th newsgroup but I seem to no be able to post messages to the group. Any idea why this could be happening. And before you ask the variable for posting to usenet is set right.

thanks

[pgrote]

Hey, I had what might be a silly question, but I would really like to have a better understanding of what happened. I was having the same problems that chopperp1 had first stated back on page 20. Which was stated as such:

Hello to All
I'm trying to get this hack working on company's web server. Everything was pretty much straight forward till I got to the point of running gateway.php. That's what I get:
"...pop3 gateway

Gateway version 2.1.0 1 group(s) gatewayed. Bad User Name!Retrieving Protocol E-mails from the 'dl360mail3.yamner.com' pop3 server ..."

I tripple checked all settings and everything looks good. User account for pop3 is correct including password. Web server talks to mail via local subnet (no filtering there). Can't figure out why this thing can't log in.
any clues? :ermm:

I continued to read along to see if I could find a solution provided. Lierduh continued to help chopperp1 out and trying to find the source of the problem. Finally Lierduh responded with this:


Also try this:

In pop.php around line 108, find:

fwrite($this->pop_connect, $message . "\n");

Change the "\n" to "\r\n"

And it ended up working for chopperp1 and for me as well. My question is, what exactly did that do? What is the difference between "\n" and "\r\n" it is something so small and so simple yet solved my problem, so I was just hoping someone (perhaps lierduh) could give me a better idea of what that does or means?

Thanks again for the great help though, and for any further insight you can give!

[allan grossman]

Lookit this -

http://ebassist.com/vb3/showthread.php?t=21609

The only thing I can figure is giganews' server time is off a bit because my host's time is accurate.

I'm not asking you to fix it, I just thought you'd be a bit amused by it ;)

cheers -

allan

[Princeton]

I've been having some problems in regards to posting to usenet ... eg. none of my posts are being posted to microsoft newsgroup

Is there something that I may be missing here?

I tried the hack above with no success.
I have send_message set to 1.

thank you

[flaregun]

great hack, worked like a charm in 10 minutes.

I have one questiosn - I've tried to ready through this long thread but can't find it - is there a way to remove the double posting when you send a message to the usenet from your forum, it seems that you show your post on the forum and then when the chron job runs you get back your own post from the usenet in the thread. I think that might confuse some of my users.

thanks in advance.

- R

[movielad]

I've finally figured out how to put a correct and valid email address within a post, but not the user's own name. It comes out a bit like:

From: username <username@domain.name>

I'm using $newthread['username'] for the username bit, but if I try to use $vbphrase to try and obtain the user's own name it doesn't work. What should I be using to do this?

Many thanks in advance!

Regards,

Martyn

[Marv]

I've finally figured out how to put a correct and valid email address within a post, but not the user's own name.
Hi Martyn,

is this email adress going to the header or the body of the post ? In other words can I reply to the message or do I have to click a mailto:link in the body of the message ?

Beside that - where and how do you get the email adress from the db into the link ? $newthread['username'] should only insert the used nickname but no email adress. May be I?m wrong or got it wrong..

[movielad]

Hi Marv,

The email address is in the headers. I should mention that I cheated - big time! I'm using the LHS side of our company email address for the usernames which makes it much easier to code the email address within the gateway script. Yes, I'd like to be able to pull the email address directly from vBulletin but like the user's name - I'm not sure how to go about doing that.

What I've done already was to amend the following:


// post it
if ($grouptype=='news'){
$response = $news->post($subject,
$group['newsgroup'],
"{$newthread['username']} <$msgid>",


to become:


// post it
if ($grouptype=='news'){
$response = $news->post($subject,
$group['newsgroup'],
"{$newthread['username']} <{$newthread['username']}@our-domain-name.com>",


Yes, I hard coded the domain, but this could easily be replaced with a variable.

Just need to work out how to get the user's own name in the first bit and we're as good as live ;)

Regards,

Martyn

[Marv]

Yes, I'd like to be able to pull the email address directly from vBulletin but like the user's name - I'm not sure how to go about doing that.
Hey guys..anyone out there who could help to fix that problem ? :ermm:
Would be appreciated like hell... :nervous:

[lierduh]

Hey guys..anyone out there who could help to fix that problem ? :ermm:
Would be appreciated like hell... :nervous:

I will include this option in the next version, where each group/list has the option of using the real email. It involves a bit more than just two lines of code although not hard to code.

[lierduh]

great hack, worked like a charm in 10 minutes.

I have one questiosn - I've tried to ready through this long thread but can't find it - is there a way to remove the double posting when you send a message to the usenet from your forum, it seems that you show your post on the forum and then when the chron job runs you get back your own post from the usenet in the thread. I think that might confuse some of my users.

thanks in advance.

- R

Go through all the settings in the AdminCP, you will find the answer.

[lierduh]

I've been having some problems in regards to posting to usenet ... eg. none of my posts are being posted to microsoft newsgroup

Is there something that I may be missing here?

I tried the hack above with no success.
I have send_message set to 1.

thank you

A number of people asking questions without even a debug message, it is virtually impossible to know the reason.

[lierduh]

And it ended up working for chopperp1 and for me as well. My question is, what exactly did that do? What is the difference between "\n" and "\r\n" it is something so small and so simple yet solved my problem, so I was just hoping someone (perhaps lierduh) could give me a better idea of what that does or means?

Thanks again for the great help though, and for any further insight you can give!

You are not using the latest version?

Microsoft servers need return carriage + line feed.

[adieball]

Hi

is there a way to exclude all content from the usenet forum from the latest Threds/Posts module in VBadvanced?

Thanks a lot in advance

Andre

[adieball]

Hi

is there a way to exclude all content from the usenet forum from the latest Threds/Posts module in VBadvanced?



Forget it ... stupid me ... found it ... typical RTFM

Andre

[The Prince]

Hi all,

I've tried to import all the messages from my news server and I ran into 2 problems. First, The gateway.php?debug=1 thing stopped after 2 hours. It's still not close to havin finished. And now if I run gateway.php as part of a task, it crashes after 5 minutes. It still imports some messages, but since I've still got 100,000 to go, this is going to be quite a nightmare.

Also, the dates of the messages are wrong. They all show the date I imported, instead of the date the messages were posted to the newsgroup.

Can anyone help?

thanks

[Nathan_R]

I'm a little late to the game, but I wanted to thank you for a fantastic hack! I absolutely love this mod.

Best regards,
~~Nathan

[Andy R]

Look at the NNTP Settings and look for the use_post_time setting.Also, the dates of the messages are wrong. They all show the date I imported, instead of the date the messages were posted to the newsgroup.

Can anyone help?

thanks