NNTP Gateway (Usenet/Newsgroups)

[The Realist]

I can send newsgroup messages but Iam not receiving them or they are not being posted into the section.

Whats up??

Brian

[antigravityboy]

Hey Gilby
Since you were online, I just wanted to see if I could get some feedback on getting the newsgroup posts to show up on my bulletin board. Right now I get this when I run gateway.php

1 group(s) gatewayed.
Logging in to east.usenetserver.com, group alt.binaries.nude.celebrities.female

I set the forum ID to 9 in nntp_groups which corresponds to the forums I named newsgroups on my bulletin board. Any help would be appreciated.
Thanks

[Brian Cruz]

Gibly,

Now that my board is downloading newsgroup posts using the cron job you specified, I've noticed that the time listed for every post is the time when it was downloaded, not the time the post was made. It didn't do that when I initially ran gateway.php to get the first batch of messages, but only since I've had it running automatically. It's strange looking at the messages and having their time all end with 5! Any idea what's causing this?

[antigravityboy]

Paid install
Hello I have tried everything I can think of but I cannot get this to work on my bulletin board. I would at this point like to pay somebody who know what they are doing to install it for me. I can pay through Paypal so if you are interested please email me and let me know how much you want to do the install.
Thank you

[v0nreagan]

Brian Cruz: AFAICS the date on posts is the date of insertion to db not the date of post on NNTP server. Even if the script would retrieve date from NNTP server I'm not quite sure it's possible to add original post date without messing about with showthread.php, showpost.php and then adding template calls.

[Gilby]

Originally posted by Brian Cruz

Now that my board is downloading newsgroup posts using the cron job you specified, I've noticed that the time listed for every post is the time when it was downloaded, not the time the post was made. It didn't do that when I initially ran gateway.php to get the first batch of messages, but only since I've had it running automatically. It's strange looking at the messages and having their time all end with 5! Any idea what's causing this?

It was programmed to do it that way. It does this because the date on posts are defined by the poster and their clocks are usually not in sync with everyone elses. What this does in the forums then is it causes some new posts to be inserted and appear before posts that were already there.

[newsguy]

Just curious:

Since VB updates sometimes change the sql table structure, what would be the effect when this gateway is installed?

Will future VB upgrades fail if they don't find the table structure they presume is there?

[Gilby]

Originally posted by newsguy
Just curious:

Since VB updates sometimes change the sql table structure, what would be the effect when this gateway is installed?

Will future VB upgrades fail if they don't find the table structure they presume is there?

This hack only adds tables and columns to the database and does not change any existing data structures, so upgrading to a new version should not be a problem for vb. However, if vb changes the data structure, then it could affect the functionality of this hack.

[rawnet]

It would be truly fantastic if a system like this existed which also adds the site's signature to any messages posted through it.

So it's advertising the site on each posting, even if the user doesn't have the site in their signature.

Can this be incorporated into the next release, or what files should we change to achieve this?

Thanks for such a useful hack - adds immeasurably to the forums,

Cheers,

Ross

[etom]

Hi Gilby.

First of all : GREAT WORK !

BUT ;)

Your mime.php seems not to handle the following mime format:

=?iso-8859-1?q?this=20is=20some=20text?=

Many usernames contain special charsets that seem to to be encoded this way.

Where is your mime.php from ? Is there a newer version with support for it ?

PS: More Info about the format:
http://www.faqs.org/rfcs/rfc2047.html

Thank You
etom

[shri]

Quick question incase anyone knows how this works.

How do I configure lynx to not bother with the cookies that global.php tries to send?

[Robert Basil]

Originally posted by rawnet
It would be truly fantastic if a system like this existed which also adds the site's signature to any messages posted through it.

So it's advertising the site on each posting, even if the user doesn't have the site in their signature.

Can this be incorporated into the next release, or what files should we change to achieve this?

Thanks for such a useful hack - adds immeasurably to the forums,

Cheers,

Ross

Ross,

This is very easy to do (We already do it)

Just edit your gateway.php and look for:


$signature .= "$newthread[username]'s Profile: $settings[profileurl]$newthread[userid]\n";
$signature .= "View this thread on SBW: $settings[threadurl]$newthread[threadid]\n";


And add any lines just above those you want appended to every outgoing message.

For example here is ours:


$signature .="Posted via Sportbike World - The internets largest community dedicated to sportbikes.\n";
$signature .="http://www.sportbikeworld.com\n";
$signature .= "$newthread[username]'s Profile: $settings[profileurl]$newthread[userid]\n";
$signature .= "View this thread on SBW: $settings[threadurl]$newthread[threadid]\n";

[rawnet]

Wow.

Tremendous.

May I ask - who do you use as your Usenet provider? Are there any recommendations?

[shri]

I use airnews.net ... awesome service with clarinet and other newsfeeds.

[saab340]

Well Its working great except, all the usenet posts that ends with "?" will get posted as a new post. (ie its not combining them like others.
(It gets quite wild, if there is a active topic, and all of a sudden you have 75 new posts :)
Does anyone else has this, or is it just us?
Thx

[Robert Basil]

Originally posted by rawnet
Wow.

Tremendous.

May I ask - who do you use as your Usenet provider? Are there any recommendations?

We use news ranger http://www.newsranger.com and they work great!

[floleb7]

i have this each time i am running gateway.php

Database error in vBulletin 2.2.2:

Invalid SQL: UPDATE forum SET replycount=replycount+1,lastpost='1016766164',last poster='Christian Daumoinx' WHERE forumid IN () mysql error: You have an error in your SQL syntax near ')' at line 1

mysql error number: 1064

what's the matter ?

[Brian Cruz]

I've noticed several newsgroups threads being downloaded incorrectly. The replies aren't going into the same thread as the original post. It doesn't happen all the time, but often enough to make things look messy. Could it possibly be related to the feature that starts new threads when a reply uses a different subject line? I'd like to try turning that off to see what happens, but I don't know where in the script that is.

[Gilby]

Originally posted by Brian Cruz
I've noticed several newsgroups threads being downloaded incorrectly. The replies aren't going into the same thread as the original post. It doesn't happen all the time, but often enough to make things look messy. Could it possibly be related to the feature that starts new threads when a reply uses a different subject line? I'd like to try turning that off to see what happens, but I don't know where in the script that is.

I think I fixed it a while ago, but I must not have updated it in this thread. So, I've uploaded my latest version. The problem that i am assuming is the case here is that I did not have it match the subject correctly by using the htmlentities function as that's how they are stored in the database (ie. a " would be " in the database, thus they don't match).

Also in this version is a bug fix for long threads that would have tons of references. Some nntp servers complain if this header gets too big, so this fixes that.

E-mail gateway support
Also in this version, you can now gateway with email. This will fetch emails from a POP email account and send via th mail account to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums).

[saab340]

Upgrade...

I dont care to use this pop e-mail thingy, can I still upgrade and use this latest upgrade? How do I disable the pop?

Just want to fix the posts, so they go to their right threads :)

Thx

[Gilby]

Originally posted by saab340
Upgrade...

I dont care to use this pop e-mail thingy, can I still upgrade and use this latest upgrade? How do I disable the pop?

Just want to fix the posts, so they go to their right threads :)

Thx

Just drop the new files in. The email support is a bonus and everything else works the same. It'll only do the email thing if the newsgroup column has an email address in there instead of a newsgroup.

[GameCrash]

Hm... I have problems with ISO-Encoded subjects and textes... for example:

Hans-J?rgen Otten

is send as

=?ISO-8859-1?Q?Hans=2DJ=FCrgen_Otten?=

by some newsreaders (I think). The problem is that the gateway.php does not translate this to Hans-J?rgen Otten. That means that the postings are not added to the correct threads...

I think that

=?ISO-8859-1?Q?{SOMETEXT}?=

means that {SOMETEXT} is encoded.

=2D would mean -

=FC would mean ?

_ would mean a space.

Is there any class or something that would translate this automatically? If not, I will have to write something...

I have found a perl module (http://aspn.activestate.com/ASPN/CodeDoc/MIME-tools/MIME/WordDecoder.html) for that, but not a php class...

[Gilby]

I have just fixed the decode problem that happens in the subject and from headers. The fix makes use of the imap_mime_header_decode (http://www.php.net/manual/en/function.imap-mime-header-decode.php) php function which requires php to be compiled with the imap functions, so it may not work on all setups, but the gateway script will still work as it checks to see if this function is available for use before using it.

[rawnet]

Hi,

I have an ISP who hosts my site, but doesn't provide Usenet.

Has anyone got any recommendations who to use for large Usenet access like this? Wish to install this hack as it would enhance my board greatly, but presumably I need a Usenet provider first? Then do I just put in a username and password within the config?

Regards,

Ross

[Brian Cruz]

Thanks for the quick fixes, Gilby! I think the downloading and insertion of messages is nearly perfect now.

There are only a few very minor things that might be improved upon, whenever you have time. First, a way to truncate very long usernames would be helpful, since long names make the "Thread Starter" and "Last Post" columns much too wide. Also, occasionally I see no name attached to a post, whereas looking at those same posts in a newsreader shows an e-mail address. Lastly, some messages that contain quotes miss coloring the very first line of a quote, like this:

> This first line is not colored
> but this second line is.

Like I said, those are just a few minor problems on an otherwise excellent hack.

[GameCrash]

Thank you - installing the imap support was a pain but this works now :)

[GameCrash]

Graphical interface for this hack now available

With this additional hack you can administrate the gateway via the admin cp. You will only get support in that this thread: https://vborg.vbsupport.ru/showthread.php?s=&threadid=36475

[Erwin]

The mailing list idea is quite cool!!! I won't use it, but it's a fantastic addition! Well done! :)

[floleb7]

Originally posted by floleb7
i have this each time i am running gateway.php

Database error in vBulletin 2.2.2:

Invalid SQL: UPDATE forum SET replycount=replycount+1,lastpost='1016766164',last poster='Christian Daumoinx' WHERE forumid IN () mysql error: You have an error in your SQL syntax near ')' at line 1

mysql error number: 1064

what's the matter ?


bump i try the last version i have always the same problem

[rawnet]

Hi,

I've now installed and setup this hack, and didn't encounter any problems until I ran gateway.php, which then tried to download after running on the server. The contents of gateway.php generated were :


7 group(s) gatewayed.
Logging in to news.gradwell.net, group Alt.Military
Logging in to news.gradwell.net, group Alt.Military.Air-Cadets
Logging in to news.gradwell.net, group Alt.Military.Army-Cadet
Logging in to news.gradwell.net, group Alt.Military.Cadet
Logging in to news.gradwell.net, group Alt.Military.Police
Logging in to news.gradwell.net, group Alt.Military.Retired
Logging in to news.gradwell.net, group Alt.Military.Uk


But according to my forums, there doesn't seem to be any messages downloaded.

Can anyone give me an idea as to what could be going wrong?

Many thanks for what will surely be an excellent hack,

Kind Regards,

Ross Williams

[rawnet]

Spot the NT user here.

Newsgroup names are case-sensitive. Have changed my newsgroups to lower case and it seems to be working now!!

:D

[renaud_]

Hi,

thanks for all,

i can't gateway an POP3 account ...
POP3: premature NOOP OK, NOT an RFC 1939 Compliant server
I can't understand what's the mistake ...

Any idea ?

[rawnet]

Hi,

My gateway.php script is doing the job, but it is timing out.

How do I increase the timeout for this script specifically?

Ross

[Erwin]

Originally posted by rawnet
But according to my forums, there doesn't seem to be any messages downloaded.

Can anyone give me an idea as to what could be going wrong?




Try a different newserver. Some newservers just don't work with this hack. This hack definitely works. Your newserver settings must be right if you are getting that output page. But the newserver may not allow downloading of messages from an external program like that.

[Gilby]

Originally posted by Erwin
Try a different newserver. Some newservers just don't work with this hack. This hack definitely works. Your newserver settings must be right if you are getting that output page. But the newserver may not allow downloading of messages from an external program like that.

It should now work with every server, and if it doesn't let me know.

[antigravityboy]

Yeah
First I wanted to say thanks to GameCrash for the cool interface addon. I finally got the script running on my bulletin board. I just have 1 problem to work out. When it gets picture files, .jpg's it does not display them as links to a picture it shows up as something like this.
begin 644 Catherine_Bell_02.jpg
M_]C_VP!#``$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`
0$!`0$!`0$!
M`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0'_VP!#`0$!`0$!`
0$!`0$!`0$!
M`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`0$!`
0$!`0$!`0$!
M`0$!`0'_P``1"`#(`FP#`2(``A$!`Q$!_\0`'P```@(#`0$!`0$`````````
M``D("@8'"P4$`P(!_\0`1!```00"`0,#`P,"`@<%!@<`!`(#!08!!P@`$1()
etc, etc,

Anybody have any idea on how I can fix this.
Thanks a lot!

[mood]

Gilby,

Since upgrading to VB 2.2.4 my gated groups show me all messages as though they would be from a user in my Ignore list though none are as they are pseudo gateway "guests"

I'm using your last release with the mail list addition.

Any ideas?

PS: Could you get any results with the three way gateway? (list/nntp/forum)

Thanks for your help

[Gilby]

Originally posted by mood
Gilby,

Since upgrading to VB 2.2.4 my gated groups show me all messages as though they would be from a user in my Ignore list though none are as they are pseudo gateway "guests"

I'm using your last release with the mail list addition.

Any ideas?


Can you point me at an example of this? I'm still running 2.2.2 here.

Originally posted by mood

PS: Could you get any results with the three way gateway? (list/nntp/forum)

Nope, not yet.

[Okiewan]

Awesome hack, can't wait to get it working :)
Had problems with my isp's news, went on someone's suggestion here and signed-up with mammothnews.net.

Checked the account with outlook... no problems. Still getting the same issues as before (with other news servers)

Help please?

here is what is displayed using showqueries:


Query: SELECT template FROM template WHERE title='options'
Time before: 0.03379499912262
Time after: 0.034965038299561

--------------------------------------------------------------------------------

Query: SELECT sessionhash,userid,host,useragent,styleid FROM session WHERE sessionhash='9322c3bbd3bd5a1472c3fe8344acbbc5' AND host='66.187.5.96' AND useragent='Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1;'
Time before: 0.041798949241638
Time after: 0.042446970939636

--------------------------------------------------------------------------------

Query: SELECT user.*,userfield.* FROM user LEFT JOIN userfield ON userfield.userid=user.userid WHERE user.userid='16119'
Time before: 0.042775988578796
Time after: 0.043499946594238

--------------------------------------------------------------------------------

Query: SELECT templatesetid,replacementsetid,userselect FROM style WHERE styleid='1'
Time before: 0.044497013092041
Time after: 0.044846057891846

--------------------------------------------------------------------------------

Query: SELECT template,title
FROM template
WHERE (title IN ('gobutton','timezone','username_loggedout','usern ame_loggedin','phpinclude','headinclude','header', 'footer','forumjumpbit','forumjump','nav_linkoff', 'nav_linkon','navbar','nav_joiner','pagenav','page nav_curpage','pagenav_firstlink','pagenav_lastlink ','pagenav_nextlink','pagenav_pagelink','pagenav_p revlink')
AND (templatesetid=-1 OR templatesetid='12'))


ORDER BY templatesetid
Time before: 0.045173048973083
Time after: 0.053406953811646

--------------------------------------------------------------------------------

Query: SELECT COUNT(*) AS count FROM session WHERE userid=0 AND lastactivity>1017184919
Time before: 0.055305004119873
Time after: 0.065440058708191

--------------------------------------------------------------------------------

Query: SELECT COUNT(DISTINCT(userid)) AS count FROM session WHERE session.userid>0 AND session.lastactivity>1017184919
Time before: 0.065888047218323
Time after: 0.068994998931885

--------------------------------------------------------------------------------

Query: SELECT * FROM usergroup WHERE usergroupid=6
Time before: 0.088593006134033
Time after: 0.089588046073914

--------------------------------------------------------------------------------

Query: SELECT varname,value FROM nntp_settings
Time before: 0.27212595939636
Time after: 0.29269301891327

--------------------------------------------------------------------------------

Query: SELECT * FROM nntp_groups WHERE enabled=1
Time before: 0.29306304454803
Time after: 0.29737997055054

--------------------------------------------------------------------------------


Warning: Cannot add header information - headers already sent by (output started at /home2/okiewan/public_html/forums3/global.php:120) in /home2/okiewan/public_html/forums3/gateway.php on line 483
1 group(s) gatewayed.
200 enews1.newsguy.com NNRP server INN 1.90.12 1-Feb-02 ready (posting ok).
381 PASS required
281 Ok
211 10644 348343 359040 rec.motorcycles.dirt
ArrayLogging in to news.mammothnews.net, group rec.motorcycles.dirt
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=80)
Time before: 1.1375050544739
Time after: 1.1587400436401

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=80)
Time before: 1.1590980291367
Time after: 1.1634229421616

--------------------------------------------------------------------------------




Thanks in advance and thanks for the great hack.
In addition, It did work with the test server listed earlier in this thread.

[Robert Basil]

Just to let Gilby know, we are running the 1.9 version of his script and VB 2.2.1 with www.newsranger.com as our news host using a crontab every 15 minutes with about 9,000 usenet messages (100+ added daily), and everything is running great!

I plan on adding a few mailing lists (with the new SMTP support) sometime this weekend and will let you know how it works.

Thanks for the great hack bro!

P.S. We are on our own dedicated server and have not noticed any spikes is processor usage at all. :)

http://www.sportbikeworld.com/status

[Okiewan]

Disregard! Posts are coming down. Got to thinking about the error about "headers already... " Changed the "last post" number, it started working. :)

[Okiewan]

New problem.. trying to get a cron job set up and she no work:


25,55 * * * * lynx-source http://dirtrider.net/forums3/gateway.php?log=cron >/dev/null


sure does look right to me, any ideas?

[Gilby]

Originally posted by Okiewan
New problem.. trying to get a cron job set up and she no work:


25,55 * * * * lynx-source http://dirtrider.net/forums3/gateway.php?log=cron >/dev/null


sure does look right to me, any ideas?

you need a space between the lynx and -source

[GameCrash]

One problem I have is that the messages are not added to the correct threads when they have different titles... fastforwards hack does this without problems (I think he uses any messageid or something like this...)

[Gilby]

Originally posted by GameCrash
One problem I have is that the messages are not added to the correct threads when they have different titles... fastforwards hack does this without problems (I think he uses any messageid or something like this...)

I designed it to do it that way simply because when the subject is changed, most likely it was done because it warrented it's own thread, or maybe it even was supposed to be a new thread.

Go back in this thread and you'll find a change that can be made to change how this works.

[GameCrash]

Really? I didn't see something like this, sorry. I'll search for it :)

[Okiewan]

thanks, will give it a try :)
I'm thinking it's not executing a *.php file?

[trainer]

is there a free newsgroup place that i can setup an accout to use this addon? or do i have to pay a site to get access to newsgroups to work with this addon

[rawnet]

Trainer - there are a few mentioned in this thread - likely that you'll have to pay for it, but only $10/month or something - worry more about your ISP charges as your database size and bandwidth go through the roof!! ;)

[rawnet]

Hello,

How do we stop Usenet postings from appearing when someone clicks on the "New Threads Since Last Visit" link in the forums?

I have installed the hack okay, but received a lot of complaints as my users became lost in the mix between my forum postings and newsgroup postings.

Any ideas how to disable this? Is a hack available? I'd also like to stop them appearing in my Latest Discussion block of vbPortal (but I suspect that's a rather different matter!!!).

Any ideas guys?

Ross

[Okiewan]

I have installed the hack okay, but received a lot of complaints as my users became lost in the mix between my forum postings and newsgroup postings.

Ditto!

[antigravityboy]

Anybody have an idea on what I have to do so that binary attachments are displayed as pictures and not cryptic code.
Still trying to figure it out.
Thanks

[Gilby]

Originally posted by antigravityboy
Anybody have an idea on what I have to do so that binary attachments are displayed as pictures and not cryptic code.
Still trying to figure it out.
Thanks

The problem is with uuencoded attachemnts that are not attached using mime. That should give you some stuff to look for, but I'll address this issue in a future release when I have a chance.

[antigravityboy]

Thanks I will check with my hosting company

[Okiewan]

Anyway to truncate author names ? Either from the newsgroups or ALL authors?

[Brian Cruz]

I've noticed that whenever one of my users includes a link in their newsgroup post, it's gets converted to a tinyurl.com link when viewed on the actual group. Can this function be turned off? It's hard for us to promote our site when the link isn't even ours!

[rawnet]

Hi,

I'm having problems with my gateway.php script not importing all the posts from the Newsgroup.

Really I only need those in the past 3 months or so, not the entire newsgroup which contains hundreds of thousands it seems(the alt.military newsgroup for example).

I'm getting the message that my script has exceeded it's maximum execution time - so, how do I import all the required newsgroup postings, or limit them to the past x posts in a particular newsgroup?

Thanks for the continued support on this dude,

Cheers for now,

Ross

[Gilby]

Originally posted by Brian Cruz
I've noticed that whenever one of my users includes a link in their newsgroup post, it's gets converted to a tinyurl.com link when viewed on the actual group. Can this function be turned off? It's hard for us to promote our site when the link isn't even ours!

TinyURl turns your long links into shorter ones that fit nicely into emails and do not wrap. You can adjust the length at when it turns them into a TinyURL by going in the settings and entering in a higher number.

[Gilby]

Originally posted by rawnet
I'm getting the message that my script has exceeded it's maximum execution time - so, how do I import all the required newsgroup postings, or limit them to the past x posts in a particular newsgroup?

You shouldn't get that error, unless the php installation you are using is running in safe mode, which is likely the case for you. Anyways, you'll need to log into your newsgrop server with another news program and find the id of the message you want to start from and enter that number in the database under the lastmsg column. The number can be found in the header of the message at the end of the Xref line.

[Brian Cruz]

Originally posted by Gilby


TinyURl turns your long links into shorter ones that fit nicely into emails and do not wrap. You can adjust the length at when it turns them into a TinyURL by going in the settings and entering in a higher number.

Thanks Gilby, I should have realized that!

BTW, I noticed something odd while looking through the database with phpMyAdmin. Every thread I look at has a msgid value of <Matthew.Hunter.2anka@toonzone.net>, even non-newsgroup threads. Matthew Hunter is the name of one if my users. Any idea why this is?

[Gilby]

Originally posted by Brian Cruz
Every thread I look at has a msgid value of <Matthew.Hunter.2anka@toonzone.net>, even non-newsgroup threads. Matthew Hunter is the name of one if my users. Any idea why this is?

Well, I don't use the msgid column in the thread table for anything. Although, I do update it and it must be doing it incorrectly. Small artifact from moving from fastforward's hack. I'll have to change that in the next release, but it causes no problems except taking up a little space and a little processor time.

[Brian Cruz]

Originally posted by Gilby


Well, I don't use the msgid column in the thread table for anything. Although, I do update it and it must be doing it incorrectly. Small artifact from moving from fastforward's hack. I'll have to change that in the next release, but it causes no problems except taking up a little space and a little processor time.

Okay. One other thing I noticed is when a user of mine named Babs&Buster4EvR makes a post, his name shows up in the newsgroup as Babs&amp; Buster4EvR, and the e-mail is BabsampBuster4EvR.2akbm@toonzone.net. It seems to be converting the ampersand to HTML. It's not a big deal, but I thought I'd mention it.

BTW, do you plan on adding a custom newsgroup e-mail option someday? That's probably the one thing I miss most from FastForward's hack.

[Gilby]

Originally posted by Brian Cruz
BTW, do you plan on adding a custom newsgroup e-mail option someday? That's probably the one thing I miss most from FastForward's hack.

What did he have in his hack? My plan is to make an email to PM gateway where someone could reply to the post made on the newsgroup privately and it'd go in the forums as a PM. I was also going to make it where the email address only worked for a week, which is why I callled it as timelimit.whatever.com for the domain. However, that might take a while to roll out as i'm woring on other projects.

[Brian Cruz]

Originally posted by Gilby


What did he have in his hack? My plan is to make an email to PM gateway where someone could reply to the post made on the newsgroup privately and it'd go in the forums as a PM. I was also going to make it where the email address only worked for a week, which is why I callled it as timelimit.whatever.com for the domain. However, that might take a while to roll out as i'm woring on other projects.

With FastForward's hack, a custom profile field had to be created that would contain a user-specified e-mail address that they wanted to appear in their usenet posts. Each post was sent to the newsgroup with that e-mail address.

The reason I liked that method was that a user could alter their e-mail address to prevent spambots from picking it up (i.e. myname@yahoo-spamblock.com). I worry that your idea may cause spam to end up in people's PM boxes.

[Gilby]

Originally posted by Brian Cruz
The reason I liked that method was that a user could alter their e-mail address to prevent spambots from picking it up (i.e. myname@yahoo-spamblock.com). I worry that your idea may cause spam to end up in people's PM boxes.

That's the reason that it'd be a time limited address. And it'd bounce any emails that did not have the email address in the to or cc line.

[jarvis]

I'm a UNIX idiot and would like to install this hack. :) I do have access to a shell, but I do not know how to do the first part of this install "running the nntp.sql".

Could anyone give me a hand here? It could be via PM or email to jarvis@mhogaming.com as to keep the thread on-topic.

Thanks in advance!! :nervous:

[GameCrash]

Do you have phpMyAdmin? That would make this much easier...

[jarvis]

Gamecrash,

I do have PhpMyAdmin... now that being said, I'm still lost. :confused:

Is this a completely new dB that I set up? I'm usually not this helpless, but I find myself needing some exact syntax to get going.

Thanks a million! :D

[Sanjiyan]

Originally posted by Gilby
E-mail gateway support
New in this version, you can now gateway with an email list. This will fetch emails from a POP email account and send via php's mail function to the mailing list's email address. To set up the list, in the newsgroup field, enter in the email address to send outgoing posts to. In the server field, enter in the pop3 server, and in the username and password fields enter in, you guessed it, the username and password to your pop account. Note: the email gateway capabilities has not been fully tested, so it might not work with your POP server. Also, it does not work in combo with a news<->forum gateway in the same forum (you can use either kind though in differnet forums within your forums). [/B]

So does that mena, I can get the forums to pick up email from a pop3 account, ie it can pickup email from a mailbox and automatically post it to a forum for me?

[Gilby]

Originally posted by Sanjiyan


So does that mena, I can get the forums to pick up email from a pop3 account, ie it can pickup email from a mailbox and automatically post it to a forum for me?

That's exactly what it means. It can also send posts to an email address too.

[Sanjiyan]

Originally posted by Gilby


That's exactly what it means. It can also send posts to an email address too.

heh get on msn gilby mate.. time for a little chat ^_^

[Sanjiyan]

what about that pop3 file in version 1.9?

do I need to upload that at all?

[Gilby]

Originally posted by Sanjiyan
what about that pop3 file in version 1.9?

do I need to upload that at all?

Yes, that's for the email capabilities.

[Gilby]

Originally posted by jarvis
Gamecrash,

I do have PhpMyAdmin... now that being said, I'm still lost. :confused:

Is this a completely new dB that I set up? I'm usually not this helpless, but I find myself needing some exact syntax to get going.

Thanks a million! :D

Same database. Just go in to that database under phpMyAdmin and there will be a box wher you can type SQL to execute, and underneith that is a file browse input where you can select an sql file to execute. You can run it from either, the file way is easier.

[Sanjiyan]

Post 1 of 2

Gilby,

Attached is a image of a linking problem I was on about on MSN, with regards to the RE: posts not joingin together under the email to forums addon part of this hack.

You asked me to add [FFML] to the prefix section on the MySQL database entry which I did, see the image in post 2 of 2.

Any ideas?

[Sanjiyan]

Post 2 of 2

[Sanjiyan]

Hi,

getting problems if I do a 'Update Thread Info - update thread post counts, original posters, last post date, attachment totals' command via the Up-Date Counters section on the admin cp.

This only started to occur then I updated to Gilbys 1.9 version of the NNTP/Email feed, I belive all the newsgroup based forums (forumid)s are affected, I cant tell cos it bombs out on the first newfeed based forum it tries to update the counters on.


The error given is:

Forum ids:

Processing forum 89

Warning: MySQL: Unable to save result set in /usr/home/wocsnet/public_html/admin/db_mysql.php on line 93



There seems to have been a slight problem with the Sanjiyan's Japanese Anime and Manga Forums database.
Please try again by pressing the refresh button in your browser.

An E-Mail has been dispatched to our Technical Staff, who you can also contact if the problem persists.

We apologise for any inconvenience.

Database error in vBulletin Control Panel 2.2.5:

Invalid SQL: SELECT lastpost,lastposter FROM thread WHERE lastpost='1017967998'
mysql error: Got error 127 from table handler

mysql error number: 1030

Date: Friday 05th of April 2002 02:45:19 AM
Script: http://www.wocs.net/admin/misc.php
Referer: http://wocs.net/admin/misc.php?action=chooser


Also the forum it fails on as well as being NNTP based, is also the email forum, ie the forum that deals with and posts the POP3 mailing list feed.

[jarvis]

I've got this hack working and love it!! Thanks!

Question to everyone, though. What kind of speeds is everyone getting from their servers. Reason I ask, is I log in via OE to my news server and get great throughput, but when I run the gateway.php I get speeds in the 500 bytes/sec area. Big difference... Could anyone have any thoughts as to the speed difference. The only thing I can think is my host has the speed throttled for incoming connections or something.

Thanks!

[Gilby]

Originally posted by Sanjiyan
Post 2 of 2

Take out the brackets, that's implied in the prefix column.

[Gilby]

Originally posted by Sanjiyan
This only started to occur then I updated to Gilbys 1.9 version of the NNTP/Email feed


And an upgrade to 2.2.5? do you know if it happened with 2.2.4 of vb and 1.9 of the gateway?

[Sanjiyan]

Originally posted by Gilby


And an upgrade to 2.2.5? do you know if it happened with 2.2.4 of vb and 1.9 of the gateway?

didnt use 1.9, I used the version you edited for me.

So 2.2.5 and 1.9 are both new.

[Gilby]

> didnt use 1.9, I used the version you edited for me.

What version did I edit for you? If I did, it was from a while back and
would not have the email support.

- Gilby

[Sanjiyan]

nope it didnt have email support, it was the version that you made for me that reported the feedback.. it was a while back ^_^

[jarvis]

Has anyone tried the "prune messages" function with this? It seemed like the first time I did this, it reset my 'last msg ID' field to 0?

[patvdv]

I absolutely *love* this hack and would dearly like to use it on my board. However the long usernames from usenet tend to be cumbersome indeed. Is there a way to hack gateway.php so that let's say only the first 20 characters of the username are uploaded? I am not really worried about chopped off usernames.

[Robert Basil]

Originally posted by patvdv
I absolutely *love* this hack and would dearly like to use it on my board. However the long usernames from usenet tend to be cumbersome indeed. Is there a way to hack gateway.php so that let's say only the first 20 characters of the username are uploaded? I am not really worried about chopped off usernames.

I agree, this would be a nice addition.

P.S. Gilby this is my #1 favorite hack (and I have over 25 of em installed, and have created about 15 of my own) thanks again! :)

[patvdv]

I figured it out myself and it seems to work OK. Ideally I would like to see the username length as a parameter in the nntp_settings table :)

In gateway.php find each instance of:

from_name($message[from])

and replace by

substr(from_name($message[from]),0,15)

to get a 15 character delimited username.

[patvdv]

Originally posted by jarvis
Has anyone tried the "prune messages" function with this? It seemed like the first time I did this, it reset my 'last msg ID' field to 0?

Yes that happens to me to. I guess you can run setlastmsg.php again as a workaround but it would be nice if the last msg ID was preserved automatically.

Better even would be if the NNTP gateway had its own pruning mechanism for those forums in which you import the usenet posts. For e.g. if you schedule the gateway job every hour, it would be *perfect* if it could also check for posts older than x days and remove them from the board :)

Gilby, would you consider such an addition?

[Gilby]

Originally posted by jarvis
Has anyone tried the "prune messages" function with this? It seemed like the first time I did this, it reset my 'last msg ID' field to 0?

Are you using the prune messages function that is in the vb control panel? I can't see how or why that would even touch the nntp table in the database.

[patvdv]

Originally posted by Gilby


Are you using the prune messages function that is in the vb control panel? I can't see how or why that would even touch the nntp table in the database.

Gilby, yes I am (also) using 'Mass Prune' from the AdminCP and the behaviour is consistently the same. Deleting all posts from a forum resets the msgid field. Btw, I am using vb 2.2.4

Superb hack though :) I hope you will continue developing it.

Regards,

Pat.

[Gilby]

Originally posted by patvdv
Gilby, yes I am (also) using 'Mass Prune' from the AdminCP and the behaviour is consistently the same. Deleting all posts from a forum resets the msgid field. Btw, I am using vb 2.2.4


Hmmm... I just tried using the mass prune (the one built into vb), and it didn't change my lastmsg field in the database. I'm confused on why this would even happen though. vB doesn't even know the nntp tables exist when it executes that. unless you are using some other prune script.

Is this mass prune in the nntp graphical interface? If so, then that'll explain it. I don't have that hack installed. In the admin control panel, the 'mass prune' found under the 'Threads & Posts' heading did not change the lastmsg field for me.

[Erwin]

Working smoothly with 2.2.5, if anyone was wondering. :)

[patvdv]

Originally posted by Gilby


Hmmm... I just tried using the mass prune (the one built into vb), and it didn't change my lastmsg field in the database. I'm confused on why this would even happen though. vB doesn't even know the nntp tables exist when it executes that. unless you are using some other prune script.

Is this mass prune in the nntp graphical interface? If so, then that'll explain it. I don't have that hack installed. In the admin control panel, the 'mass prune' found under the 'Threads & Posts' heading did not change the lastmsg field for me.

Gilby, it seems weird to me too but yes I am using the standard vbulletin 'Mass Prune' option to delete all posts. I don't have the graphical NNTP admin panel installed. But I did some more tests and apparently the problem does not appear when I only prune the full contents on 1 single forum. Previously I had selected the parent forum including subforums. I will run some more tests if I get the chance :)

[v0nreagan]

Originally posted by rawnet
Hello,
How do we stop Usenet postings from appearing when someone clicks on the "New Threads Since Last Visit" link in the forums?


Using older stuff from Usenet hack - you need a new field in thread table.

ALTER TABLE thread ADD isusenetpost TINYINT(2) DEFAULT 0 not null;

then open your gateway.php and find:

// Create thread
$DB_site->query("INSERT INTO thread (threadid,title,lastpost,forumid,open,replycount,p ostusername,postuserid,lastposter,dateline,iconid, visible,attach,msgid,prefix) VALUES (NULL,'".addslashes(htmlspecialchars($subject))."','".$date."','$forumid','1','0','".addslashes(from_name($message[from]))."','0','".addslashes(from_name($message[from]))."','".$date."','0','1','$attachcount','".addslashes($message[msgid])."','".addslashes($theprefix)."')");

replace it with (changes enbolded)

// Create thread modded
$DB_site->query("INSERT INTO thread (threadid,title,lastpost,forumid,open,replycount,p ostusername,postuserid,lastposter,dateline,iconid, visible,attach,msgid,isusenetpost,prefix) VALUES (NULL,'".addslashes(htmlspecialchars($subject))."','".$date."','$forumid','1','0','".addslashes(from_name($message[from]))."','0','".addslashes(from_name($message[from]))."','".$date."','0','1','$attachcount','".addslashes($message[msgid])."','1','".addslashes($theprefix)."')");

From now on each thread created by gateway.php will be flagged as originating from usenet (isusenetpost=1)...

Open search.php and find two lines:

$wheresql.=" AND thread.open<>10";

in my case around line 551 and 601...
Directly below each of them add:

// NNTP Gateway mod
if (!$includeusenetpostsingetnew) { $wheresql.=" AND thread.isusenetpost=0"; };
// End of mod

Having isusenetpost in thread table also allows you to separate threrad counts or create separate links for "View new Usenet posts" ( see example here (http://www.satcritics.com/forum)...

[patvdv]

I had the same idea but implemented it differently, without the added table field:

Open search.php and find:

// ###################### Start get new #######################
if ($action=="getnew") {
// generate query
// do it!
if ($bbuserinfo[userid]!=0 or $bbuserinfo[lastvisit]!=0) {
$forumsql=getallforumsql();

After that, add:

// HACK: search for normal or usenet posts
if ($usenet=="true") {
$forums=$DB_site->query("SELECT forum FROM nntp_groups");
$forumsql.=" AND forumid IN (0";
while ($forum=$DB_site->fetch_array($forums)) {
$forumsql.=",$forum[forum]";
}
$forumsql.=")";
} else {
$forums=$DB_site->query("SELECT forum FROM nntp_groups");
$forumsql.=" AND forumid NOT IN (0";
while ($forum=$DB_site->fetch_array($forums)) {
$forumsql.=",$forum[forum]";
}
$forumsql.=")";
}
// HACK: search for normal or usenet posts

Then update your forumhome_newposts template to include 2 links rather than one:

To look for normal posts:
<a href="search.php?s=$session[sessionhash]&action=getnew&usenet=false">

To look for usenet posts:
<a href="search.php?s=$session[sessionhash]&action=getnew&usenet=true">

Not being a PHP wizard I am not 100% sure that this would cover all angles but it seems to work for me fine so far :)

[omniweapon]

I'm not a server admin, and the server is set on safe mode. Which is really pissing me off. Also I have to use phpMyAdmin since they have Telnet disabled. Any help on how to reorganize tables and such would be appreciated.

[akaler]

Hey, Gilby... Thanks for the awesome hack!! =)

I got a question. I see you fixed the mime decode problem in the subject and from fields. I pruned my usenet groups, recompiled php with imap support, and re-imported everything... and sure enough, the froms and subjects were fixed.

There's still some mime in the text, though, that goes away. Especially when some of the header leaks into the text of the message.

Anyway to apply the same decoding to the body of the messages, too?

[akaler]

Oh... I have another question.

Some of the groups I mirror have this one single harassing user who posts something like 20-50 messages a day. I'd like to somehow filter out his messages. I feel that it could make these groups usable again.

I tried pruning by username. Nothing came up... so I then added his various user names as different users on my system. THEN I could prune by username. What then happened was half good. In prune by username, threads he started showed up, but no posts.

Any idea how to filter out the bad seeds? The hacky idea I have above would work if only imported posts, not just threads, could somehow be attached to a username. I know vb.org accomplished this somehow. They grabbed the hacks forums from vbulletin.com, then when people registered here, it would attach their user name to the name they used on their old messages, and whala... full vb user based functions could be applied to old messages.

[Dontom]

Gilby... Thanks for the great hack!!

I would like to exclude Usenetposts which contain a x-archive = no in their header - has anyone already done this? or is there a hint at which part of the script i can sort those posts out?
Thank you!
Tom

[CJi]

I've battled with this hack for the best part of the last 3 hours and I've given up in total frustration. :tired:

No matter *what* I do, all I can get out of gateway.php is this:

1 group(s) gatewayed.
Logging in to news.usenet.com, group alt.sports.airsoft

nothing else.

This is what I have in nntp_groups:

alt.sports.airsoft
<nothing>
98
0
news.usenet.com
1003989135
<password>
1

Help :cry:

[Gilby]

Originally posted by CJi
No matter *what* I do, all I can get out of gateway.php is this:

1 group(s) gatewayed.
Logging in to news.usenet.com, group alt.sports.airsoft

nothing else.


Try the attached nntp.php file in place of the one you are using. this will output the responses from the nntp server.

[Gilby]

Originally posted by Dontom
Gilby... Thanks for the great hack!!

I would like to exclude Usenetposts which contain a x-archive = no in their header - has anyone already done this? or is there a hint at which part of the script i can sort those posts out?
Thank you!
Tom

Look at the statement that checks if it is a forum post that is being read from teh newsgroup (if ($message[useragent] != $settings[useragent]). Add an exclusion for that header too. You may need to modify something else so that the header shows up in the message array.

[Gilby]

Originally posted by akaler

Any idea how to filter out the bad seeds? The hacky idea I have above would work if only imported posts, not just threads, could somehow be attached to a username. I know vb.org accomplished this somehow. They grabbed the hacks forums from vbulletin.com, then when people registered here, it would attach their user name to the name they used on their old messages, and whala... full vb user based functions could be applied to old messages.

A simple SQL query can be used periodically to delete them. Use something like "DELETE FROM post WHERE username='name of bad seed'"

You'll also need to run DELETE FROM thread WHERE postusername='name of bad seed'"

Or you can see my last post and add another thing to the if statement.

[CJi]

ok, thanks for that Gilby. I've got a bit more information. It logs in ok, but then does nothing? I tried it with uk.test last night and got some action, but there are posts in alt.sport.airsoft (not many granted), but still some. I'd have thought it would at least import those few? Here's what I got:

1 group(s) gatewayed.
200 Welcome to Usenet.com, 5.5g6, S8
381 PASS required
281 Ok
211 6 621 626 alt.sport.airsoft selected
Logging in to news.usenet.com, group alt.sport.airsoft

[akaler]

Originally posted by Gilby
A simple SQL query can be used periodically to delete them. Use something like "DELETE FROM post WHERE username='name of bad seed'"

You'll also need to run DELETE FROM thread WHERE postusername='name of bad seed'"

Or you can see my last post and add another thing to the if statement. I think that vB deletes threads by username, even usersnames used in usenet posts. But the SQL query you gave me to delete posts by username... wow, that's useful!

So is the filter heading instructions. Great info, thanks!!

BTW... I've found two possible issues...

The first is posting to Supernews. I couldn't get it to post WITHOUT removing "Message-ID: <$msgid>\r\n" from the $response string. Supernews always ACCEPTED the messages just fine (240), but they just weren't showing up. I guess that either they have a beef with the way you generate the msgid OR they prefer to generate their own, and their server gets "insulted" by the fact you're offering your own. I don't know.

This didn't happen using Giganews, incidentally. Everything posted there just fine w/o modification. I only moved to Supernews a couple days ago, and still have my Giga account 'till the end of the month in fact. I believe it always worked well with Easynews (they have a $1 trial I checked out).

Side note to others reading -- Those services I just mentioned are probably the "big 3" of services offering quality feeds with high rentention, in case anybody is interested. If you want something cheaper but still good ($5mo), check out airnews.net. SUPER cheap (free) but still quality is readfreenews.net.

Reading access is free and needs no password. To get posting access to readfreenews, net, you have to email drechsau+usenet@yuck.net (Mike Horwath), with real name, real email isp email address, username wanted, password wanted.


Alright... next possible bug...

Sometimes when pulling groups that have an a large number of messages.. generally 25-80 thousand... the httpd process (with php as a module) begins swelling in memory access VERY VERY rapidly. Say 2mb growth every few seconds. After 1000 messages, I might have a 400mb process! It will then proceed to fill up the whole of available memory AND swap. But it'll still work. After the first time this happened, I've been killing these processes gone wild, though.

Sometimes this doesn't happen. Sometimes I can tell it to grab 40k messages or whatever... and see perfectly ordinary memory usage... a growth of just a few k every few seconds. Normally, it ends up being something like a 130mb or so after tends of thousands of message havebeen gotten, not just a few hundred.

This tends to happen more AFTER a reboot or a successul restart of Apache. What this seems to indicate to me is that some sort of memory routine isn't being initialized properly... but after php and apache have been used for awhile... everything that could be initialized in apache has been... That's just my theory, I could be wrong.

No matter what it is, though, this might be more of a php issue (i'm using the latest non-beta self-compiled version up on the php.net site... 4.1.2) than a problem with your script. Since it's memory related, I even thought it might be kernel related, but I just updated quite a few revisions and it still happens. I even recompiled php with different options. Same problem. Guess I'll have to wait for the stable release of 4.2 to see if it still happens.

Alrighty... thanks a bunch for your help and continued attention, Gilby! This is a great hack. You should really be compensated for this. Consider having some sort of donation system that parallels vbportals. I say vb portals, because the combination of everything they do to get donations is very successful. Offering betas only if you've donated and the other things they do really does seem to get them more donations than normal.

[CJi]

Is it at all possible that, when a local (vB) reply to a newsgroup is deleted, it gets cancelled off the newsgroup as well?

Just a thought :)

[The Realist]

Installed ok but Im getting the following error:

<br>
<b>Warning</b>: Failed opening 'class.POP3.php3' for inclusion (include_path='.:/php/includes:/usr/share/php') in <b>/home/virtual/site1/fst/var/www/html/forum/gateway.php</b> on line <b>25</b><br>
<br>
<b>Warning</b>: Cannot add header information - headers already sent by (output started at /home/virtual/site1/fst/var/www/html/forum/gateway.php:25) in <b>/home/virtual/site1/fst/var/www/html/forum/gateway.php</b> on line <b>483</b><br>
0 group(s) gatewayed.

Any help?

Brian

[The Realist]

Uploaded class.POP3.php3 to the forums dir and now no errors. But all Im getting is this:

0 group(s) gatewayed.

and no posts from newsgroup in the section?

[Gilby]

Originally posted by akaler
The first is posting to Supernews. I couldn't get it to post WITHOUT removing "Message-ID: <$msgid>\r\n" from the $response string. Supernews always ACCEPTED the messages just fine (240), but they just weren't showing up. I guess that either they have a beef with the way you generate the msgid OR they prefer to generate their own, and their server gets "insulted" by the fact you're offering your own. I don't know.

Most news clients generate their own message ids so I'd guess that there is something with the format that the message id is in. I'm not aware on if there is a standard format for the message id, just that it needs to be unique, but I'll look into it to see if there is one that I was not aware of. It could also be that the supernews server doesn't accept a message id that has the host part of it different from where it's being sent from, so you may want to look at the posts that are made to it and see what the message id looks like. Then use that host for the setting in the nntp gateway.
Originally posted by akaler
Sometimes when pulling groups that have an a large number of messages.. generally 25-80 thousand... the httpd process (with php as a module) begins swelling in memory access VERY VERY rapidly.


Hmmm... I hope it's a problem with something other than my code. ;) However, I'll take a look at this when I get a chance and maybe try out some large imports to see if I can duplicate this.

Meanwhile, you can set a limit of how many posts you want per pull per newsgroup by putting the following code before the line that contains (for ($current = $lastmsg+1; $current <= $max; $current++)), about line 90:
if ($max - $current > 1000) {
$max = $current + 1000;
}

If it's a problem with my code, then php will free the memory when it's done executing the script (it may also do this is the problem is elsewhere).

Originally posted by akaler

Alrighty... thanks a bunch for your help and continued attention, Gilby! This is a great hack. You should really be compensated for this. Consider having some sort of donation system that parallels vbportals. I say vb portals, because the combination of everything they do to get donations is very successful. Offering betas only if you've donated and the other things they do really does seem to get them more donations than normal.

I've been thinking of possibly calling it "donationware" which simply means if it's useful for you and you can afford to contribute some cash, then please donate. So, if you want to donate, you can use this PayPal link: https://www.paypal.com/xclick/business=mail%40gilby.com&item_name=NNTP+Gateway+for+vBulletin+-+Donation&item_number=nntp-gateway

[Gilby]

Originally posted by The Realist
Uploaded class.POP3.php3 to the forums dir and now no errors. But all Im getting is this:

0 group(s) gatewayed.

and no posts from newsgroup in the section?

Did you set up a newsgroup to import? In the nntp_groups table, you need to add a row that has the server, newsgroup, username, password, etc for the newsgroup you want to gateway in. If you don't know how to add a row in the mysql table, you may want to install the GUI that was made for this hack: https://vborg.vbsupport.ru/showthread.php?threadid=36475

[Gilby]

Originally posted by CJi
1 group(s) gatewayed.
200 Welcome to Usenet.com, 5.5g6, S8
381 PASS required
281 Ok
211 6 621 626 alt.sport.airsoft selected
Logging in to news.usenet.com, group alt.sport.airsoft

Hmmm... What is the lastmsg column set to in the database? It should be set to 0, but you can try setting it at 6.

[jarvis]

I've had this hack working great for over a month now, but now I'm getting the following error:

"mysql error: Lost connection to MySQL server during query

mysql error number: 2013"

Has my host possibly changed something that is killing this script? Like I said, I have had this working for some time now and just started getting these error messages. Does mySQL have some sort of timeout setting?

Thanks!

[patvdv]

I have been having the same problem when pulling in a big number of posts from the newsgroups. I wonder if this linked to PHP's max_execution_time? Is there anyway the post retrieval could be 'chunked' into distintinctive processes?

[akaler]

cool, gilby! thanks for all the tips.

if you need access to any of my news accounts, pm me. snews in particular has some insane retention on some groups. hopefully, the memory problems goes away when php or other components are updated. 4.1.2 IS known to be pretty buggy, and it doesn't look like anyone else is having any memory issues, per say.

and i'll definitely kick you some money next pay period! i hope everyone does.

[akaler]

Originally posted by patvdv
I have been having the same problem when pulling in a big number of posts from the newsgroups. I wonder if this linked to PHP's max_execution_time? Is there anyway the post retrieval could be 'chunked' into distintinctive processes? well, i for one, DID need to increase php's max execution time. but i think mysql timeouts are something else entirely.

I changed my max_execution_time in /etc/php.ini to something crazy, like 6000 seconds. That's 6000 CPU seconds, though, so it's really overkill. If you look at CPU time with top for a really long process, you'll see cpu differs from real-time in a very large way.

Again, this shouldn't fix mysql timeouts. Message eva2000 and point him at this thread. He's the quicker picker upper for MySQL!

[akaler]

Hey, Gilby, got a question...

(heck, come to think of it, this may have been asked and answered before in the past 365 something messages -- i'm sorry if it has)

If someone is subscribed to a thread with email notification, do you know if importing messages is supposed to alert them? It seems like if users go to their CP panel, they'll sometimes see subscribed threads as being active, but they'll never actually be notified in any other way.

It'd be cool if they could be. While most users like Usenet via vB, some do say 'why not just use Google?'. The more normal vB options work for threads, forums, users, etc., the more edge you have over something like Google Groups. Google is so BASIC, while vB is more... amazing and far more usable on an everday basis. Especially with all the wonderful hacks!

Anyhoo -- next things I'm going to install are Community Bulletin Generator and some sort of Top 10 Active Threads hack. Maybe the syndicated content hack, I dunno.. Hopefully they work with usenet generated threads. I bet they will. A few other hacks I've installed work GREAT, I'll tell you that! The only one that has any weirdness is Quick Reply. I think it replies to the first message in a usenet thread, and not the last message. This confuses the normal usenet folks who are reading on a threaded basis.

Other than the weirdness with Quick Reply, and the non-updating of subscribed threads and forums... everything is quite fab vB integration wise. This is certainly a superior step up from using Google or your news reader!

[winkler1]

Great hack. We had difficulty getting lynx to invoke gateway.php properly - it would return the forum main page, although we'd visited the site w/lynx to get cookied. Log files show an HTTP/1.0 request; viewing gateway.php with IE5 is an HTTP/1.1, and works fine... any ideas? I just wrote a windows app in the meantime with an embedded IE control on a cron.

Here's a wierd case - at Time1) user posts to vbulletin, and at Time2) gateway.php ships the msg to news server..then @Time3) the user edits the message. Does the new content ever get repropagated? My tests indicate not. Sorry if this is an FAQ, 'edit' is a lousy search term!
Jeff

[Gilby]

Originally posted by akaler
If someone is subscribed to a thread with email notification, do you know if importing messages is supposed to alert them? It seems like if users go to their CP panel, they'll sometimes see subscribed threads as being active, but they'll never actually be notified in any other way.


It's supposed to, and it did when I first tested that feature after putting it in, however, I have begun to wonder if it's actually being sent out or not. It might not.

[Gilby]

Originally posted by winkler1
Great hack. We had difficulty getting lynx to invoke gateway.php properly - it would return the forum main page, although we'd visited the site w/lynx to get cookied. Log files show an HTTP/1.0 request; viewing gateway.php with IE5 is an HTTP/1.1, and works fine... any ideas? I just wrote a windows app in the meantime with an embedded IE control on a cron.

I don't understand your problem. If gateway.php is executed, then it'll import and export the newsgroup posts. However, with the added logging variable in the url that I have in the example, it will not output anything, unless there is a major error, in which case if this is a linux/unix setup, it'll send an email to the cronjob owner.
Originally posted by winkler1

Here's a wierd case - at Time1) user posts to vbulletin, and at Time2) gateway.php ships the msg to news server..then @Time3) the user edits the message. Does the new content ever get repropagated? My tests indicate not. Sorry if this is an FAQ, 'edit' is a lousy search term!
Jeff

It does not repropogate it. The message may have replies and cancelling the message and reposting it would break the threads. What I do on my forums is set it up to have a low edit time and leave it at that.

[winkler1]

I don't understand your problem.

it's pretty strange..gateway is at http://<host>.com/forums/gateway.php. When we use lynx to get the page (leaving off log=cron to keep things simple), the HTML shown belongs to the vBulletin main welcome page... the server log file shows a 200 code, with a big content length being sent back.
Gateway.php is set up, going and working just fine when we visit it with a normal web browser.
Ever seen this problem?

[morrow]

What are the dependencies. How do I edit the SQL Database? Are there any other questions as a newbie that I should ask?

Thanks

[Gilby]

Originally posted by winkler1


it's pretty strange..gateway is at http://<host>.com/forums/gateway.php. When we use lynx to get the page (leaving off log=cron to keep things simple), the HTML shown belongs to the vBulletin main welcome page... the server log file shows a 200 code, with a big content length being sent back.
Gateway.php is set up, going and working just fine when we visit it with a normal web browser.
Ever seen this problem?

I do not know why that would be happening, unless you have your board turned off where only the admin can view the board. Another possibility is that you put the gateway.php in the admin folder instead, but I would guess you'd get other errors if you did that. Aside from those, it's probably some sort of rewrite thing that is defined in the server, such as it redirecting all host.com requests to www.host.com

[Gilby]

Originally posted by morrow
What are the dependencies. How do I edit the SQL Database? Are there any other questions as a newbie that I should ask?


The only dependancies are having vBulletin 2.2 and a connection to a newsgroup server or a pop email account.

The easiest way to edit the MySQL database is to use phpMyAdmin and just hit the browse button when in your forums database and upload the sql file. You can also do it at the command line. In future version, I will probably make it where the database will be automatically checked to see if it's set up and if not it'll load the defaults.

[Neo]

I may sounds like I complete idiot but I have no clue what this does O_O

[Gilby]

Originally posted by neo
I may sounds like I complete idiot but I have no clue what this does O_O

This hack imports and exports posts to/from USENET and other newsgroups into your forums. It allows one to tap into existing discussions that are happening on usenet and making vBulletin live up to it's slogan "Instant Community." :)

I am guessing that you may not know what newsgroups are and that is why you do not know what this hack does. You may want to check out: http://www.faqs.org/faqs/usenet/what-is/part1/

[eiSecure]

Okay, when I installed the hack, everything went great, but when my users post, it only shows up on my forums, and not on the usenet topic.

For example, when I post something, the reply shows up on my forum, but when I use Outlook Express to view the newsgroup, it doesn't show up.

When I run my gateway.php file, this is what it outputs:2 group(s) gatewayed.
Logging in to newscache2.freenet.de, group alt.computer.security
Posting Message from Alex: 500 What?
Posting Message from Alex: 500 What?
Posting Message from chenko: 500 What?
Posting Message from chenko: 500 What?
Posting Message from Alex: 500 What?
Logging in to newscache2.freenet.de, group alt.comp.software
Posting Message from Alex: 500 What?
Posting Message from chenko: 500 What?
Posting Message from chenko: 500 What?

Could somebody help me out?

Thanks,
Alex

[patvdv]

Hi,

I am wondering if it would be possible to extend this hack in such a way that it can check which current vb threads have no NNTP counterpart anymore and that it would automically close the vb thread.

[SeanM]

For some reason my gateway.php file was corrupted when uploaded.. Reuploaded and it is now working!

I've had this running on a previous webserver without any problems. But im now moving to a new server and when I go to run gateway.php I get this error message:

if ($calledby != "interface") { $ad = ""; header("Content-Type: text/plain"); include("global.php"); } else { $ad = "../"; $headers_sent=1; } // Load NNTP classes include($ad."nntp.php"); // POP3 class include($ad."class.POP3.php3"); //include("parser.php"); include($ad."mime.php");
Fatal error: Call to a member function on a non-object in /home/path/to/where/gateway.php on line 54

I've verified that all installation steps were covered correctly.

Any help is appreicated!

Sean

(The path is actually correct, just removed to security reasons)

[leeherron]

Nice package. I have it working, but get these two warnings:

Warning: Failed opening 'class.POP3.php3' for inclusion ...
Warning: Cannot add header information - headers already sent ...

Just thought I'd post and ask what I should do ..

Thanks for the work.

Ps-Edit: Uploaded class.pop3.php3 to the forum directory (where mime.php et al were uploaded) and the problems cleared up. Note: file is included in package, but no reference to it are -- it is not listed as one of the files to upload.

[freeshares1]

After reading the 379 messages in this thread with great interest. I really want to install this. However i have a couple of questions to people that are already using it.

If i added say 2 newgroups. Each with a max of 1000 messages what sort of increase in server load would i see? (Bandwidth + space) I guess this is the only concern i have.

Thanks in advance

Phil

[www.tourc.ca]

I installed this hack at www.tourc.com for automotive forums..... I love it...

I'm just in the process of mirroring all the automotive groups...
it takes a while to catch up with the groups the first few runs of newnews.pl, but is super quick after that... I noticed very little bandwidth increase at all!..... barley enough to measure.... in fact the full mirror (I'm about half done, but I've done some huge groups with 45,000 messages per group and more) of all the automotive groups around 150 or 200 groups will be about 500meg of bandwidth.... (half gig) thats not much, and the runs of newnews.pl after the first mirror take almost no bandwidth....

the biggest problem I encountered was that I had so many groups and messages my searchindex table started getting out of hand... It reached about a gig, and I was only a quarter done, so I killed all the records and set the nntp options to not index the groups that are based on newsgroups and only index local groups.... no that that is done, my whole dbase is arround 40 meg as of may 22, but should be around 80 or so when done!...

the only bandwidth increase I can really see after the initial mirror is done, is all the newsgroup visitors coming to my site! :)

excellent hack or whatever... thanx!

NOTE: if you want to test the hack you can at my site www.tourc.ca and use the following username and pass

username: testuser
password: testuser

enjoy!

[freeshares1]

I love this hack, It seems to take hardly anyspace or bandwidth on my server however i do have 1 small question.

I have 2 newsgroups in a private forum but it has added all the newsgroups to the post count. Is there anyway i can have these 2 forums taken off the post count????

Any ideas????

Phil
www.gamblingforums.net

[asarian]

I hardly think my small addition qualifies as a hack, as I wrote it in about 3 minutes; but here is a very simple way to have users post via the gateway via their regular email address, and not one put together from makeshift message-id.

In gateway.php, search for,


$get_userinfo=$DB_site->query("SELECT signature,usertitle,customtitle FROM user WHERE userid=$newthread[userid] LIMIT 1");
$userinfo=$DB_site->fetch_array($get_userinfo);
$signature = "\n--\n";
$signature .= $newthread[username];


Replace it with,


$get_userinfo=$DB_site->query("SELECT signature,usertitle,customtitle,email FROM user WHERE userid=$newthread[userid] LIMIT 1");
$userinfo=$DB_site->fetch_array($get_userinfo);
$signature = "\n--\n";
$signature .= $newthread[username];
$my_email = $userinfo[email];


In gateway.php, search for,



if ($grouptype=='news'){
$response = $news->post($cat . unhtmlspecialchars($newthread[title]),$group[newsgroup], "$newthread[username] <$msgid>", $message,"Date: ".date("D, j M Y H:i:s O",$newthread[dateline])."\r\nMessage-ID: <$msgid>\r\nOrganization: $settings[organization]\r\nUser-Agent: $settings[useragent]\r\nX-Newsreader: $settings[useragent]\r\nX-Originating-IP: $newthread[ipaddress]$ref");


Replace it with,


if ($grouptype=='news'){
$response = $news->post($cat . unhtmlspecialchars($newthread[title]),$group[newsgroup], "$newthread[username] <$my_email>", $message,"Date: ".date("D, j M Y H:i:s O",$newthread[dateline])."\r\nMessage-ID: <$msgid>\r\nOrganization: $settings[organization]\r\nUser-Agent: $settings[useragent]\r\nX-Newsreader: $settings[useragent]\r\nX-Originating-IP: $newthread[ipaddress]$ref");


- Mark

System Administrator Asarian-host.org

---
"If you were supposed to understand it,
we wouldn't call it code." - FedEx

[omniweapon]

Forgive my being stupid, but where it says "Go into your database and modify the settings in the nntp_settings table." I don't know what to do. I don't know crap about SQL, and our host decided to disable Telnet for us because it's "safer." :p

[omniweapon]

Oh, and i use phpMyAdmin.

[GameCrash]

If you use the graphical interface (https://vborg.vbsupport.ru/showthread.php?threadid=36475) you don't need to know any sql...

[omniweapon]

Thanks for the hack. Only problem is I don't know how to find a usenet group's server. Also, I keep getting an error saying that thread.msgid does not exist.

[dost]

It was working properly but now when I'd like to read a thread from a newsgroup, all users are displayed as if they were on my ignore list. What could cause such a problem?

[jschefdog]

I have been trying to get this hack to work for the last two days. I have read through every post in this thread, but could not find a solution. I have been using the admin interface supplied by GameCrash to setup newsgroug/forum details. I'm having the same problem that many people reported. Everything seems fine except that when I run gateway.php, I get nothing in the forum. After a few seconds I get the message:

1 group(s) gatewayed. Logging in to my.newserver.com, group group.i.want

The IE windows says Done, but nothing was copied into the forum.

I tried some of the debugging suggestions here, including the error reporting line in gateway.php and the last nntp.php debug version that Gilby posted. With those I get

Notice: Undefined variable: calledby in c:\inetpub\wwwroot\test_vb\gateway.php on line 21

Warning: Cannot add header information - headers already sent by (output started at c:\inetpub\wwwroot\test_vb\gateway.php:21) in c:\inetpub\wwwroot\test_vb\gateway.php on line 23

Warning: Cannot add header information - headers already sent by (output started at c:\inetpub\wwwroot\test_vb\gateway.php:21) in c:\inetpub\wwwroot\test_vb\admin\functions.php on line 1545

Warning: Failed opening 'nntp.php' for inclusion (include_path='.;c:\php4\pear') in c:\inetpub\wwwroot\test_vb\gateway.php on line 31

Warning: Cannot add header information - headers already sent by (output started at c:\inetpub\wwwroot\test_vb\gateway.php:21) in c:\inetpub\wwwroot\test_vb\gateway.php on line 491
1 group(s) gatewayed.
Fatal error: Cannot instantiate non-existent class: net_nntp in c:\inetpub\wwwroot\test_vb\gateway.php on line 58

I have no idea what this means. My situation is somewhat unique since I am trying to import a company internal newsgroup, so there is no other server I can try. I can view the newsgroup just fine using Outlook Express with no userid or password setting, so I don't think it is a permission issue. Any suggestions or advice would be appreciated.

[jschefdog]

Originally posted by jschefdog
I have been trying to get this hack to work for the last two days...

I have been doing some more investigation and discovered that our internal news server is using something called NNTPCache (see www.nntpcache.org for more details). Could this cause problems for the NNTP hack?

I did a clean install of vBulletin 2.2.5 on a test system (PHP 4.2.0 and MySQL 3.23.49-nt). I installed NNTP hack 1.9 without installing the admin interface. I used phpMyAdmin to set it per the readme.txt files. Same problem. Everything seems OK, but no forum is ever created and no threads are imported.

[jschefdog]

One more piece of information that might help. I tried running gateway.php again with the debugging version of nntp.php that Gilby posted and showqueries=1. Here is the output (changed the server and group names since they are inside our firewall). The reponse from the server would seem to indicate that it's connecting, but IE says done in a few seconds and it never retrieves any posts. Any tips on how to debug this problem would be appreciated. I'm trying this on a test system, not my production vBulletin install, so I can try just about anything.

1 group(s) gatewayed.
200 internal.server.name.com NNTPcache server V2.3.3 [see www.nntpcache.org] (c) 1996-1998 Julian Assange May 30 2000 ready (posting ok).
Logging in to internal.server.name.com, group group.name
Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=10)
Time before: 0.524605035782
Time after: 0.528924942017

--------------------------------------------------------------------------------

Query: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=10)
Time before: 0.529515981674
Time after: 0.533061981201

--------------------------------------------------------------------------------

Query: UPDATE session SET lastactivity=1024014495,location='/test_vb2/gateway.php?showqueries=1' WHERE sessionhash='b70bb4c090bf80e2c6b0b5e5b1ab3daa'
Time before: 0.535089015961
Time after: 0.536486029625

--------------------------------------------------------------------------------

Query: UPDATE user SET lastactivity=1024014495,inforum='0' WHERE userid='1'
Time before: 0.53685605526
Time after: 0.538607001305

[jschefdog]

I don't know if anyone's listening. I think I have found the source of the problem, but don't know how to fix it. If I add an echo to line 86 of gateway.php (v1.9) as follows:

echo $max = $news->max();

I get the following message when I run it.
1 group(s) gatewayed.
200 news.server.company.com NNTPcache server V2.3.3 [see www.nntpcache.org] (c) 1996-1998 Julian Assange <proff@iq.org> May 30 2000 ready (posting ok).
Logging in to news.server.company.com, group company.news.test
ok).

For some reason the funtion that is trying to determine the max post number is picking up the last part of the prompt returned by the news server login. I don't know PHP so can't figure out why this is happening. Any suggestions?

I put a test post into the forum and discovered that it will add a new post to the newsgroup, but it never downloads any threads probably due to the problem described above.

[jschefdog]

Well, I may not know PHP, but I was able to figure it out by using the echo command to see what was happening. The problem was the extremely long (151 chars) login response being returned by our news server. The fix is to edit line 75 in the nntp.php file:

$response = fgets($fp, 128);

and change the number to 256. After this change all the threads were downloaded.

[jschefdog]

OK, I spoke too soon. Got all the posts downloaded, but I am also seeing the "ignore list" problem. When I open a thread, every post says:

This person is on your Ignore List. To view this post click [here]

Did anyone every figure out what causes this problem? I am running vBulletin 2.2.5 like the others who reported it.

[jschefdog]

A search of the vBulleting community forum uncovered that the ignore list problem was fixed in version 2.2.6. It affected any post made by a userid which was not in the user table. Since the NNTP hack set the userid to zero for all imported posts, they were affected by this bug. If you have this problem try upgrading to v2.2.6.

[saab340]

I just upgraded vbulletin from 2.2.4 to 2.2.6 and now my usenet groups are not downloading??? I see it connects to usergroups, but nothing is downloaded. I am using little modded 1.8 version of nntp.

Thanks for help.

Turned the erro log on, and I am getting bunch of these:

Use of undefined constant lastmsg....
Use of undefined constant forum.....
Use of undefined constant flush....
Use of undefined constant prefix - assumed 'prefix' in <b>/home...

And also my host upgraded their PHP. so thats why I upgraded my vbulletin, it was giving some errors. php 4.2.1

[Bro_Joey_Gowdy]

<a href="http://www.joeygowdy.com/forum/gateway.php" target="_blank">http://www.joeygowdy.com/forum/gateway.php</a>

4 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to .dixie-net.com, group alt.music.christian

Ive inputted all the usenet servers i want downloaded to my site... yet it still doesnt work... any thoughts ?

[Dmitriy]

I don't konw if this has already been asked before, I could not find an answer: Does the php have to be compiled --with-imap in order for this hack to work?

[Bro_Joey_Gowdy]

Originally posted by Bro_Joey_Gowdy
http://www.joeygowdy.com/forum/gateway.php

4 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to *.dixie-net.com, group alt.music.christian


Ive inputted all the usenet servers i want downloaded to my site... yet it still doesnt work...


So does anyone have any thoughts ?

[Bro_Joey_Gowdy]

Lovely - just found out my isp doesnt support usenet --

so does anyone know of any cheap servers that do ?

[Dave#]

Originally posted by Dmitriy
I don't konw if this has already been asked before, I could not find an answer: Does the php have to be compiled --with-imap in order for this hack to work?

no

[Marv]

Hi Gilby,

come into my arms ;) This is what I was looking for since I started my own bb.
I must confess that I?m an old UBB-user and changed the bb-soft just with beginning of last week, so I?m not very familiar with php yet.

Gilby,
I run into some problems while installing your hack and after reading this 300 posts here, I still haven`t found a solution for it.

I?m on a shared server with php 4.2.1 installed and using NNTP Gateway 1.9 with the latest nntp.php you attached here few threads above.

I tried to figure out how this can work for me, but now after 6 hours I?m tiered and about to give up..

The nntp tells me while gateway.php is doing his job:
1 group(s) gatewayed.
200 Welcome to post.usenet.com. No Spamming is allowed! Please vist http://www.usenet.com for premium Usenet Newsgroup services, 5.5h2, S0
Logging in to post.usenet.com, group test
211 12 41913 41924 test selected340 Ok, recommended ID <3d150ce9$1@post.usenet.com>Posting Message from Testuser: 340 Ok, recommended ID <3d150ce9$1@post.usenet.com>

After that it?s running in a timeout and the browser stops - don?t know wether the servers timout or a timelimit in the nntp-script files and
the articles which seems to be send cannot be found in the targeted newsgroup.

Could you give me a little support for this ?

And by the way...you dunno how hard is it to read 300 articles in english...when your own language is german.. uhh.. :ogre: :surprised:


Hope you keep up the good work - IMHO this hack is godlike (guess the UT-players know what i mean..) and I would appreciate, that there would be others like that coming up in the next weeks :knockedout:

As Said: congratz, Gilby..
Marv.

[Marv]

Gilby,

i missed to tell you of another problem. Everytime I click the gateway.php all messages in our board are send again. That means not new articles - the same posts as the last 50 times before :)

Another thingy is that you can?t hit the reply-Button (send eMail to the author) cause the email-adress is totally wrong.. (for example (Boardname Admin, origin-eMail: webmaster@domain.com turns into name "Admin" and email "Admin.6p21i@(tempdirectory).domain.com") - means, it won?t work.

Any suggestions for that, Gilbert?

[Marv]

uhmm...

installed the GUI now and gave the "Manual Syncronisation" a try.
But - havn?t I expected an error? - the echo is
Fatal error: Cannot redeclare stripslashesarray() in /homepages/xx/xxxxxxxxxx/htdocs/forum/admin/global.php on line 33

But line 33@global.php is like this:
// get rid of slashes in get / post / cookie data
function stripslashesarray (&$arr) {
while (list($key,$val)=each($arr)) {
if ($key!="templatesused" and $key!="argc" and $key!="argv") {
if (is_string($val) AND (strtoupper($key)!=$key OR ("".intval($key)=="$key"))) {
$arr["$key"] = stripslashes($val);
} else if (is_array($val) AND ($key == 'HTTP_POST_VARS' OR $key == 'HTTP_GET_VARS' OR strtoupper($key)!=$key)) {
$arr["$key"] = stripslashesarray($val);
}
}
}
return $arr;
}


Okay..I?m a php-newbie so I can?t read the php-script und get the point of what it tries to tell me. I have no glue what?s going wrong..

Is there a personal interpreter out there for me ;) ?

[Marv]

when I read all these articles from people keep on debugging some (mostly the own) minor errors, wouldn´t this addon be better placed in a beta-forum?

I don´t wanna say that this addon was a way early released - but people like me which aren´t to familiar with php and/or mySQL can´t do a debugging by their own and running in a little frustration :ermm:

Just my two thoughts..

[Dmitriy]

This addon has many issues, but they usually realted to servers, not the script.

Basically, this addon is not easy to add and it may not work on all types of servers.

- You need to have access to newsgroup server provided by your webhost and not all webhosts have it.

- Even if you have newsgroup access provided by your ISP, ISP may have it setup in such way that it will not work from outside their network, meaning that if you can access newsgroups from home, you may not be able to use the same account to access from your webserver.

- Even if you have access to news server, your host's news server (or any news server for that matter) may not have the access to newsgroup that you interested in.

- You need to make sure that TCP port 119 is open on your server

- You may need to compile the php with imap support, as stated by Gilby in one of these posts.

- In order to complie php with imap, you also need to install imap c-client library on the server first. From here:
ftp://ftp.cac.washington.edu/imap/
Warning:(a big pain to install)

- If you need to install imap c-client libraries, you need to be sure that imapd is running of your server. If it's not running, you may need to add imapd to your /etc/services file and kill -HUP inetd

As you can see, a lot of things must go right before this addon will work.

Some people lucky and have all of these things already setup on the server, but some are not.

I know, I'be been though all these steps and finally got this thing to work.

The addon itself is a pretty much complete version, not beta, it's just it's a lot more difficult to get it to work than most other add-ons.

[Bro_Joey_Gowdy]

Lovely - just found out my isp doesnt support usenet --

so does anyone know of any cheap ones that do ?

In other words - anyone know of any cheap usenet account providers ?

[Marv]

Originally posted by Dmitriy
This addon has many issues, but they usually realted to servers, not the script.

Basically, this addon is not easy to add and it may not work on all types of servers.

Dmitriy, may be my posting came out wrong.

What you?ve mentioned is absolutly correct. It?s not as obvious to users like me, that an installation note can?t include & cover all types of local server-configurations.

What I would like to see is a little troubleshooting advice in there. As far as I can see most people here have run into 3 to 5 serious problems until they?ve got this addon to work. I would appreciate if the documentation could include those 5 cases with a little more detailed explanations. On the other side this needs additional time and work on it - and I don?t know if I could afford that if I would walk in Gilby?s shoes..

Sure - I can?t expect this. I have to be more than glad, that Gilbert had the time, the know-how and the mood to write this gateway-script - and finally to release this to public.

So..you?re right - the addon isn?t beta. And the more I think this over the more I got a glue, that it?s may be me who is in early beta...

Ermm... umm..for my taste this is a bit too much truth for today.. :speechless:

[Bro_Joey_Gowdy]

hmmm......

i believe im being ignored...

:(

[Marv]

Hi everybody,

their are some things which we should keep in mind before we should use this addon, like the following restrictions which you find on most (german) newservers:

Users of the newsserver are expected to follow this set of rules.
Disregarding the rules may cause termination of access privileges without further notice.

Accurate Sender Address
The e-mail addresses in From:, Reply-To:, and Sender: fields must belong to you and they have to be valid. Using identifiers of other individuals without their permission or e-mail addresses that will bounce is not permitted. (For more details see paragraph 5.3 of the FAQ.)


No Control Messages
Users are not allowed to post control messages at this server. The only exception is to cancel your own articles.



Gilby?s fabulous hack doesn?t include the origin real email-adresses of the boardmembers to the postings which are being send to the newsgroups.
This is a serious problem in most countries and newsgroups and one of the bigger offenses you can run into..

I just don?t know how, but - in smple words - would it be that difficult to read out the eMail-adresses from the db so that Gilby?s addon can generate the valid emailadresse like
From: "Marvin Brandon" <marvin.brandon@gmx.de>

Does anyone has a glue how we could solve this ? Any suggestions ?


Another question..:
In point 2 of the newsserver rules they talk about "No Control Messages ". Is this NNTP-gateway script doing something like that ?

Marv.

[Marv]

hi,

meanwhile the script is up & running. Okay..gateway.php still doesn?t tell me "All done" after it?s finished but that?s no problem at all - beside that it works perfectly and gets the job done. That?s what matters.

May be someone could assist me with a minor problem. gateway.php has to be run in your browser everytime you want to synchronize posts with the newsserver. (BTW: I have to do this with Opera or Netscape cause MSIE always wants to [i]download[/] gateway.php instead of executing the file..)

As someone mentioned a few posts before you could use opera and the built-in refresh-option or do it seperatly in a html-file like this <html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=windows-1252">
<meta name="GENERATOR" content="Microsoft FrontPage 4.0">
<meta name="ProgId" content="FrontPage.Editor.Document">
<meta HTTP-EQUIV="Refresh" CONTENT="900">
<meta NAME="Robots" CONTENT="NOINDEX,NOFOLLOW">
<title>Iframe Newsgroup Synchronizer</title>
</head>

<body>
Iframe Newsgroup-Synchronizer

<iframe src="http://www.yourdomain.com/forums/gateway.php" width="100%" height="400" name="gateway.php in a box">
<p>Ihr Browser kann leider keine eingebetteten Frames anzeigen:
Sie k&ouml;nnen die eingebettete Seite &uuml;ber den folgenden Verweis
aufrufen: <a href="http://www.yourdomain.com/forums/gateway.php">manuelle Newsserver-Abfrage</a></p>
</iframe>
</body>
</html>


We?re on a shared server and so we?re not allowed to run cronjobs. Is it possible to let this job be done by another php.file ?

What I was thinking of:
I guess the most users here - assumed 98 percent of all german bb?s are installed on shared web-servers - can?t do cronjobs.

A solution to that could be to include a - let me call it - "pusher-option" which forces gateway.php to execute without any browser output. (in case the script won?t work proper vb?s db is already configged to send you an eMail-notification every time the error appears)

To reduce bandwith and transfer-volume a new function should be added to this that gateway.php is only executable every 15 (or whatever) minutes.

In practice it would happen like this:
User clicks the newsgroup-forum-link --> this one opens the forum and at the same time is executing gateway.php in a hidden way (without output to the users browser).
After that certain others are viewing this newsgroup-forum but only after [x]-minutes the gateway.php should be executed again by a click on the forum-link.
Otherwise on Boards with a heavy user load your transfer-volume would increase rapidly - and it?s not necsessary to synchronize with the newsserver every 60 seconds and let the users wait another 30 seconds until gateway.php is through with this job until they can view the threads inside..

Meanwhile I?m quite sure this belongs to the request-Forum, but I`ll leave it in here also. May be someone?s stumbling over this posting and.... :classic:

Regards,
Marv.

[Bro_Joey_Gowdy]

anyone know of any cheap usenet account providers ?

[Bro_Joey_Gowdy]

Is this hack compatable with http://www.usenet-access.com ??

(thinks to himself - if I ask enough questions some is bound to reply)

[Marv]

Hi Joey,

it should, Joey. The Newsgate Script is still using standard-ports and
therefore I see no reason why it shouldn´t work.

*a friendly smile through the line*,
Marv.

[Marv]

um...that means.. so far as I remember - hadn?t some of us serious probs to
keep this script going with presupposed username/passwords?
In case thats correct, guess you have to check the whole thread.. so I hope
I?m wrong ;)

[Bro_Joey_Gowdy]

Thanks Marv,

God bless you !!

:D

[mcncyo]

does this hack with verison 2.2.6?

[Bro_Joey_Gowdy]

Originally posted by cyo
does this hack with verison 2.2.6?

Yes

:cool:

(p.s. - awesome avatar - Im a Jesus Freak as well)

[Frenck]

My forum is using the attachment to files hack, but the gateway script seems to put the attachments into the database?!

How to make this script working with the attachments to files hack?

Besides this little problem, the script is running fine! Great work!

[Odoin]

This might have been asked before, but there is so many posts I do not have time to search through them...

My question is... Is there a way to limit the amount of messages inported by the NNTP client? I mean like this one news group has over 5000+ messages and the client will not download them all at one shot it seems to time out before it gets done. Then I have to take the last message id and mod the forum with it, and start over but then the dates are messed up and it looks like all the messages were posted today. Any ideas?

Thank you in advance.

Sincerely,

Mark Laudenbach

[Schorsch]

am I right that when:



- I have a virtual account with 100 MB Webspace, MAX. 100 MB database
- not allowed to run cronjobs

I can forget this useful addon ??

I'm asking since I'm really scared about that my provider is cancelling my account when I'm installing this.

cu
Schorsch

[Dmitriy]

I wonder how I can I exclude the posts in the usenet forusm from being included in the detnew and getdaily searches.

This script downloads a lot of posts from usenet and it confuses many of my members when they want to see new posts since last visit or new posts in the past 24 hours.

Can someone suggest how to hack the search.php to exclude certain forums from being counted towards the latest posts?

[Odoin]

This is something I need as well... When 500 messages are posted from the news groups a day it makes it hard for the users doing searchs.

[beheader]

Originally posted by Odoin
This might have been asked before, but there is so many posts I do not have time to search through them...

My question is... Is there a way to limit the amount of messages inported by the NNTP client? I mean like this one news group has over 5000+ messages and the client will not download them all at one shot it seems to time out before it gets done. Then I have to take the last message id and mod the forum with it, and start over but then the dates are messed up and it looks like all the messages were posted today. Any ideas?


There are only two ways I have been able to get around this was to either run a dnews server behind my firewall with a suck account from http://www.newshosting.com, or setup a newscache under linux.

Then set the limits so that only new posts are downloaded and not the 47,000 old posts.

Other than that- unless the script author makes a mod (or someone else) we are stuck with that situation.

[beheader]

Hi all,

Here's my notes on getting this mod working fulling under medium volume....

I. Installation and Config

The installation and config instructions were fine.

II. Server Choices

The mod as it stands communicates with any news server (tested 5 different servers: INN, Leafnode, Leafnode+, Dnews, Hampster, and Rumormill).

Assessment:

INN: Too industrial strength, pain to configure... don't do it (I did and now I need therapy)

Leafnode and Leafnode+: Excellent and simple. Must install from source. Ability to limit initial draw of articles from up stream server. Operates in "suck" client mode only. This is GPL software, and is currently still maintained.

Dnews: Excellent and simple. Faster than Leafnode variations. However is expensive! $95.00 for a single concurrent connection- and the prices only climb from there....

Hampster: Forget it. I'm not even going into it. But it's old, slow, and unsupported.

RumorMill: Forget it, old, Slow, and unsupported. (However if ti did work it would be a great use for an old Mac you have laying around)

My final Choice:

Leafnode.

Why? Because Leafnode compiled flawlessly, added newsgroups flawlessly, is free, is GPL, has initial suck limits in place for new group additions, has limits for message size in place (keeps some idiot who posts a binary to a text group from screwing your server and forums up, and runs on linux- which saved me a win2k license....

III. Issues with the script itself.

The mod itself works brilliantly, with the exception of the following issues:

1. If you pull a text group directly from a server with a high retention (Like 4000 articles), the download will *timeout* (Along with the php script), leaving some house keeping things un-done in regard to the import of the messages. One of these things is fairly severe, in that when the gateway script is run again, the entire newsgroup will be pulled *again*- since the message numbers are not update when the import fails.

2. There are no size limitations in place in the script proper to avoid the importing of a multipart CD image. Just today, I had to reconfigure my local newsserver to avoid a binary that was posted to comp.sys.mac.games.action- the binary should not have been there, but 680 megs of usenet posts ended up in my sql database. This is a mess and not fun.

3. It appears that the mod doesn't poll for an initial group download in the same way as a standard newsreader.

Three separate caching servers (dnews, Leafnode, and Leafnode+) all did not recognize the poll to an empty newsgroup, and subsequently download the articles. I had to subscribe to the newsgroup first in Outlook, then the messages would download normally.

While I have not had a chance to check the code for this function, I will do so in the coming days and find out if I can isolate a mod for the mod to make sure that a subscription in the control panel will be enough to trigger the newsgroup to go active.

IV. Final config- how to make it workable....

My solution consists of:

1. a suck feed from http://www.newshosting.com ($25.00 a month- 500 MB a day)

2. Internally configured Leafnode server with initial post fetch, and size limits in place.

3. CBQ (Class Based Queing) implemented on a Linux firewall to keep usenet traffic from affecting other servers on my system.

Essentially, the Leafnode server is throttled to 56k of incoming bandwidth, and the forums server connects to it via 10/100 Full Duplex Ethernet.

It was at this final config that the usenet section at ASWP.net became usable with any kind of volume. It was a big job, and very intense in the level of tweaking needed, but we have it classified as rock solid at this point.

So here's the short list of what you need to make it workable:

1. Sucking usenet server, which you have control over, or that someone is willing to alter the config for.

2. A fast semi-dedicated server box with a lot of disk space.

3. The ability to throttle your bandwidth.

Well, anyhow- thats my take on it....

[ThunderVipor]

I'm very interested in this functionality, but have no experience with MySQL (in case it isn't obvious by my subject line :) ). Can someone please tell me how to run the nntp.sql script?

Thanks much.

[glenvw]

This is what I am looking for for sure.

Does it actually bring visitors from the newsgroups to your site? That would be awesome.

I am looking for someone intersted in installing. Any takers?

PS I also do not know how to run the nntp.sql script.

[SeanM]

I have installed this script numerous times.. It is really quite easy, if you need some help send me a message @ webmaster@weathertalk.net and I will help you as much as possible.

SeanM

[Brian Cruz]

Is Gilby still around? I'm hoping he'll update this for vB3 when it comes out, particularly to take advantage of the new threaded view feature.

[glenvw]

oops

[fechten]

Installed and this works great for pulling down the NNTP feed, but when I try to reply to a usenet thread from the forum and then sync it up I get:

441 Posting Failed (Message-ID doesn't conform to NNTP standards)

I have html, vb code, smilies, and icons turned off for the nntp forum. Is there any other config that I should do or do posts from vb to usenet not work with this gateway?

Thanks,
Craig

[SRozhon]

Thank you so much for those pics Sanjiyan! They helped me figure out what I was doing wrong. Great idea!! Oh, and this is an awesome hack Gilby! Thanks!!

[SRozhon]

Originally posted by Gilby


It should now work with every server, and if it doesn't let me know.

I think I have it set up right now, but I keep getting an IE message that says it isn't able to connect. I am using my isp server. Is that the problem?

When I try to syncronise manually with the GUI by gateway I get this error message:
Warning: Cannot add header information - headers already sent by (output started at /home/virtual/site1/fst/var/www/html/forums/admin/adminfunctions.php:19) in /home/virtual/site1/fst/var/www/html/forums/gateway.php on line 20

Fatal error: Cannot redeclare stripslashesarray() in /home/virtual/site1/fst/var/www/html/forums/admin/global.php on line 33

Please help! This is the coolest hack I've seen yet!! :bunny:

[SRozhon]

ok, I took everything out and started over. This time I used the GUI by gamecrash. I am no longer getting the above error this time, but get this now instead.

2 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to netnews.insightbb.com, group comp.graphics.apps.photoshop
Not connectedNot connectedCould not connect to NNTP-serverLogging in to netnews.insightbb.com, group comp.graphics.apps.paint-shop-pro
Not connectedNot connected

I'm not sure why it isn't connecting. I think it is set up right. Anyone have any suggestions??

[certify]

This is exaclky what I am looking for. :)

Great script!

[Gardener]

This is the first hack I have tried to install for VB, and I love it. The install went well with very few problems, except for one of the news groups I added. I added alt.binaries.pictures.gardens as one of the groups, and most of the pictures just show up as a page full of characters instead of the actual picture. I don't think this is a script problem, but rather a problem of not having something installed in either Apache or PHP to decode these pictures. Can someone give me a hand.

Thanks

[Gilby]

Originally posted by Gardener
I don't think this is a script problem, but rather a problem of not having something installed in either Apache or PHP to decode these pictures. Can someone give me a hand.

This is an incompatibility with script where it does not support attachments that are not mime attachments and are uuencoded, which is what OE does for attachments.

[Gardener]

Thanks. I noticed that some of the pictures do show up as attachments, but most don't. I'll just have to remove that group. Thanks for your help.

[Gardener]

Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

[Gardener]

Originally posted by Gardener
Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

Anyone?

[Gilby]

Originally posted by Gardener
Is there any way to attach a footer to the end of postings made from my site and posted to usenet? Such as "Posted via my website".

Find the lines (around line 550) that start with "$signature .= " and change those to how you want the signature to be.

[Gardener]

Thanks for the help. I had almost forgot about asking this.

[MaXxed]

I have two issues with this and would welcome some help :)

1. Using the graphical interface, I cant seem to move a newsgroup (rec.travel.australia+nz) - I think this may be due to the '+' in the name. When clicking on the groups settings, I just get blank fields.

2. How does this hack handle outgoing attachments? I have them disabled by default as I am using text groups, but I am curious what would happen.

Thanks for any/all help.
Max.

[SRozhon]

I am getting this after manual sync. what should I do?

1 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to netnews.insightbb.com, group comp.graphics.apps.photoshop
Not connectedNot connected

[Gardener]

Gilby,

I posted before aobut trying to use this hack to download some biniary groups, but I was having trouble with the uuencoded attachments. I've been looking for a way to solve this, and someone sent me here: http://www.php.net/manual/en/function.base64-decode.php

Would this work? I'm willing to pay for this if you can get it working. PM me with the details.

Thanks

[qnx]

I am testing the NNTP gateway to the public news server at inn.qnx.com and trying to pull all articles from the group "qdn.cafe" (less than 600 posts). it always fails after message number 562:

...
Getting message number 561: Thread found; 'Re: This is great....' from Bill Caroselli \(Q-TPS\)
Getting message number 562: Thread found; 'Re: This is great....' from Mario Charest

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 10 bytes) in /var/www/html/forums/admin/functions.php on line 1318

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 11 bytes) in Unknown on line 0
...

the php.ini has the default setup of 8M memroy. and I have installed the gui interface from GameCrash.

the posts in that newsgroup are all less than 1M. does the gateway code NOT release the memory after pulling each article? this means we will run out of 8M php allowed memory if we have many posts?
Is there a way to fix it without increasing the 8M limit?

Thanks!
Frank

[qnx]

Now it's getting worse!
I doubled memory_limit to 16M in php.ini and try to hit gateway.php again, this time it gets a few more posts:
...
Getting message number 568: Thread found; 'Re: This is great....' from Eric Norton
Getting message number 569: Thread found; 'Re: This is great....' from Mario Charest
<br>
<b>Fatal error</b>: Allowed memory size of 16777216 bytes exhausted (tried to allocate 644127 bytes) in <b>/var/www/html/forums/admin/functions.php</b> on line <b>1330</b><br>
<br>
<b>Fatal error</b>: Allowed memory size of 16777216 bytes exhausted (tried to allocate 0 bytes) in <b>Unknown</b> on line <b>0</b><br>

The worst part is now if I go to the forums main page, I see all the duplicates!! it seems gateway.php doesn't know that it has already pulled some of the posts. It gets ALL posts again! Doesn't gateway.php updates the "last_post" count after successfully pulled ONE post? it seems gateway.php only updates the "last_post" count when it finishes polling all posts. If it fails (in my cases), it lost the opportunity to update the "last_post count.

Frank

[qnx]

In addition to the "last_post" count, I suggest we use the message-id from the post as a unique id to avoid duplicates. before we insert a new post, we do a select on msgid to make sure it is NOT a duplicate. if we index the msgid, this overhead should be trivial but the benefit is huge. You can import multiple times without worrying about duplicates. btw, we should probably raise the msgid to varchar(255) which is the largest that we can index I believe. we can have better unique-ness with 255 chars I hope.

Frank

[MaXxed]

Is there a problem with subscribing to a usenet thread and getting email notifications of replies? It doesnt seem to be working for me?

[ryancooper]

Hey Guys; great hack!
I got it to work with a test server but when I try to use my ISP's I get this and nothing else:

1 group(s) gatewayed.
Logging in to news.epix.net, group rec.arts.disney.parks

can this mean I am not getting into my server? I have my user name and password set up.

[ryancooper]

Hey Guys; great hack!
This is the error I get when I try to post a message?

Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?

What does that mean?

[Jose]

Nice hack! Easy to install, and works nicely. My only wish is to be able to set up a newsgroup or POP email that is read-only. That is, posts/emails are read and posted to the forum, but forum replies or new threads are not sent outside the forum. I'm thinking of a different value for the 'enabled' field, or via the use of the "undocumented" prefix.

Any volunteers to try to do this? If no takers, I might do it next weekend.

[Gardener]

Jose,

I use the hack too, and love it. You should be able to set the news group to read only in the Forums & Moderators >> Permissions section of your admin panel. I haven't tried it for the news groups, but it should work.

[Jose]

Originally posted by Gardener
Jose,

I use the hack too, and love it. You should be able to set the news group to read only in the Forums & Moderators >> Permissions section of your admin panel. I haven't tried it for the news groups, but it should work.

Yeah, but I wanted to use it so that I could import resources (like from a moderated newsgroup), and give group members the ability to read and discuss them in the forum, without the replies leaving the forum. I looked at the code and it's not a difficult change to do. I'll probably do it and test it over the weekend and post here if anyone is interested.

Jose

[Chris732]

Originally posted by Gardener
Gilby,

I posted before aobut trying to use this hack to download some biniary groups, but I was having trouble with the uuencoded attachments. I've been looking for a way to solve this, and someone sent me here: http://www.php.net/manual/en/function.base64-decode.php

Would this work? I'm willing to pay for this if you can get it working. PM me with the details.

Thanks

Yes that would be a nice feature :) Let me know if you find out anything

[Souly]

Originally posted by ryancooper
Hey Guys; great hack!
This is the error I get when I try to post a message?

Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?
Posting Message from Ryan C.: 500 What?

What does that mean?

I have the same problem - any Answers?

[efoo]

just installed this hack in order to archive some email lists and it works great - thanks. However, I've got one hurdle to overcome when it comes to automating the run of gateway.php. My board is off-limits to guest users, so trying to execute gateway.php without first logging into the board manually results in a "you are not logged in" page when the script tries to make all the posts. Is there any way to write a cron job that can both log into the forum and run gateway.php (from lynx)?

[efoo]

just installed this hack in order to archive some email lists and it works great - thanks. However, I've got one hurdle to overcome when it comes to automating the run of gateway.php. My board is off-limits to guest users, so trying to execute gateway.php without first logging into the board manually results in a "you are not logged in" page when the script tries to make all the posts. Is there any way to write a cron job that can both log into the forum and run gateway.php (from lynx)?

[Cluster One]

I'm having a bit of trouble.

What i've done is I've installed hamster to pull messages from various newsgroups onto my computer, then setup the addon to download these headers from my pc to the website (Basically my ISP has IP masked access that only allows my IP here to login, but not my websites IP)

My problem is when I run the gateway script, it goes for a few minutes then asks me to download the file? Did i mess something up?

I mean it is actually obtaining the headers and putting them in the forums, its just annoying to get that window.

[MaXxed]

I have a potential issue. I have been running this hack on my forums for a few months now without problems, it has been working excellently.

Today I have found that my news provider has suspended my posting privelages. This may imply that I will need to change providers.

Should I simply change providers (and the settings on the hack) - I am concerned that I will have thousands of duplicate posts on the first sync.

Any idea how I could make the transition painlessly?

[Gardener]

I'm currently using this hack for newsgroups, and it works great. Now I'm trying to add a mailing list archive, and this is the error I am getting:

POP3: premature NOOP OK, NOT an RFC 1939 Compliant server

[SRozhon]

I love this hack. I use it to pull in several newsgroups too. I hope you are able to get the maillist to work. Please share if you do. :-)

[Gardener]

Well, I can get it to work on my ISP's POP3 account, but not on the POP3 accounts on my server. I'm still trying to find out how to fix this.

[Domenico]

I installed this hack and I love it but quoting a message is not done right at all.

Can this be fixed because quiting is very important using usenet ?

[qnx]

is this hack dead?
I noticed the author (Gilby) hadn't said a word in the forum for more than three months.
Lots of questions/suggestions left unanswered.

[Gardener]

Originally posted by qnx
is this hack dead?
I noticed the author (Gilby) hadn't said a word in the forum for more than three months.
Lots of questions/suggestions left unanswered.

I think so. His last post was on 10-26-02.

[nuno]

Whenever i run gateway.php i always get an error, looks like a connection timeout. :confused:
It prompts me to download gateway.php, i get messages from 1999 until 2000, about 200K of data and then times out. :confused:
If i rerun gateway.php again, all messages are duplicated.
What am i missing? :help:

[nuno]

Whenever i run gateway.php i always get an error, looks like a connection timeout. :confused:
It prompts me to download gateway.php, i get messages from 1999 until 2000, about 200K of data and then times out. :confused:
If i rerun gateway.php again, all messages are duplicated.
What am i missing? :help:

[nuno]

Whenever i run gateway.php i always get an error, looks like a connection timeout. :confused:
It prompts me to download gateway.php, i get messages from 1999 until 2000, about 200K of data and then times out. :confused:
If i rerun gateway.php again, all messages are duplicated.
What am i missing? :help:

[Gardener]

Originally posted by nuno
Whenever i run gateway.php i always get an error, looks like a connection timeout. :confused:
It prompts me to download gateway.php, i get messages from 1999 until 2000, about 200K of data and then times out. :confused:
If i rerun gateway.php again, all messages are duplicated.
What am i missing? :help:

Are you talking about downloading messages from the year 1999 and 2000? How many messages are you trying to download?

[nuno]

About 27,000.
I get all the messages prior to June 2000, that's about 600 posts.
Then i get a message saying the connection to the host was reset.
I can't get any messages after June 2000. :(

[nuno]

About 27,000.
I get all the messages prior to June 2000, that's about 600 posts.
Then i get a message saying the connection to the host was reset.
I can't get any messages after June 2000. :(

[nuno]

About 27,000.
I get all the messages prior to June 2000, that's about 600 posts.
Then i get a message saying the connection to the host was reset.
I can't get any messages after June 2000. :(

[nuno]

About 27,000.
I get all the messages prior to June 2000, that's about 600 posts.
Then i get a message saying the connection to the host was reset.
I can't get any messages after June 2000. :(

[Dean C]

Test

[Xenon]

another test

[Gardener]

Originally posted by nuno
About 27,000.
I get all the messages prior to June 2000, that's about 600 posts.
Then i get a message saying the connection to the host was reset.
I can't get any messages after June 2000. :(

It sounds like a problem with your news provider, not the script. One not though, if you are trying to do this on a shared server you do know that your host is going to drop you soon don't you? This mod can be very resource intensive. I'm on a dedicated server with dual PIII with 512 MB ram and a 27000 post dump will just about bring my server down.

[nuno]

Originally posted by Gardener


It sounds like a problem with your news provider, not the script. One not though, if you are trying to do this on a shared server you do know that your host is going to drop you soon don't you? This mod can be very resource intensive. I'm on a dedicated server with dual PIII with 512 MB ram and a 27000 post dump will just about bring my server down.
It's a Dedicated Server.
The news server is news.php.net.

[nuno]

Originally posted by Gardener


It sounds like a problem with your news provider, not the script. One not though, if you are trying to do this on a shared server you do know that your host is going to drop you soon don't you? This mod can be very resource intensive. I'm on a dedicated server with dual PIII with 512 MB ram and a 27000 post dump will just about bring my server down.
It's a Dedicated Server.
The news server is news.php.net.

[Gardener]

The news server is news.php.net.

You are never going to get it downloaded. It's just too much. I tried downloading php.general on a test server here at home with a P4 and 1 gig of ram and it choked on the amount of data. There are just too many posts there to download them all. Your best bet is to go back about 6 months and get a post number and enter that number for the last post number.

[nuno]

Originally posted by Gilby


You shouldn't get that error, unless the php installation you are using is running in safe mode, which is likely the case for you. Anyways, you'll need to log into your newsgrop server with another news program and find the id of the message you want to start from and enter that number in the database under the lastmsg column. The number can be found in the header of the message at the end of the Xref line.

I'm getting timeouts too.
PHP is not running in safe mode.
If i change the lastmsg column field in db and re-run gateway.php all i get is a DNS error, a file not found error.

[nuno]

Originally posted by Gardener


You are never going to get it downloaded. It's just too much. I tried downloading php.general on a test server here at home with a P4 and 1 gig of ram and it choked on the amount of data. There are just too many posts there to download them all. Your best bet is to go back about 6 months and get a post number and enter that number for the last post number.

heheh done. :)
Grabbed 30k posts, all at once, and no connection timeout, you just have to fool the browser. :D

[Tigga]

d

[gopherhockey]

This hack absolutely rocks. By far this is going to add more functionality to our forums than anything I've seen.

Thank you, thank you THANK YOU!!!

INSTALL............... :banana:

[gopherhockey]

I apologize if this has already been answered here, but this thread is getting long and my searches resulted in no answers.

What I would like to do is have the ability to post a sticky thread into the forum without that one being sent out to the related usenet newsgroup.

I guess this becomes a 2 part question:

1. Can you post a sticky thread and will that screw up the system?

2. Is there a way to post in my forum and not have it be sent out to the public.

(I tried running the setlastmsg thing, but that seems to only work on incoming, not outgoing)

Thanks!

[Schorsch]

edit: sorry for double posting!

[Schorsch]

great hack!! :bunny: some questions:

how can I leave out the signatures and user titles when posting to a group ?

edit: and how can I change the quote color ?

and when I look into a thread I can't see the poster name, always just "guest"

[SRozhon]

Originally posted by nuno


heheh done. :)
Grabbed 30k posts, all at once, and no connection timeout, you just have to fool the browser. :D

Can I ask How did you do this? Mine always timesout. I love this script though.

Another question--I'd like to be able to delete the posts without opening them up first (sometimes I can tell what is spam or unrelated to the topic without opening them) and since I edit the garbage out this would make things much easier. Is there a way?

Question 3:
Is there a way to not include the previous replies, etc. that show up under the users post? A lot of times it is just the same thing over and over in each persons post--and this just seams to take up space. Just wondering... Thanks!

[Schorsch]

I have another important question:

How can I exclude the newsgroups forums from the search function ??????

[Marv]

First - it?s a really great hack so far and I want to thank the author for sharing it with us.

But as Schorsch mentioned before - most people in serious newsgroups are not very amused by the produced signature.
Couldn?t there be a way that this hack reads out the mail-adress from the database and takes that as signature ? May be like

Contact: emailadress@poster.com
---
posted via NNTP-gateway through http://board.yourdomain.com
Abuse: abuse@yourdomain.com


Not at least I would aprecciate that the mailadress of the poster is his own. Actual the poster-adress in those ng-postings is set to "membername.12345@yourdomain.com - and that?s far from beeing neat if you have a <catchall>-mailfunction running on your domains - never had that much spam before... :surprised: <args>

And beside that other ng-users can calm down.. - actual it?s really producing lots of trouble...

Just my two thoughts - but I think it?s really important.

Marv.

[Marv]

Oups..forgotten: does anybody know if this hack works with the 3.0 RC3 ?

Thx,
Marv.

[Gilby]

Originally posted by MaXxed
2. How does this hack handle outgoing attachments? I have them disabled by default as I am using text groups, but I am curious what would happen.

It'll post a URL to the file on your server.

[Gilby]

Originally posted by Gardener
Gilby,

I posted before aobut trying to use this hack to download some biniary groups, but I was having trouble with the uuencoded attachments. I've been looking for a way to solve this, and someone sent me here: http://www.php.net/manual/en/function.base64-decode.php

Would this work? I'm willing to pay for this if you can get it working. PM me with the details.

Thanks

That won't work. The script currently supports mime encoded attachments but doe snot support uuencoded attachements.

[Gilby]

Originally posted by Jose
Nice hack! Easy to install, and works nicely. My only wish is to be able to set up a newsgroup or POP email that is read-only. That is, posts/emails are read and posted to the forum, but forum replies or new threads are not sent outside the forum. I'm thinking of a different value for the 'enabled' field, or via the use of the "undocumented" prefix.

Any volunteers to try to do this? If no takers, I might do it next weekend.

If you go the email route, just set it up to pull the messages from teh pop server and then send the messages to a "dump" address (where it'll be deleted and not delivered). Or just comment out the lines that sends out the email or newsgroup post.

[Gilby]

Originally posted by efoo
just installed this hack in order to archive some email lists and it works great - thanks. However, I've got one hurdle to overcome when it comes to automating the run of gateway.php. My board is off-limits to guest users, so trying to execute gateway.php without first logging into the board manually results in a "you are not logged in" page when the script tries to make all the posts. Is there any way to write a cron job that can both log into the forum and run gateway.php (from lynx)?

You might be able to create a dummy account and then just set the username and password as variables in teh gateway script so that whenever the gateway script is run it'll use that username and pass.

[Gilby]

Originally posted by Cluster One
I'm having a bit of trouble.

What i've done is I've installed hamster to pull messages from various newsgroups onto my computer, then setup the addon to download these headers from my pc to the website (Basically my ISP has IP masked access that only allows my IP here to login, but not my websites IP)

My problem is when I run the gateway script, it goes for a few minutes then asks me to download the file? Did i mess something up?

I mean it is actually obtaining the headers and putting them in the forums, its just annoying to get that window.

Sounds like you have your web browser set up incorrectly so that it won't display text files in teh browser window, but instead prompts you to save it.

[Gilby]

Originally posted by MaXxed
I have a potential issue. I have been running this hack on my forums for a few months now without problems, it has been working excellently.

Today I have found that my news provider has suspended my posting privelages. This may imply that I will need to change providers.

Should I simply change providers (and the settings on the hack) - I am concerned that I will have thousands of duplicate posts on the first sync.

Any idea how I could make the transition painlessly?

You need to get the message id number of the last post on your new server, which is unique to each news server and set it in the msgid column so that it'll pull starting from there.

[Gilby]

Originally posted by Gardener
Well, I can get it to work on my ISP's POP3 account, but not on the POP3 accounts on my server. I'm still trying to find out how to fix this.

What mail server do you have?

[Gilby]

Originally posted by qnx
is this hack dead?
I noticed the author (Gilby) hadn't said a word in the forum for more than three months.
Lots of questions/suggestions left unanswered.

I've been busy working on other projects. Sorry about putting this hack on the back burner.

[Gilby]

Originally posted by gopherhockey
1. Can you post a sticky thread and will that screw up the system?

2. Is there a way to post in my forum and not have it be sent out to the public.

A sticky thread will work normally (but of course wont be sticky on the news server). To post to the forums and not have it go to the newsgroup, you'll need to make the post and then change the value in the post table to make the gateway script think it was already sent by changing the "isusenetpost" column to 1 for that post.