vb.org Archive - NNTP Gateway (Usenet/Newsgroups)

Page 19 of 25

« First

Last »

Show 40 post(s) from this thread on one page

vb.org Archive (https://vborg.vbsupport.ru/index.php)

- vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)

- - NNTP Gateway (Usenet/Newsgroups) (https://vborg.vbsupport.ru/showthread.php?t=35247)

lierduh

02-04-2004 06:26 AM

The two sections were copied from the file I downloaded from page 44. So I know the new file contains the fix.

You apparently are using the older one, because you have to manually alter the code. There are two sections need to be changed.

When I said Proxy server, I was wondering if you were getting the old file instead of the new gateway_RC2.zip file. Your ISP could have a transparent proxy.

====
Edit, I just checked your site. Apparently the link for a new thread is working, but not the thread with inserted post. Because you changed the code around line 36?, but not the line 27?. :-)

allan grossman

02-04-2004 08:36 AM

Weirdness.

The reason I didn't change the code in 27? is that the code was identical to what you posted. I changed it anyway just now for grins, but if new threads are working (and it seems they are) then I can just wait for the problem to go away all by itself ;)

Thanks, lierduh!

allan grossman

02-05-2004 01:34 PM

Seems to be working just fine, lierduh - thank you very much.

One more thing I could use if you're still into developing this - can we get the gateway to honor an 'X-No-Archive: Yes' header?

thanks - I think after this I'll probably leave you alone for awhile.

cheers -

allan

lierduh

02-05-2004 05:19 PM

Quote:

Originally Posted by allan grossman

One more thing I could use if you're still into developing this - can we get the gateway to honor an 'X-No-Archive: Yes' header?

Allan,

Around line 748, find the code:
$response = $news->post(snip......)\r\nX-Originating-IP: $newthread[ipaddress]$ref");

change the ending to:

$response = $news->post(snip......)\r\nX-Originating-IP: $newthread[ipaddress]\r\nX-No-Archive: Yes$ref");

Edit: read your post again. You may be after something different. Do you want the forums to delete these posts after a while? hence does not archive them?

allan grossman

02-05-2004 07:02 PM

Quote:

Originally Posted by lierduh

I want the forum to not pick them up at all, lierduh. Is it possible?

allan grossman

02-05-2004 07:09 PM

Quote:

Originally Posted by allan grossman

I want the forum to not pick them up at all, lierduh. Is it possible?

To clarify a little - Google (and before them, Dejanews) won't grab posts where 'X-No-Archive: Yes' is in the header. I need for posts with this header to not make it into my forums.

thanks - you've been swell on this.

lierduh

02-06-2004 06:15 AM

Around line 153, find this line:

$thisheader=getHeaders(s..................

Add the following lines after that line:

PHP Code:


		
			
if (isset($thisheader['X-No-Archive'])) {

     if ($thisheader['X-No-Archive'] == 'Yes') {

          continue;

     }

}

I have not tested that. So back up your old file.:)

allan grossman

02-06-2004 08:26 AM

edit:

Didn't work, sorry :(

Would it be easier to parse the whole post (including header) for

X-No-Archive:<space>Yes

and just bail out of the whole import routine if the string exists? That'd get the damn thing, I'll bet ;)

lierduh

02-07-2004 09:14 PM

Try
if ( trim($thisheader['X-No-Archive']) == 'Yes') {
or

if ( stristr($thisheader['X-No-Archive'], 'yes') {

allan grossman

02-08-2004 10:28 AM

Actually, the first one appears to be almost working now - but if someone with X-No-Archive set replies to a post made through the gateway the post is getting through.

If you compare alt.guitar.bass to my gateway of the newsgroup at http://ebassist.com - there's a poster named "Lynrd" who has X-No-Archive set.

If you look at the past couple hundred posts in the NG I haven't pulled one of his posts - but he just replied to a thread I started and the gateway pulled it in.

The thread I started was called "The final solution - or, 'Hey, Lynrd!'" - and I've got no posts from him in the last two days - although there are four or five in the newsgroup. But - when he replied to the post I made through the gateway the gateway picked it up.

Any thoughts?

I'm gonna try the two other lines you gave me to try.

thanks ;-)

lierduh

02-09-2004 07:56 AM

Could you post the entire headers of the post he replied and one of his other posts.

Also for debugging purpose, add this line after that getHeader line

print_r($thisheader);

allan grossman

02-09-2004 08:26 AM

Okay -

this one showed up in the forum

Path: internal1.nntp.sjc.giganews.com!border2.nntp.sjc.g iganews.com!nntp.giganews.com!cyclone-sf.pbi.net!216.218.192.242!news.he.net!newsfeed1.e asynews.com!easynews.com!easynews!bigfeed2.bellsou th.net!bigfeed.bellsouth.net!bignumb.bellsouth.net !news.bellsouth.net!bignews3.bellsouth.net.POSTED! 50720933!not-for-mail
From: Lynrd <iamnot@this.com>
Newsgroups: alt.guitar.bass
Subject: Re: The final solution - or, 'Hey, Lynrd!'
Organization: Department of Redundancy Department
Message-ID: <402b4ddd.213014098@server1.loosenet4free.com>
References: <allan.11bapm@ebassist.nospam.com>
X-Newsreader: Forte Agent 1.93/32.576 English (American)
X-No-Archive: yes
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Lines: 37
X-Trace: bhmkggakljkaanefdbdpiflmbcekedmfhojhikkbagflhcbojj cppjnbmkeghkjnaebckapdlcgdmfgdjabdbdkfgappaidgiloo kconkabkfbidlnmbhlcnknekmbegpccnnhdpgmpjjbpkekfngo dfappodfma
NNTP-Posting-Date: Sun, 08 Feb 2004 09:52:40 EST
Date: Sun, 08 Feb 2004 08:57:33 -0600
Xref: intern1.nntp.aus1.giganews.com alt.guitar.bass:354721

but this one did not:

Path: internal1.nntp.sjc.giganews.com!border2.nntp.sjc.g iganews.com!nntp.giganews.com!cyclone-sf.pbi.net!216.218.192.242!news.he.net!newsfeed1.e asynews.com!easynews.com!easynews!bigfeed2.bellsou th.net!bigfeed.bellsouth.net!bignumb.bellsouth.net !news.bellsouth.net!bignews6.bellsouth.net.POSTED! not-for-mail
From: Lynrd <iamnot@this.com>
Newsgroups: alt.guitar.bass
Subject: Re: You?
Organization: Department of Redundancy Department
Message-ID: <40243584.75709184@server1.loosenet4free.com>
References: <imf720dok6ldvtjphvhli0u7sfoqjcvhi2@4ax.com>
X-Newsreader: Forte Agent 1.93/32.576 English (American)
X-No-Archive: yes
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Lines: 38
X-Trace: ofjmidbaofeaohdodbdpiflmbcekedmfhojhikkbagflhcboea gmlgmoifpnnejlkaikhjbiabbhpmpljabdbdkfgappaidgokda icblneaplmodlnmbhlcnknekmbegfnkjinielkkcgcglnjgdkc epeeifcdbb
NNTP-Posting-Date: Fri, 06 Feb 2004 19:49:04 EST
Date: Fri, 06 Feb 2004 18:52:20 -0600
Xref: intern1.nntp.aus1.giganews.com alt.guitar.bass:354456

The first of the two new lines didn't work i I'm gong back to the one that did in post #727

Also, I found the "get headers" line was missing from the script (one too many cut-and-pastes, I'm afraid) - so let's not do anything different until I've got solid information on what works now that I've got the script back where it needs to be.

Sorry to have wasted a little of your time on this, lierduh - I didn't see the line was missing. I'll let you know what happens after I run this for a couple days.

lierduh

02-09-2004 08:40 AM

You need to use the third option in this case (pun intended).

So insert these:

PHP Code:


		
			
if (isset($thisheader['X-No-Archive'])) { 

     if ( stristr($thisheader['X-No-Archive'], 'yes') ) {

          continue; 

     } 

}

allan grossman

02-09-2004 10:23 AM

Quote:

Originally Posted by lierduh

You need to use the third option in this case (pun intended).

So insert these:

PHP Code:


		
			
if (isset($thisheader['X-No-Archive'])) { 

     if ( stristr($thisheader['X-No-Archive'], 'yes') ) {

          continue; 

     } 

}

I think we're almost there.

Thanks, lierduh.

allan grossman

02-09-2004 10:20 PM

If I wanted to check for upper- and lowercase would I do it like this?

PHP Code:


		
			
if (isset($thisheader['X-No-Archive'])) { 

     if ( stristr($thisheader['X-No-Archive'], 'Yes' || 'yes') ) { 

          continue; 

     } 

}

lierduh

02-09-2004 10:29 PM

Quote:

Originally Posted by allan grossman

If I wanted to check for upper- and lowercase would I do it like this?

PHP Code:


		
			
if (isset($thisheader['X-No-Archive'])) { 

     if ( stristr($thisheader['X-No-Archive'], 'Yes' || 'yes') ) { 

          continue; 

     } 

}

stristr is case INsensitive.:)

allan grossman

02-09-2004 10:32 PM

Ah. Then I don't need to do anything.

Let me run it for a couple days and I'll let you know what happened.

Thanks a million, lierduh - I'm hoisting a glass in your direction right now :)

Bro_Joey_Gowdy

02-12-2004 03:25 PM

Im now getting another error in line 37 of gateway.php
http://www.joeygowdy.com/forum/gateway.php

allan grossman

02-12-2004 03:39 PM

Which version of the gateway - the vB2.x version on page 1 or the one lierduh modified for vB3 on page 44?

The reason I ask is that in lierduh's version line 37 is commented out. Can you paste line 37 into a message here for us?

thanks -

fluxforums

02-20-2004 01:03 AM

I have installed this hack along with the VB3 RC2 gateway upgrade along with ALTER TABLE post ADD ref TEXT NOT NULL;

My nntp_groups reflect....

biz.marketplace.web-design varchar(200) No
General Web Design varchar(32) No 0
prefix varchar(255) No
lastmsg int(11) No 0
news.comcast.giganews.com varchar(64) Yes NULL
xxxxxx varchar(32) Yes NULL
yyyyyy varchar(32) Yes NULL
enabled int(11) No 1

When running....
http://etcetc/gateway.php I receive...

0 group(s) gatewayed.

Am I missing something rudimentary here?

fluxforums

02-22-2004 05:14 AM

I have resolved that error by correcting the forum variable with an ID as read in the readme.

Now I am receiving...

1 group(s) gatewayed.
Logging in to news.comcast.giganews.com, group xxx.xxxx.xxx
Posting Message from FluxForums: 500 syntax error or unknown command
Posting Message from FluxForums: 480 authentication required
Posting Message from FluxForums: 500 syntax error or unknown command

And neither threads are exported (which I dont mind) to the newsgroup or threads imported (which of course I would like). This occurs in other newgroups within giganews.com server as well.

BTW: These are direct copy and paste names that I have verified to work in my Outlook Express so I am not sure why the authentication error is comming up.

I am running VB3 RC3

*NOTE: I believe I found the answer to this on Page 12 of this thread. It looks like giga does not allow access outside its networks. I will signup for another usenet provider and try it.

Quote:

Originally Posted by fluxforums

fluxforums

02-22-2004 06:49 AM

Bingo!

Now if only I can figure out why the thread and post count has updated but I do not see the actual threads in the forum.

Quote:

Originally Posted by fluxforums

fluxforums

02-22-2004 07:13 AM

Changed the forums viewing age. Good to go.

Is there a way to prevent threads from being exported into a newsgroup?

Quote:

Originally Posted by fluxforums

Bingo!

Now if only I can figure out why the thread and post count has updated but I do not see the actual threads in the forum.

Also, is there a way to prevent threads from being imported into a newsgroup?

Cluster One

02-23-2004 07:35 AM

Is there another way to run the gateway.php script? I've ran it on my webhost and it basically kills the server. Sucks up all the RAM and CPU usage.

allan grossman

02-23-2004 08:28 AM

Quote:

Originally Posted by fluxforums

Changed the forums viewing age. Good to go.

Is there a way to prevent threads from being exported into a newsgroup?

Nope. I'm afraid there isn't.

allan grossman

02-23-2004 08:32 AM

Quote:

Originally Posted by Cluster One

Is there another way to run the gateway.php script? I've ran it on my webhost and it basically kills the server. Sucks up all the RAM and CPU usage.

It shouldn't be too bad once the newsgroups are synchronized. Have you got all the messages in the group or are you still trying to fill up the forum with older posts?

I run my gateway against one group that gets maybe 200 posts a day - my cron job runs every ten minutes. It doesn't work my server all that hard but of course, YMMV.

Cluster One

02-23-2004 06:54 PM

I set the last message to only get the last 30 days worth of posts from the groups, doubt anyone is going to read a message that was posted on 6/2/2003 anyways.

I installed the script, I run in it mozilla and all it says is this:

2 group(s) gatewayed.
Logging in to news.usenetserver.com, group alt.tv.angel
Logging in to news.usenetserver.com, group alt.tv.buffy-v-slayer

Nothing happens after that, it just sits there. I've even left it like that for over an hour.

allan grossman

02-23-2004 08:05 PM

Look on page 24, Cluster One - there's a version of nntp.php that does verbose output - maybe you can use that to determine where the problem is. It's in post #351.

Good luck!

Cluster One

02-23-2004 09:10 PM

group(s) gatewayed.
200 Powered by Usenet Server -- fe13 -- http://www.usenetserver.com/ (Tornado v1.0.5.353)
381 More Authentication Required
480 Authentication Required
Logging in to news.usenetserver.com, group alt.tv.angel
200 Powered by Usenet Server -- fe13 -- http://www.usenetserver.com/ (Tornado v1.0.5.353)
381 More Authentication Required
480 Authentication Required
Logging in to news.usenetserver.com, group alt.tv.buffy-v-slayer

Thanks for the help, but still it says 480 Authentication Required, but I have my login/pass setup correctly in the SQL database? Any other ideas?

I attached an image of my configuration.

allan grossman

02-24-2004 01:34 AM

Quote:

Originally Posted by Cluster One

Maybe - it sounds to me like the news server isn't getting your password. Check this out -

http://www.mibsoftware.com/userkt/nntpext/0032.htm

Code:

    AUTHINFO USER username

    AUTHINFO PASS password



    The original AUTHINFO is used to identify a specific entity

    to the server using a simple username/password combination.

    It first appeared in the UNIX reference implementation.



    When authorization is required, the server will send a 480

    response requesting authorization from the client. The

    client must enter AUTHINFO USER followed by the username.

    Once sent, the server will cache the username and may send

    a 381 response requesting the password associated with that

    username. Should the server request a password using the 381

    respose, the client must enter AUTHINFO PASS followed by

    a password and the server will then check the authentication

    database to see if the username/password combination is valid.

    If the combination is valid or if no password is required,

    the server will return a 281 response. The client should then

    retry the original command to which the server responded with

    the 480 response. The command should then be processed by

    the server normally. If the combination is not valid, the server

    will return a 502 response.



    Clients must provide authentication when requested by the server.

    It is possible that some implementations will accept authentication

    information at the beginning of a session, but this was not the

    original intent of the specification. If a client attempts to

    reauthenticate, the server may return 482 response indicating

    that the new authentication data is rejected by the server.

    The 482 code will also be returned when the AUTHINFO commands

    are not entered in the correct sequence (like two AUTHINFO

    USERs in a row, or AUTHINFO PASS preceding AUTHINFO USER).



    All information is passed in cleartext.



    When authentication succeeds, the server will create an email

    address for the client from the user name supplied in the

    AUTHINFO USER command and the hostname generated by a reverse

    lookup on the IP address of the client. If the reverse lookup

    fails, the IP address, represented in dotted-quad format, will

    be used. Once authenticated, the server shall generate a Sender:

    line using the email address provided by authentication if it

    does not match the client-supplied From: line. Additionally,

    the server should log the  event, including the email address

    This will provide a means by which subsequent statistics generation

    can associate newsgroup references with unique entities - not

    necessarily by name.



3.1.1.1 Responses



        281 Authentication accepted

        381 More authentication information required

        480 Authentication required

        482 Authentication rejected

        502 No permission

Hope that helps a little -

Cluster One

02-24-2004 02:07 AM

Well it seems to be getting my login and password, its just not pulling anything from the server.

allan grossman

02-24-2004 09:31 AM

Quote:

Originally Posted by Cluster One

Well it seems to be getting my login and password, its just not pulling anything from the server.

If it was getting your password you wouldn't be getting the 381 error ;)

Cluster One

02-24-2004 09:56 PM

Doh, you are correct sir, thank you. Well I have tried pretty much everything and I am about ready to toss in the towel. I've re-installed it twice. It just shows the same screen.

Could it be something in the nntp.sql that didn't go in correctly? or need to be changed?

Cluster One

02-25-2004 09:23 PM

Okay, i've added my login/pass into the SQL. It shows up there, but somehow the server is just not recieving it.

Its mind boggling.

lierduh

03-01-2004 08:23 PM

I have uploaded a newer gateway.php to page 44 to handle multiple attachments.

https://vborg.vbsupport.ru/showpost....&postcount=658

recumbent615

03-07-2004 01:41 AM

I have been using this hack for months now, and Love it. I have only used the Email ( POP3) part of the software but resently have been attempting to suck down about 30-40K messages from a news group. I have run into several issues with this, first I time out. This is a manageble issue, I just take the last message # and plug it into the last message # field under edit, and start the process again. The larger issue is the post date. I would have expected that the post date would use the date from the news group. and it looked like it did for the first 2k ( the first pass that timed out ) but not for any of the follow on grabs. Is there something I can do to correct this ? or better yet is there someplace that I can stop the timeout. I've upped the run time in php.ini to 14400 ( 4 hours ) and it still says that there was a 60 second time out, that is just not right as it does run for 15-20 minutes before it times out.

Any Ideas

Kevin B

recumbent615

03-07-2004 05:35 PM

OK I have not found a solution for this issue, but I believe that the failures are the reason that the date is off. Is there a way to limit the Number of new messages that the program will download in a single session ? It seems like I get around 2000 to 2200 messages each time before the timeout, What I'd like to do is only download 1000 messages and then diconnect, I believe that this will take care of the date issue.

Kevin

lierduh

03-07-2004 07:25 PM

The gateway downloads where the last download left by the message number. You can find the message number for your server in the header. eg.

Xref: freenews.iinet.net.au aus.photo:73361

The 73361 is the message number. Set this number in your database (lastmsg), it will only download the messages higher than this message number.

recumbent615

03-07-2004 09:00 PM

OK, Here is the solution for regulating the # of messages to download on each pass.

in gateway.php find

PHP Code:


		
			
    // retrieve each message and process

    if ($grouptype == 'news'){

        $lastmsg = ($group[lastmsg] >= $news->min()) ? $group[lastmsg] : $news->min();

        $max = $news->max();

    } else {

        $lastmsg = 0;

        $max = $Count;

    }

    for ($current = $lastmsg+1; $current <= $max; $current++){

and replace it with this

PHP Code:


		
			
    // retrieve each message and process

    if ($grouptype == 'news'){

        $lastmsg = ($group[lastmsg] >= $news->min()) ? $group[lastmsg] : $news->min();

        $max = $news->max();

    } else {

        $lastmsg = 0;

        $max = $Count;

    }

    // Total to grab at one time

    $grab = 500;

     

    if (( $max - $lastmsg ) > $grab ) {

        $max = $lastmsg + $grab ;

        echo "Downloading messages from ".$lastmsg." to ".$max."<br>";

    }    

    for ($current = $lastmsg+1; $current <= $max; $current++){

To deal with the Date Stamp issue I have made one change that I plan on removing when I've uploaded all of the back posts from the news groups.

PHP Code:


		
			
            if ($group[lastmsg] == 0){

                $date = strtotime($message[date]);

            } else {

                $date = strtotime($message[date]);

                // $date = time();

            }

stroke25

03-13-2004 06:55 PM

Im trying to change the email settings used with usenet posts.Right now it adds numbers and letters in between user name and Domain name..Example

KGB.11h4mb@pokerhelper.com KGB.11g8ob@pokerhelper.com StopTheKicking.127uvy@pokerhelper.com

I would like it to be username@pokerhelper.com

All times are GMT. The time now is 05:08 PM.

Page 19 of 25

« First

Last »

Show 40 post(s) from this thread on one page

X vBulletin 3.8.12 by vBS Debug Information
Page Generation 0.02422 seconds Memory Usage 1,911KB Queries Executed 10 (?)
More Information
Template Usage: (1)ad_footer_end (1)ad_footer_start (1)ad_header_end (1)ad_header_logo (1)ad_navbar_below (1)bbcode_code_printable (8)bbcode_php_printable (12)bbcode_quote_printable (1)footer (1)gobutton (1)header (1)headinclude (6)option (1)pagenav (1)pagenav_curpage (4)pagenav_pagelink (1)pagenav_pagelinkrel (1)post_thanks_navbar_search (1)printthread (40)printthreadbit (1)spacer_close (1)spacer_open Phrase Groups Available: global postbit showthread	Included Files: ./printthread.php ./global.php ./includes/init.php ./includes/class_core.php ./includes/config.php ./includes/functions.php ./includes/class_hook.php ./includes/modsystem_functions.php ./includes/class_bbcode_alt.php ./includes/class_bbcode.php ./includes/functions_bigthree.php Hooks Called: init_startup init_startup_session_setup_start init_startup_session_setup_complete cache_permissions fetch_threadinfo_query fetch_threadinfo fetch_foruminfo style_fetch cache_templates global_start parse_templates global_setup_complete printthread_start pagenav_page pagenav_complete bbcode_fetch_tags bbcode_create bbcode_parse_start bbcode_parse_complete_precache bbcode_parse_complete printthread_post printthread_complete
Messages: