vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)
-   -   NNTP Gateway (Usenet/Newsgroups) (https://vborg.vbsupport.ru/showthread.php?t=35247)

Gilby 04-19-2002 12:07 AM

Quote:

Originally posted by CJi
No matter *what* I do, all I can get out of gateway.php is this:

1 group(s) gatewayed.
Logging in to news.usenet.com, group alt.sports.airsoft

nothing else.

Try the attached nntp.php file in place of the one you are using. this will output the responses from the nntp server.

Gilby 04-19-2002 12:14 AM

Quote:

Originally posted by Dontom
Gilby... Thanks for the great hack!!

I would like to exclude Usenetposts which contain a x-archive = no in their header - has anyone already done this? or is there a hint at which part of the script i can sort those posts out?
Thank you!
Tom

Look at the statement that checks if it is a forum post that is being read from teh newsgroup (if ($message[useragent] != $settings[useragent]). Add an exclusion for that header too. You may need to modify something else so that the header shows up in the message array.

Gilby 04-19-2002 12:17 AM

Quote:

Originally posted by akaler

Any idea how to filter out the bad seeds? The hacky idea I have above would work if only imported posts, not just threads, could somehow be attached to a username. I know vb.org accomplished this somehow. They grabbed the hacks forums from vbulletin.com, then when people registered here, it would attach their user name to the name they used on their old messages, and whala... full vb user based functions could be applied to old messages.

A simple SQL query can be used periodically to delete them. Use something like "DELETE FROM post WHERE username='name of bad seed'"

You'll also need to run DELETE FROM thread WHERE postusername='name of bad seed'"

Or you can see my last post and add another thing to the if statement.

CJi 04-19-2002 01:11 PM

ok, thanks for that Gilby. I've got a bit more information. It logs in ok, but then does nothing? I tried it with uk.test last night and got some action, but there are posts in alt.sport.airsoft (not many granted), but still some. I'd have thought it would at least import those few? Here's what I got:

1 group(s) gatewayed.
200 Welcome to Usenet.com, 5.5g6, S8
381 PASS required
281 Ok
211 6 621 626 alt.sport.airsoft selected
Logging in to news.usenet.com, group alt.sport.airsoft

akaler 04-19-2002 04:56 PM

Quote:

Originally posted by Gilby
A simple SQL query can be used periodically to delete them. Use something like "DELETE FROM post WHERE username='name of bad seed'"

You'll also need to run DELETE FROM thread WHERE postusername='name of bad seed'"

Or you can see my last post and add another thing to the if statement.

I think that vB deletes threads by username, even usersnames used in usenet posts. But the SQL query you gave me to delete posts by username... wow, that's useful!

So is the filter heading instructions. Great info, thanks!!

BTW... I've found two possible issues...

The first is posting to Supernews. I couldn't get it to post WITHOUT removing "Message-ID: <$msgid>\r\n" from the $response string. Supernews always ACCEPTED the messages just fine (240), but they just weren't showing up. I guess that either they have a beef with the way you generate the msgid OR they prefer to generate their own, and their server gets "insulted" by the fact you're offering your own. I don't know.

This didn't happen using Giganews, incidentally. Everything posted there just fine w/o modification. I only moved to Supernews a couple days ago, and still have my Giga account 'till the end of the month in fact. I believe it always worked well with Easynews (they have a $1 trial I checked out).

Side note to others reading -- Those services I just mentioned are probably the "big 3" of services offering quality feeds with high rentention, in case anybody is interested. If you want something cheaper but still good ($5mo), check out airnews.net. SUPER cheap (free) but still quality is readfreenews.net.

Reading access is free and needs no password. To get posting access to readfreenews, net, you have to email drechsau+usenet@yuck.net (Mike Horwath), with real name, real email isp email address, username wanted, password wanted.


Alright... next possible bug...

Sometimes when pulling groups that have an a large number of messages.. generally 25-80 thousand... the httpd process (with php as a module) begins swelling in memory access VERY VERY rapidly. Say 2mb growth every few seconds. After 1000 messages, I might have a 400mb process! It will then proceed to fill up the whole of available memory AND swap. But it'll still work. After the first time this happened, I've been killing these processes gone wild, though.

Sometimes this doesn't happen. Sometimes I can tell it to grab 40k messages or whatever... and see perfectly ordinary memory usage... a growth of just a few k every few seconds. Normally, it ends up being something like a 130mb or so after tends of thousands of message havebeen gotten, not just a few hundred.

This tends to happen more AFTER a reboot or a successul restart of Apache. What this seems to indicate to me is that some sort of memory routine isn't being initialized properly... but after php and apache have been used for awhile... everything that could be initialized in apache has been... That's just my theory, I could be wrong.

No matter what it is, though, this might be more of a php issue (i'm using the latest non-beta self-compiled version up on the php.net site... 4.1.2) than a problem with your script. Since it's memory related, I even thought it might be kernel related, but I just updated quite a few revisions and it still happens. I even recompiled php with different options. Same problem. Guess I'll have to wait for the stable release of 4.2 to see if it still happens.

Alrighty... thanks a bunch for your help and continued attention, Gilby! This is a great hack. You should really be compensated for this. Consider having some sort of donation system that parallels vbportals. I say vb portals, because the combination of everything they do to get donations is very successful. Offering betas only if you've donated and the other things they do really does seem to get them more donations than normal.

CJi 04-21-2002 09:40 AM

Is it at all possible that, when a local (vB) reply to a newsgroup is deleted, it gets cancelled off the newsgroup as well?

Just a thought :)

The Realist 04-21-2002 01:33 PM

Installed ok but Im getting the following error:

<br>
<b>Warning</b>: Failed opening 'class.POP3.php3' for inclusion (include_path='.:/php/includes:/usr/share/php') in <b>/home/virtual/site1/fst/var/www/html/forum/gateway.php</b> on line <b>25</b><br>
<br>
<b>Warning</b>: Cannot add header information - headers already sent by (output started at /home/virtual/site1/fst/var/www/html/forum/gateway.php:25) in <b>/home/virtual/site1/fst/var/www/html/forum/gateway.php</b> on line <b>483</b><br>
0 group(s) gatewayed.

Any help?

Brian

The Realist 04-21-2002 01:39 PM

Uploaded class.POP3.php3 to the forums dir and now no errors. But all Im getting is this:

0 group(s) gatewayed.

and no posts from newsgroup in the section?

Gilby 04-23-2002 02:15 AM

Quote:

Originally posted by akaler
The first is posting to Supernews. I couldn't get it to post WITHOUT removing "Message-ID: <$msgid>\r\n" from the $response string. Supernews always ACCEPTED the messages just fine (240), but they just weren't showing up. I guess that either they have a beef with the way you generate the msgid OR they prefer to generate their own, and their server gets "insulted" by the fact you're offering your own. I don't know.

Most news clients generate their own message ids so I'd guess that there is something with the format that the message id is in. I'm not aware on if there is a standard format for the message id, just that it needs to be unique, but I'll look into it to see if there is one that I was not aware of. It could also be that the supernews server doesn't accept a message id that has the host part of it different from where it's being sent from, so you may want to look at the posts that are made to it and see what the message id looks like. Then use that host for the setting in the nntp gateway.
Quote:

Originally posted by akaler
Sometimes when pulling groups that have an a large number of messages.. generally 25-80 thousand... the httpd process (with php as a module) begins swelling in memory access VERY VERY rapidly.

Hmmm... I hope it's a problem with something other than my code. ;) However, I'll take a look at this when I get a chance and maybe try out some large imports to see if I can duplicate this.

Meanwhile, you can set a limit of how many posts you want per pull per newsgroup by putting the following code before the line that contains (for ($current = $lastmsg+1; $current <= $max; $current++)), about line 90:
Code:

        if ($max - $current > 1000) {
                $max = $current + 1000;
        }

If it's a problem with my code, then php will free the memory when it's done executing the script (it may also do this is the problem is elsewhere).

Quote:

Originally posted by akaler

Alrighty... thanks a bunch for your help and continued attention, Gilby! This is a great hack. You should really be compensated for this. Consider having some sort of donation system that parallels vbportals. I say vb portals, because the combination of everything they do to get donations is very successful. Offering betas only if you've donated and the other things they do really does seem to get them more donations than normal.

I've been thinking of possibly calling it "donationware" which simply means if it's useful for you and you can afford to contribute some cash, then please donate. So, if you want to donate, you can use this PayPal link: https://www.paypal.com/xclick/busine...r=nntp-gateway

Gilby 04-23-2002 02:19 AM

Quote:

Originally posted by The Realist
Uploaded class.POP3.php3 to the forums dir and now no errors. But all Im getting is this:

0 group(s) gatewayed.

and no posts from newsgroup in the section?

Did you set up a newsgroup to import? In the nntp_groups table, you need to add a row that has the server, newsgroup, username, password, etc for the newsgroup you want to gateway in. If you don't know how to add a row in the mysql table, you may want to install the GUI that was made for this hack: https://vborg.vbsupport.ru/showthrea...threadid=36475


All times are GMT. The time now is 03:05 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.01860 seconds
  • Memory Usage 1,779KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (1)bbcode_code_printable
  • (8)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (3)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (10)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete