vb.org Archive

vb.org Archive (https://vborg.vbsupport.ru/index.php)
-   vBulletin 2.x Full Releases (https://vborg.vbsupport.ru/forumdisplay.php?f=4)
-   -   NNTP Gateway (Usenet/Newsgroups) (https://vborg.vbsupport.ru/showthread.php?t=35247)

allan grossman 09-22-2003 09:45 PM

woohoo!

Good on you, bitg. I'm glad you got it fixed.

PhoenixBB 09-22-2003 11:08 PM

Thank you Allan, those instructions made much more sense. :D

Quote:

From a terminal window type "man diff" (no quotes) and smack the enter key. Use the spacebar to scroll down and the "q" key to quit. Lotsa good stuff there. "diff --help" will give you the short version.
Mwuahahaha I've no idea what a terminal window is. I tried the MSDos prompt out of curiosity, that looks like a 'terminal' thing, but of course I got 'bad command'.

So this is all from Linux eh? Might as well be Chinese for some of us. ;) Think I better read that link you gave if I plan on doing any more hacks. What's the betting I end up with a blank look on my face. <snerk>

The gateway appears to be working ok, but my main problem now is making sure it strips the vbcode properly on the outgoing.

Always seems to leave some of the color codes in which look weird in the newsgroup.

Thanx for your help. Couldn't have figured this without ya. ;)

Phoe

PhoenixBB 09-22-2003 11:18 PM

Allan, you've commented out the mime.php instead of the class.pop3.php. I assume this is an accident and the mime isn't supposed to be commented out?

Thanx!

allan grossman 09-22-2003 11:33 PM

Quote:

Originally Posted by PhoenixBB
Allan, you've commented out the mime.php instead of the class.pop3.php. I assume this is an accident and the mime isn't supposed to be commented out?

Thanx!

Oops.

I fixed it.

Thanks!

allan grossman 09-22-2003 11:39 PM

Quote:

Originally Posted by PhoenixBB
Thank you Allan, those instructions made much more sense. :D

Mwuahahaha I've no idea what a terminal window is. I tried the MSDos prompt out of curiosity, that looks like a 'terminal' thing, but of course I got 'bad command'.

So this is all from Linux eh? Might as well be Chinese for some of us. ;) Think I better read that link you gave if I plan on doing any more hacks. What's the betting I end up with a blank look on my face. <snerk>

The gateway appears to be working ok, but my main problem now is making sure it strips the vbcode properly on the outgoing.

Always seems to leave some of the color codes in which look weird in the newsgroup.

Thanx for your help. Couldn't have figured this without ya. ;)

I didn't know you were running Windows, Phoenix - the "man diff" probably didn't make any sense at all :)

Glad to be of service, though.

Holidazed 09-23-2003 12:28 AM

Allan, I am now getting the following error when synchronizing (only if a post is due to go out from my system).
________________
1 group(s) gatewayed.
200 NewsRave v1.01 (posting ok)
381 More authentication needed
281 Ok
211 17410 962764 980173 rec.games.frp.dnd Logging in to news-west.newscene.com, group rec.games.frp.dnd
340 340 <3f6fa158$0$99022$45beb828@newscene.com> (desired)
240 Article posted!Posting Message from DK:
240 Article posted!
220 980173 <3F6FCAE7.127B@cox.net> articleGetting message number 980173: New thread; 'Value of captured spellbooks?' from Werebat
________________

Obviously some of that is messages received. Where is says "381 More Authentication needed" is confusing me. My outbound message is being posted and it recieves messages.

What is it authenticating?

allan grossman 09-23-2003 01:00 AM

I looked up the error number -

381 is a password request from the server. Everything's working fine - what you're seeing is normal authentication.

Holidazed 09-23-2003 05:29 PM

OK, I was just confused. Thank you for your help.

limey 09-23-2003 07:04 PM

A nice addon would be to add an email obfuscator to the script so that emails that get printed via yoursite are not tried on on your email server by spam bots.

should be easy to impliment.

allan grossman 09-23-2003 07:45 PM

Quote:

Originally Posted by limey
A nice addon would be to add an email obfuscator to the script so that emails that get printed via yoursite are not tried on on your email server by spam bots...

Hmmmm.

I may play with that a little. My mail server's getting hammered (well, not really but spam traffic is up) and I'd rather change the email domain to something that can't be resolved.

wolfman 10-07-2003 04:55 PM

Thanks for this hack

irubin 10-09-2003 10:09 PM

Hello,

I have been recieving the following error when gateway.php attempts to post articles:

Posting Message from user: 441 (607) Article Rejected -- The 'References:' field requires one or more values

Any ideas?

Roy

wolfman 10-10-2003 05:54 PM

I have this installed and working great, once I got all the posts downloaded.

I just need to know if there is a way to keep these posts from showing up when a user clicks on the View new post link.

They like having the access to the groups but want to be able to use it more as an additional resource to search.

liquidblaze 10-12-2003 03:47 PM

i installed the hack and apparently all is working fine, i just ran gateway.php for the first time (still runnning - since 3 mins now) and i was asking myself how high the server load is if i run this script and for the crontab, what would be a reasonable time frame? i guess 10minutes is way too much

allan grossman 10-23-2003 02:14 PM

Quote:

Originally Posted by wolfman
I have this installed and working great, once I got all the posts downloaded.

I just need to know if there is a way to keep these posts from showing up when a user clicks on the View new post link.

They like having the access to the groups but want to be able to use it more as an additional resource to search.

There is, wolfman :)

I made a search button that excludes Usenet posts. Pretty easy, really - on my system I made a button that linked to

../forumdir/search.php?s=&action=getlatest&forumid=4,5,8,9,12, 14,19,22,23

Substitute my forum numbers for the ones you want to search.

cheers -

liquidblaze 10-23-2003 02:27 PM

Quote:

Originally Posted by allan grossman
There is, wolfman :)

I made a search button that excludes Usenet posts. Pretty easy, really - on my system I made a button that linked to the search

hmm didnt even think about this but im gonna integrate it :)

allan grossman 10-23-2003 03:47 PM

Quote:

Originally Posted by liquidblaze
i installed the hack and apparently all is working fine, i just ran gateway.php for the first time (still runnning - since 3 mins now) and i was asking myself how high the server load is if i run this script and for the crontab, what would be a reasonable time frame? i guess 10minutes is way too much

Once you get the newsgroups populated server load is negligible. I gateway three relatively low-traffic groups every 15 minutes (~100 posts a day) and the script takes less than ten seconds to execute.

lierduh 11-02-2003 03:08 AM

I initially did a bit of hacking to this srcript regarding threading problem. You can see the post here:

https://vborg.vbsupport.ru/showpost....&postcount=587

allan grossman did a great follow up about this at:

https://vborg.vbsupport.ru/showpost....&postcount=635

After moving to vB V3 plateform, I have found the gateway is not compatible with V3. Below is my note to get the script to work with V3. I attached the hacked gateway.php with this post, there are quite a few places I need to fix, so a "diff" may not be a good way to see.

At the moment, I can only confirm my site is working with the Threading References problem fixed. A forum member can now also disable the signature when post to the gateway by uncheck "Show Signature". I must also point out: this is a quick hack to get my site up and running, so it may not be optimused nor it can be considered even near stable. You use it at your own risk. Some of the things I can think of after using this include: completely destroy your data, cause nasty replies from normal USENET users, the server may blow up...:)

===============================
For installation, please follow the Readme file contained in the zip file.
===============================

Edit: Fixed extra long References limitation.
Edit: Dec 4, fixed a typo in SQL string regarding attachment, possibly inherited from the original gateway.php. Please note, I have not been bothered to make the attachment fully working. Uploading fixed gateway.php
Edit: Dec 19 2003, uploading a new gateway.php which works with v3 Gamma. Modified the way of how to separate the new local posts and the new usenet posts.
Edit: Jan 21 2004, confirm my Gamma version of gateway.php also works for RC2.
Edit: Jan 23 2004, removed the older attachment gateway.php which was only good for vB3 Beta 7.

Please note, my current patched gateway.php does not work with "tableprefix". ie. if you defined tableprefix within config.php.

Rewrote the install instruction to save a lot of confusion.

Edit: Feb 2 2004

*Made "Xxxx Wrote:" working when reply/quote to a post.
*Added the ability to strip [font=xxx] BB tags.

Edit: Feb 4 2004

*Fixed the wrong thread link problem for the "goto newpost" at the forum list
*Uploaded the fixed gateway_RC2.zip

Edit: Feb 12 2004

*Added ability to strip nested BB tags.
*Uploaded a newer gateway_RC2.zip file.

Edit: Mar 2 2004

*Added ability to handle multiple attachments as in vb3.
*Uploaded gateway_Mar2.zip

(I have not tested this, but you should use this version if you want the attachment link(s) to work.)

Now confirmed the attachment is working. Removing older gateway.php

Edit: Mar 24 2004

*Discovered search index no longer worked with vB3 Gold due to a changed function name
*Uploading a fixed gateway.php (filename gateway_Mar24.zip)

Edit: May 12 2004

* I have done some major changes to the scripts. Improvement include:

== TABLE_PREFIX is now supported.
== Messages imported from usenet are now inserted into threads based
on Reference in the header instead of the subject.
== Built mechanism to avoid two instances of the gateway.php running
at the same time. This can easily happen if someone clicks the link in
the Who Is On Line page. This should also solve the problem of double
up posts and wrong post counters.
== Massive speed improvement!
== Cleaned up the codes.
== Implemented parentid for imported posts
== lastmessage is updated now after each message retrieval. This way
no message will be imported twice, even if the script is crashed at
some stage.
== Fixed a bug which occurs in long header lines. Message may not be
retrieved due to this bug.

This time instead of a quick hack, I have gone through almost each line. So it should be a much better script.

For people who has been using my previous gateway.php. It is strongly recommended to upgrade to this latest release. Please add the following records to your table:

Code:

INSERT INTO `nntp_settings` VALUES (NULL, 'Is Gateway Running', 'is_running', '0', 'Check if another instance of gateway.php is running', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Last Time Run', 'last_run', '1084316100', 'Last time the gateway.php was run', NULL, 0, 0);

Note for vB3. The thread view URL works with only "t=" instead as well as "threadid=". So you might like to change the 'threadurl' in the nntp_settings table. Check the attachment URL setting too. The format should be:

'http://forums.yourdomain.com.au/attachment.php?attachmentid='

Please also replace the nntp.php and mime.php with ones within the new zip file. I have made changes to nntp.php and mime.php files to fix bugs and make improvement.

(For people who downloaded the file earlier. Please download again. I made a stupid mistake in the gateway.php script.) The new file name is vb3_NNTP_Gateway_May12a.zip note the "a" in the file name)

Edit May 14 2004

**Build X-No-Archive header honouring system
**Improved speed
**Bug fixes
**Verbose output when run manually. Easier to find problem when interating with NNTP server.

change the honor_no-archive record from 0 to 1 to stop importing messages with X-No-Archive header, just as Google Group does.

For upgrade from the May12 release. Please replace all php files and insert two records into nntp_settings table
Code:

INSERT INTO `nntp_settings` VALUES (NULL, 'Honour No-Archive', 'honor_no-archive', '0', 'Do not import the message if the message has the X-No-Archive header set to yes. Google Group honours this Non-standard header.', NULL, 0, 0);
INSERT INTO `nntp_settings` VALUES (NULL, 'Last Max postid', 'last_postid', '0', 'The max postid in post table when the script was last run', NULL, 0, 0);

Edit: May 15 2004

* Fixed typos in nntp.sql and gateway.php.
* Modified setlastmsg.php to work with current versions.
* Packaged setlastmsg.php into the zip file.
* Reworded readme file regarding lastmsg.

Edit: May 16 2004

I have released the gateway under vB3 section of the forums. Please use the new thread for support. Latest files will always be posted to the new thread.

https://vborg.vbsupport.ru/showthread.php?t=65152

rishel 11-19-2003 06:36 PM

Ok. Here is a problem when trying to use Giganews. Giganews boasts it has 50,000 supposed threads in a particular NG.

However, when we run the gateway to start importing ng threads, some are no longer available.

This causes a timeout on our server. So I downloaded a normal newsgroup reader: Agent and sure enough, some posts were missing from a year ago, and some were not.

Does anyone have a fix or a workaround for this. Our PHP.ini file has been modified for testing purposes to allow an unlimited amount of time, but this still does not work.

Any ideas?

allan grossman 11-19-2003 07:19 PM

I asked this in email but I'll ask it again here :)

Why do you want year-old newsgroup posts? With Usenet usually if you get the last month's posts that's more than anybody will ever read.

Plus - although you can prevent folks from searching the Usenet forums if you want they still get added to the database when you rebuild your search index. That's a heck of a lot of space for something nobody's gonna read ;)

rishel 11-20-2003 04:07 PM

Because of the content, the content in our ng group doesnt pass away, its always needed. Its automotive based, and there are a wide array of years that people want to search for.

Its a shame really, I was really hoping to bring in a lot more.

allan grossman 11-20-2003 04:20 PM

Ah. I gotcha now.

Did you have any luck trying trying to download fewer headers at once?

liquidblaze 11-27-2003 08:25 AM

I just wanted to sync the NG with my board for the first time on the new host and got this:

Quote:

Invalid SQL: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.title,post.threadid,threa d.forumid,thread.msgid,thread.prefix FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND (thread.forumid=25)
mysql error:

mysql error number: 0
any ideas what I could do?

liquidblaze 11-27-2003 09:15 PM

I found the problem. had to re-run
ALTER TABLE thread ADD prefix varchar (24);
to get the sync working again - it was a problem with an importing script i had. thanks for this great hack man :)

KevinM 11-29-2003 01:05 PM

I am attempting to put this into a clean vb3 board.

I am using the gateway script adapted by lierduh, with the rest being from Gilby's original.

I get the following error when running gateway.php for the first time:

Quote:

Database error in vBulletin 3.0.0 Beta 7:

Invalid SQL: SELECT post.postid,post.username,post.userid,post.datelin e,post.attachmentid,post.pagetext,post.showsignatu re,post.ipaddress,thread.pollid,thread.title,post. threadid,thread.forumid FROM thread,post WHERE post.isusenetpost=0 AND thread.threadid=post.threadid AND post.dateline=thread.dateline AND post.userid=thread.postuserid AND (thread.forumid=2) mysql error: Unknown column 'post.attachmentid' in 'field list'

mysql error number: 1054
Any ideas?

jarvis 11-29-2003 02:54 PM

All,

I have had this hack running on/off for over a year now. One of the big things that keeps me from consistently using it is the search feature and new posts count when a user logs in.

Anyways, I have fixed the search feature to discriminate between Usenet and normal forum posts but have not yet figured out how to disciminate between the new posts counter to exclude the two types of posts.

Rambling, I know, but I would like to see two numbers versus one large. :D

Anyways, I am by no means a php coder. I am horrible in coding, but I did figure it should be pretty easy to exclude a forumid from the $newposts routine?

Any help would be greatly appreciated!! :D

Thanks!

lierduh 12-03-2003 07:31 PM

Quote:

Originally Posted by KevinM
I am attempting to put this into a clean vb3 board.

I am using the gateway script adapted by lierduh, with the rest being from Gilby's original.

I get the following error when running gateway.php for the first time:



Any ideas?

Are you sure you are using gatewas.php patched by me?

attachmentid is not in any of the SELECT queries in my gateway.php. However, I did find a typo in one of the SQL statement regarding attachment, which does not give errors like the one you have anyway. I have uploaded the fixed gateway.php. Please grab the new one if you want to try it out.

csimo 12-07-2003 05:28 PM

I'm not trying to cross any lines here, but does this work with vb3? If so it would be a lifesaver for me. Instructions? Info?

Thanks!

nkouts 12-07-2003 07:45 PM

Quote:

Originally Posted by jarvis
All,

I have had this hack running on/off for over a year now. One of the big things that keeps me from consistently using it is the search feature and new posts count when a user logs in.

Anyways, I have fixed the search feature to discriminate between Usenet and normal forum posts but have not yet figured out how to disciminate between the new posts counter to exclude the two types of posts.

Rambling, I know, but I would like to see two numbers versus one large. :D

Anyways, I am by no means a php coder. I am horrible in coding, but I did figure it should be pretty easy to exclude a forumid from the $newposts routine?

Any help would be greatly appreciated!! :D

Thanks!

csimo - would love to know as well :)

Gardener 12-08-2003 08:59 PM

lierduh,

Great work on this hack. I had it up and running on the beta release. Then I upgraded to gamma and reinstalled the hack and the latest gateway.php file you have and this is the error I'm getting:

Fatal error: Call to undefined function: convert_url_to_bbcode() in \www\vb3\gateway.php on line 181

lierduh 12-09-2003 08:02 PM

Quote:

Originally Posted by csimo
I'm not trying to cross any lines here, but does this work with vb3? If so it would be a lifesaver for me. Instructions? Info?

Thanks!

Not, but I have a modified version:

https://vborg.vbsupport.ru/showpost....&postcount=658

lierduh 12-09-2003 08:18 PM

Quote:

Originally Posted by Gardener
lierduh,

Great work on this hack. I had it up and running on the beta release. Then I upgraded to gamma and reinstalled the hack and the latest gateway.php file you have and this is the error I'm getting:

Fatal error: Call to undefined function: convert_url_to_bbcode() in \www\vb3\gateway.php on line 181

I have this line on the top:
require_once("includes/functions_bbcodeparse.php");

The function is in that file. If it is not there anymore, please do a search for that function (convert_url_to_bbcode). If you use Linux, you can try:

#grep "function convert_url_to_bbcode" * -r

If this function does not exist anymore. Just delete the line containing this function in the gateway.php. It is only used to convert url to bbcode.

BenS 12-14-2003 08:31 PM

When I run gateway.php it gives me a blank screen, any ideas?

BenS 12-14-2003 08:46 PM

Ok, now it's asking me if I want to download gateway.php, what's wrong???

Gilby 12-14-2003 09:01 PM

Quote:

Originally Posted by BenS
Ok, now it's asking me if I want to download gateway.php, what's wrong???

Sounds like you have your browser set up to download and save text files. The gateway script will output what it is doing in text instead of html so it tells the browser that in the header of the response.

lierduh 12-18-2003 11:16 AM

Quote:

Originally Posted by Gardener
lierduh,

Great work on this hack. I had it up and running on the beta release. Then I upgraded to gamma and reinstalled the hack and the latest gateway.php file you have and this is the error I'm getting:

Fatal error: Call to undefined function: convert_url_to_bbcode() in \www\vb3\gateway.php on line 181

I have uploaded a new gateway.php for working with the v3 Gamma. Please refer to my original post.

KevinM 12-20-2003 03:00 PM

Thanks lierduh, I have this working for vb3, however none of the posts are searchable (there are always no results). Is there something I need to alter?

lierduh 12-24-2003 10:09 PM

Quote:

Originally Posted by KevinM
Thanks lierduh, I have this working for vb3, however none of the posts are searchable (there are always no results). Is there something I need to alter?

That rings a bell, I might not have that implemented at the time. I will go through the code later to fix this.

Edit:

I just had a look, the code actually has the search index builder at line 262 and 353. In this case, I am not sure why the posts are not searchable for you. Perhaps make sure you have:

// build_post_index function
require_once("includes/functions_databuild.php");

in the code.

dethfire 12-29-2003 04:48 AM

I get these errors

1 group(s) gatewayed.
Could not connect to NNTP-serverLogging in to groups.google.com, group rec.autos.sport.misc
Not connectedNot connected

KevinM 12-31-2003 08:24 PM

Lierduh> I've upgraded to gamma (forum & gateway) but still have the same problem. All the lines are in the code. Searching is enabled on the boards, I just get 'sorry no matches'. I've had a look at the db in phpmyadmin - there are no entries in the 'word' table (from memory I think that is where the index is built). Same thing with thread preview. I'll keep looking..

Dethfire> I'm not aware that google groups provide a usenet/newsgroup feed. You need a provider like www.teranews.com or www.newsfeeds.com


All times are GMT. The time now is 09:11 PM.

Powered by vBulletin® Version 3.8.12 by vBS
Copyright ©2000 - 2025, vBulletin Solutions Inc.

X vBulletin 3.8.12 by vBS Debug Information
  • Page Generation 0.02322 seconds
  • Memory Usage 1,882KB
  • Queries Executed 10 (?)
More Information
Template Usage:
  • (1)ad_footer_end
  • (1)ad_footer_start
  • (1)ad_header_end
  • (1)ad_header_logo
  • (1)ad_navbar_below
  • (2)bbcode_code_printable
  • (16)bbcode_quote_printable
  • (1)footer
  • (1)gobutton
  • (1)header
  • (1)headinclude
  • (6)option
  • (1)pagenav
  • (1)pagenav_curpage
  • (4)pagenav_pagelink
  • (1)pagenav_pagelinkrel
  • (1)post_thanks_navbar_search
  • (1)printthread
  • (40)printthreadbit
  • (1)spacer_close
  • (1)spacer_open 

Phrase Groups Available:
  • global
  • postbit
  • showthread
Included Files:
  • ./printthread.php
  • ./global.php
  • ./includes/init.php
  • ./includes/class_core.php
  • ./includes/config.php
  • ./includes/functions.php
  • ./includes/class_hook.php
  • ./includes/modsystem_functions.php
  • ./includes/class_bbcode_alt.php
  • ./includes/class_bbcode.php
  • ./includes/functions_bigthree.php 

Hooks Called:
  • init_startup
  • init_startup_session_setup_start
  • init_startup_session_setup_complete
  • cache_permissions
  • fetch_threadinfo_query
  • fetch_threadinfo
  • fetch_foruminfo
  • style_fetch
  • cache_templates
  • global_start
  • parse_templates
  • global_setup_complete
  • printthread_start
  • pagenav_page
  • pagenav_complete
  • bbcode_fetch_tags
  • bbcode_create
  • bbcode_parse_start
  • bbcode_parse_complete_precache
  • bbcode_parse_complete
  • printthread_post
  • printthread_complete