NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

[mzeitler]

Hi,

I am pretty happy with the add-on. However some of the threads are duplicated since the usenet poster crossposts into multiple groups.

Is there a way to prevent this duplication from happening?

Since I compbine multiple usenet groups into one master group on my site, this is really annoying.

Matthias

[amaarvell]

hi guys

I am still having the problem I posted on reply 499
the anoying thing is occasionally the odd post gets through (3 so far)
this is wracking my head .. please help me someone

[bkaul]

63669

The link is to showthread.php?p=0#post0 rather than the actual post number, when the post is one that is generated by the gateway. Any ideas on what modifications would be necessary to fix this bug?
Clarification: I think this is only true of the initial post in a newly created thread, before any replies. Replies to existing threads appear to work properly.

[tadast]

Rik,
Invalid SQL:
UPDATE nntp_settings
SET value = WHERE varname = 'last_postid';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 2
Error Number : 1064
Date : Wednesday, April 18th 2007 @ 04:59:06 PM
Script : http://forums.networkingland.com/admincp/cronadmin.php?do=runcron&cronid=19
Referrer : http://forums.networkingland.com/admincp/index.php
IP Address : 212.35.77.232
Username : admin
Classname : vB_Database


There are ' missing in the gateway.php code, change


SET value = " . ((empty($get_max_postid))?(0):

($get_max_postid['postid'])) . " WHERE varname = 'last_postid'


to

SET value = '" . ((empty($get_max_postid))?(0):

($get_max_postid['postid'])) . "' WHERE varname = 'last_postid'


That would fix the problem.

[panda1]

There are ' missing in the gateway.php code, change


SET value = " . ((empty($get_max_postid))?(0):

($get_max_postid['postid'])) . " WHERE varname = 'last_postid'


to

SET value = '" . ((empty($get_max_postid))?(0):

($get_max_postid['postid'])) . "' WHERE varname = 'last_postid'


That would fix the problem.

Thank you tadast for the help!


Now I have this NTTP newsgroup hack working fine with VB 3.5.6 in my new forum http://forums.networkingland.com.

But I am still facing posting problems, I am using www.teranews.com news server with posting allowed.


Frankly speaking, I am very happy with this hack, it saved me months of work! many thanks to the coder who created this hack in the first place and to all of you fellows for the help.

[amaarvell]

Guys I have this installed but I cant get it working

Gateway debug says its fine
but the cron fails every time

vbulletin say its nothing to do with them and my web host says its nothing to do with them

the script appears to be installed correctly
and my head is in bits trying to sort it

please help

[amaarvell]

Now I have this NTTP newsgroup hack working fine with VB 3.5.6 in my new forum http://forums.networkingland.com.

you are running the gateway on the same usenet groups as me and on the same vb version .. any idea what i am doing wrong ... www.vistareadygames.com

cheers

[X-Man70]

when i start the gateway in debug mode 1, i become an erreo msg. can someone help me ?

Datenbankfehler in vBulletin 3.6.5:

Invalid SQL:

SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle, post.visible AS postvisible,
thread.visible AS threadvisible
FROM post as post LEFT JOIN thread as thread
ON (thread.threadid = post.threadid
AND post.userid = thread.postuserid
AND post.postid = thread.firstpostid)
WHERE post.isusenetpost = 0
AND post.postid > 0
AND thread.forumid = 40;

MySQL-Fehler : Lost connection to MySQL server during query
Fehler-Nr. : 2013

[panda1]

you are running the gateway on the same usenet groups as me and on the same vb version .. any idea what i am doing wrong ... www.vistareadygames.com

cheers

Hi Amaarvel,

You are right we are running the same VB version 3.6.5 and the same newsgroup server!

It wasn't a staright forword job, actully it took me some time to fix it ( I am not a coder)

Which satge you are stuck in ?

Can you post the debug log if you reached that stage.

More details will help both of us.

[amaarvell]

Hi Panda
as far as i know its set right
if i use the debug it says its gateway'ed open

but thats it
the cron runs the gateway script every 15 minutes
but in the last 10 days i have got 4messages through

I know its a microsoft server but it should be a whole lot faster than that

any help i get will save the samaratins a head ache

[panda1]

Hi amaarvell,

It is not normal to get few messages in days or even hours.

You can try the following free usenet server : freetext.usenetserver.com ( posting not allowed) no need for username or password to start importing usenet messages.

I thing the gateway.php script is not consistent in it's behavior and it's results.

Right now one of my usenet groups is not updated since May 2nd ,2007 without any known reason from teranews.com or VB.

From admin Control Panel go maintenance and open update counters and rebuild some items there, it might help!.

Also, from admin Control Panel go maintenance and open repair/ optimize tables and repair some tables there, it might help!.

As I told you I am not a coder or programer I am a networking man!

Good Luck!

[amaarvell]

oh crap .. lol
nightmares upon night mares

it downloaded a lot of posts but 2 and 3 duplicates of each one .. how do i stop it doing that
as i now have to find a way to delete 40,000 duplicate posts

hehehe aint i the lucky one
insodently i used msnews.microsoft.com as a server

[panda1]

Don't panic my friend!

You can do duplicate deletion from VB control Panel - Maintenance- update counters- Delete Duplicate Threads.

This will solve the duplication problem, i guess!

If worked with this freetext.usenetserver.com server then you might have problems with teranews.com.

good luck!

[fenris]

Hy,

ich hoffe hier kann jemand Deutsch und mir weiterhelfen.

habe den Hack installiert und es klappt so weit alles sehr gut. Wenn ich aber nun z.B. a.b.mom auslesen lasse und dann im Forum auf den entsprechenden Beitrag klicke, versucht das Forum z.B. eine gepostetet Datei zu öffnen - was natürlich nicht klappt. Wie kann ich das den einstellen, das z.B. nur die Header ausgelesen werden und nicht die ganzen Dateien ? Oder gibt es z.B. nur die Möglichkeit neue Posts mit der Dateiendeung txt anzeigen zu lassen ?

Gruß

Fenris

[TMM-TT]

63669

The link is to showthread.php?p=0#post0 rather than the actual post number, when the post is one that is generated by the gateway. Any ideas on what modifications would be necessary to fix this bug?
I also had this problem until I changed the code in includes/functions_nntp.php

Somewhere around line 591 there should be something like this:

//So that thread preview works
$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET firstpostid = $postid
WHERE threadid = $threadid
");
... was changed to ...

//So that thread preview works
$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET firstpostid = $postid,
lastpostid = $postid
WHERE threadid = $threadid
");
After this I didn't have any trouble with it... I'm not sure it's a correct way to fix it, but it at least removes the zero. :)

[digital3]

As someone who has never used USENET at all, let alone attempted to find a good newsgroup or set a forum up to run it I am at a loss here. Installing the mod was simple, of course, but configuring it to actually work is another matter. There are a lot of settings in this thing under Gateway Newsgroups Management and I have no clue where to get the information I need to place in these fields. I found a newsgroup that I want to join but where do I get the info I need to connect to it? I can't find these details anywhere on their site. Do I need to create a sticky thread for this Newsgroup "before" I try adding it? There should really be a guide somewhere on how to set this thing up from ground zero. Not everyone has used USENET before and this readme makes a lot of assumptions that the end user already knows certain basic things.

[itdevil]

As someone who has never used USENET at all, let alone attempted to find a good newsgroup or set a forum up to run it I am at a loss here. Installing the mod was simple, of course, but configuring it to actually work is another matter. There are a lot of settings in this thing under Gateway Newsgroups Management and I have no clue where to get the information I need to place in these fields. I found a newsgroup that I want to join but where do I get the info I need to connect to it? I can't find these details anywhere on their site. Do I need to create a sticky thread for this Newsgroup "before" I try adding it? There should really be a guide somewhere on how to set this thing up from ground zero. Not everyone has used USENET before and this readme makes a lot of assumptions that the end user already knows certain basic things.

Same here... Looking for public accessible Usenet server, but unable to find any. There a few paid one i found, alas none free...

Anyone have any server were they could share to use with this great mod ?

[Geraldm]

Hi,

I've installed everything as instructed in the readme text file. I have setup a new forum and added a NNTP newsgroup. But when I run the gateway.php I get the following error:

Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection refused (111))

Not connected
Not connected


The NNTP server address is correct as well as the username and password. The details have been checked with the NNTP reader called Agent and it works fine.

Does anyone know why I am getting this error when using the gateway?

Regards,
Gerald.

[Geraldm]

Ok, I found what was going on ...... My host (hostgator) was blocking port 119 (NNTP). I've spoken to them and they have now unblocked that port and the script now works perfectly.

Thank you! :)

Cheers ....
Gerald.





Hi,

I've installed everything as instructed in the readme text file. I have setup a new forum and added a NNTP newsgroup. But when I run the gateway.php I get the following error:




The NNTP server address is correct as well as the username and password. The details have been checked with the NNTP reader called Agent and it works fine.

Does anyone know why I am getting this error when using the gateway?

Regards,
Gerald.

[Geraldm]

Hi,

I have another question. When importing USENET messages, is there a way to set the maximum length of the original posters username?

My board is set to only accept usernames with a maximum length of 25 characters, but when importing USENET messages this setting is overwritten. How can I tunicate the usernames?

Regards,
Gerald.

[SaN-DeeP]

I am looking for an immediate fix unable to post/export messages to newsgroups, I am able to receive/import messages properly .

Here is additional information, this was working fine 20 days back and now stopped, I am unable to fix/debug the issue.

Connecting to server, server says: 200 Text.GigaNews.Com
Server responded after user name: 381 more authentication required
Server responded after password: 281 News.GigaNews.Com
Info for microsoft.public.broadbandnet.hardware at text.giganews.com: 211 19459 1927 21385 microsoft.public.broadbandnet.hardware

Can we post? server responded: 440 posting not allowed
Posting Message 'Why would one decide to cool a computer with water instead of air ?' from Watercooling. Result: 500 syntax error or unknown command

[Alfa1]

Would it be possible to promote your forum on usenet with this hack? i.e. have each post by a forum member include a reference to your site? If so, then this may yield quite some new members.
Is there a 3.6 port of this hack? I couldn't find it.

[Eccentricity]

I am looking for an immediate fix unable to post/export messages to newsgroups, I am able to receive/import messages properly .

Here is additional information, this was working fine 20 days back and now stopped, I am unable to fix/debug the issue.

Connecting to server, server says: 200 Text.GigaNews.Com
Server responded after user name: 381 more authentication required
Server responded after password: 281 News.GigaNews.Com
Info for microsoft.public.broadbandnet.hardware at text.giganews.com: 211 19459 1927 21385 microsoft.public.broadbandnet.hardware

Can we post? server responded: 440 posting not allowed
Posting Message 'Why would one decide to cool a computer with water instead of air ?' from Watercooling. Result: 500 syntax error or unknown command
This isn't a problem with your code. The server you are using isn't liking your login credentials (381 more authentication required) and isn't allowing posting.

Use this free server here and you will be able to post: nntp.aioe.org (Or get your credentials sorted out with giganews).

Also, if that Microsoft feed is your only one, just use one of the Microsoft Public Access servers. They all allow posting and they are free also. msnews.microsoft.com is one such server.

[Eccentricity]

Would it be possible to promote your forum on usenet with this hack? i.e. have each post by a forum member include a reference to your site? If so, then this may yield quite some new members.
Is there a 3.6 port of this hack? I couldn't find it.

Each post includes a link to your thread and the member's profile in the footer of the post. You just need to add your domain name in place of the default one in the Gateway Settings menu.

There is no 3.6 port and I doubt there will be as long as this one works. There are a few bug fixes scattered out inside this thread so it might be worth doing a page by page scan to see what all is here. The hack, in it's original forum, is quite buggy but the fixes donated by members in this thread make it bearable.

[Alfa1]

Many thanks for your helpful answer. You write that after the fixes are applied, it becomes bearable. What bugs remain?

With this being such a interesting and useful add-on, I do hope that someone will try to port this, include the listed fixes and straighten out the rest of the bugs.

[Eccentricity]

Many thanks for your helpful answer. You write that after the fixes are applied, it becomes bearable. What bugs remain?

With this being such a interesting and useful add-on, I do hope that someone will try to port this, include the listed fixes and straighten out the rest of the bugs.The last bug I have is that all imported posts are dated December 31, 1969 in postbit view. That is annoying the crap out of me. I am here looking for a fix for this now.

[TMM-TT]

I don't know yet, if anyone has reported this bug - i neither don't know if I've mentioned this earlier. But there is actually a problem with usenet-servers that are configured as peer-servers, especially if the forum-usenet-gateway is spotted on the same server as there are outgoing peers. I've got a mail that made me realize this. I'm quoting myself here:

I think you've just discovered some very strange bug here. For a time ago, I moved the usenet-server to another one, internally because the main server got too high loaded with the service. By moving it to another server that only takes care of the usenet everything started to work again. The problem here seem to be the firewall that forwards traffic to my internal server. It can't handle the NAT correct so instead of taking the connection as a news-reader, all traffic are handled like they were newspeers. This is normally, by regular news-readers fixed by itself, since they are sending the string "MODE READER" as a handshake-command. The nntp-gateway don't do this, so instead of work properly some permissions are wrong on connection.

I've just tested this, and it may be fixed by adding a line into includes/nntp.php - I tried this before but something happened after that and the gateway didn't work as good as I thought. I think it works this time, but I can't confirm... (this part is quite important /TMM)

Find the lines for the handshake part. It should be around line 80 or so. You are looking for:

fputs($fp, "AUTHINFO user $username\r\n");

Above this line, add:

fputs($fp, "MODE READER\r\n");
$response = fgets($fp, 128);

By patching like this, you're telling the server that you are a client and not a server.

[KidCharlemane]

This fix did not work for me but I emailed you directly about it.

[TMM-TT]

This fix did not work for me but I emailed you directly about it.

I think that part is just a lastread-issue. Because of the local change of servers the lastread data is probably mismatching with yours. When the gateway connects to the server it reads (I think) the firstpost-lastpost-information from the server (211*) and compares it with the data stored in the database. At least it's something like that. If your value is larger than the lastpost-data sent by server (which it probably is what happened here**), the gateway won't download any new articles, since it thinks that you already have them. Resetting that value in the database might help, with a small risk that there will pop up some dupes.


**) I run a larger server and a smaller, and currently, traffic are forwarded (with a shitty NAT-rule) to the smaller due too high load on the larger. The smaller, unfortunately, have other data stored, that makes the counters differ a bit (and that is bad too)

*) When fetching posts from usenet this command is sent to the server
GROUP news.software.nntp
The server answers with
211 41 63 103 news.software.nntp

Where the data received stands for (from the example):
41 = estimated number of articles in group, 63 = first article number in the group, 103 = last article number in the group

I'm not sure if the gateway does like this, because I didn't find anything about it in the source, but since there is a lastread-field in the admincp, some similar calculations are made, that for the moment prevents new posts to reach you... And yes, that is probably my fault.. :o

[vze2yqtm]

I've added most of the hacks that were added to this thread and then installed it.. It actually took me a couple of hours to go through all the posts. Now I have a problem. I didn't install all the files that came with the original files like the setlastmsg.php, mime.php, pop.php, the XML file (cpnav_nntp_gateway.xml) because it didn't specify it in the Read Me as far as I can tell I used option 2 for installation because it was mentioned earlier in this thread. Now when I try to run mysite.com/gateway.php?debug=1 I get this error:

Parse error: syntax error, unexpected T_ELSE in /home/username/public_html/gateway.php on line 385

I'm running 3.6.7.

Anyone knows what the problem could be? It looks to me like it could be some kind of error with the code DKendall supplied only because the error seems to be coming from that line, but I'm no PHP expert and I'm not really sure.

[TMM-TT]

I've added most of the hacks that were added to this thread and then installed it.. It actually took me a couple of hours to go through all the posts. Now I have a problem. I didn't install all the files that came with the original files like the setlastmsg.php, mime.php, pop.php, the XML file (cpnav_nntp_gateway.xml) because it didn't specify it in the Read Me as far as I can tell I used option 2 for installation because it was mentioned earlier in this thread. Now when I try to run mysite.com/gateway.php?debug=1 I get this error:

Parse error: syntax error, unexpected T_ELSE in /home/username/public_html/gateway.php on line 385

I'm running 3.6.7.

Anyone knows what the problem could be? It looks to me like it could be some kind of error with the code DKendall supplied only because the error seems to be coming from that line, but I'm no PHP expert and I'm not really sure.
And what does line 385 say?

[vze2yqtm]

Looks like this is what it says:

$kf = killfile_match();
else


Here's a little bit more of the code in case that part isn't enough:

elseif ($nntp['grouptype'] == 'mail'
AND $group['prefix']
AND stristr($message['subject'], $group['prefix']) == false)
{
logging("Skip, not matching prefix: \"" . $group['prefix'] . "\"");
}

if(!$skip_post)
{
$kf = killfile_match();
else
{
$threadid = 0;
$attachmentid = 0;
$parentid = 0;

if ($nntp['grouptype'] != 'news')
{
$message['text'] = stripfooter($message['text']);
}

[TMM-TT]

Ok.. Something is just missing there... This part:

$kf = killfile_match();
else

looks like this in my gateway.php:

$kf = killfile_match();
if ($kf)
{
logging("Skip, killfile \"" . $kf . "\" match.");
}
else


and after that, it looks right :)

[Fabsboards]

Are there any plans to update this for 3.6.7? If not, are there any other similar mods?

[Alfa1]

This is the only mod for Usenet.

[vze2yqtm]

It seems like I have everything working fine, but now I'm getting a 411 error no such group. I am using about 20 different newsgroups and I know some of them are good. Why am I getting this error for most of them? The servers are replying and giving info like no posting and stuff like that, why is it saying they don't exist?

Here's an example:

Connecting to server, server says: 200 aioe.org InterNetNews NNRP server INN 2.4.4 (20060818 snapshot) ready (posting ok).

Info for nntp.aioe.org at nntp.aioe.org: 411 No such group nntp.aioe.org

[TMM-TT]

411 [...snip...] I am using about 20 different newsgroups and I know some of them are good. Why am I getting this error for most of them?

Connecting to server, server says: 200 aioe.org InterNetNews NNRP server INN 2.4.4 (20060818 snapshot) ready (posting ok).

Info for nntp.aioe.org at nntp.aioe.org: 411 No such group nntp.aioe.org

It seems that something in your configuration is wrong? In the example, you are requesting the newsgroup "nntp.aioe.org" and I don't think that group exist on that server. As you say, error 411 means that the group doesn't exist, and there could be two reasons why you get that message; a) the group really don't exist there b) your configuration may be wrong somewhere (ie you have configured aioe as a server but have put the servername in the wrong field, as shown from the example).

Another thing that may be important for aioe-users: They only allow 25 posts/day (http://news.aioe.org/spip.php?article3) per ip. If you're running a server with many users, those 25 may be used up quite fast.

[vze2yqtm]

The problem is I'm getting this with about 15 other newsgroups. It could be my settings since they are basically the same for all of them. Here's a screenshot of the settings I have (I've attached them here for future viewers and also uploaded them temporarily for ease of viewing):

https://vborg.vbsupport.ru/

Most of the newsgroups I'm attempting to use appear to not use username and passwords so that's the reason it's not included.

Newsgroup/Mailing List Email: I put in link
Forum: Forum I want it to show in
Prefix: I've left blank
Last Message: 0
Server: I put in the link again
The rest I've left blank (username and password are not required on these newsgroups) and enabled it.

Am I filling it out wrong?

By the way I appreciate your help. Thank you.

[TMM-TT]

The problem is I'm getting this with about 15 other newsgroups. It could be my settings since they are basically the same for all of them. Here's a screenshot of the settings I have (I've attached them here for future viewers and also uploaded them temporarily for ease of viewing):

http://customrequest.com/nntpgatewayscreenshot.gif

Most of the newsgroups I'm attempting to use appear to not use username and passwords so that's the reason it's not included.

Newsgroup/Mailing List Email: I put in link
Forum: Forum I want it to show in
Prefix: I've left blank
Last Message: 0
Server: I put in the link again
The rest I've left blank (username and password are not required on these newsgroups) and enabled it.

Am I filling it out wrong?

By the way I appreciate your help. Thank you.
That's true... aioe doesn't allow username/password at all, it's free to use but with a 25-post-per-day-only-limit... What newsgroups are you looking for, actually?

The only wrong I can see in your attachment is that the top field for the newsgroup-name should be an existing newsgroup. The current you have in your screenshot doesn't exist :)



200 aioe.org InterNetNews NNRP server INN 2.4.4 (20060818 snapshot) ready (posting ok).
GROUP nntp.aioe.org
411 No such group nntp.aioe.org

I think my newsbot is a bit out of date now, but if you need a complete list of that server's groups, you can look here (http://news.tornevall.net/list/list.nntp.aioe.org).

There's only one thing I'm thinking of now - does the gateway work when sending empty identification strings, since aioe don't use authentication?

[vze2yqtm]

It worked!!! Thanks again.

I changed Newsgroup/Mailing List Email to one of the groups from the link you provided (I picked alt.accounting) and got some posts. I'm actually only testing the script for now, I have not placed it in a live section of my forum. It's in an admin viewable only section.

As far as authentication, I'm not sure, but everything else seems to work fine, I guess it was just that I didn't pick the right Newsgroup name and that seemed to fix everything. I'm still testing so I may be back with more questions, but I hope everything works for now. I'll probably end up using one of the paid newsgroup solutions, but I want to get all of my testing out of the way with free newsgroups for now.

[tpearl5]

anyone get this working on 3.6.8? If so what did you have to do?

[tpearl5]

Nevermind, I got it working, I just had to up the database timeout cause it encountered a large import.

[connkid25]

I got this working on 3.6.8. Works good except for the scheduled tasks in VB. I instead want to do this by cron with the server itself. Does anyone know how to get this to work using the crontab manager?

I honestly have no clue how to set up a cron job using that and the instructions arent much help.

[dotcomguy]

I got this working on 3.6.8. Works good except for the scheduled tasks in VB. I instead want to do this by cron with the server itself. Does anyone know how to get this to work using the crontab manager?

I honestly have no clue how to set up a cron job using that and the instructions arent much help.

It depends on your Web host. Your Web host may have a control panel where you can enter cron jobs. Or, you might have to contact your Web host.

If you have access to add cronjobs directly, the format would be:


*/10 * * * * /usr/bin/lynx -source "http://www.example.com/forums/gateway.php"

[connkid25]

It depends on your Web host. Your Web host may have a control panel where you can enter cron jobs. Or, you might have to contact your Web host.

If you have access to add cronjobs directly, the format would be:


*/10 * * * * /usr/bin/lynx -source "http://www.layover.com/forums/gateway.php"


Thanks for your reply. I have a control panel and I took a screen shot. I assume I just need a command with the direct location of the file? This is a dedicated server so I have full access.

[dotcomguy]

You would add "/usr/bin/lynx -source "http://www.example.com/forums/gateway.php" as the command (provided that is the correct path to your lynx and that lynx is installed)

[connkid25]

You would add "/usr/bin/lynx -source "http://www.example.com/forums/gateway.php" as the command (provided that is the correct path to your lynx and that lynx is installed)

I just checked. It is the correct path and it is installed. Just added that. I have it set to update every 10 mins so we'll see if it works.

Thanks for taking the time to help me out with this. It's much appreciated.

EDIT: That worked! Thank you! Just had to remove the first " and everything updated.

[z3r0]

Can anybody point me in the right direction for removing the >> indents as this looks very messy.

Thanks

[Mum]

OK i've read through all the pages, but i can't figure out what's wrong. I've just installed it, and getting this error

Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected


I know it works because i tested it in my desktop news reader, no username or password needed. Here is a screen cap of my settings. Thanks in advance for your help :)

http://img528.imageshack.us/img528/588/scr3ddd47kr9.th.png (http://img528.imageshack.us/my.php?image=scr3ddd47kr9.png)

[AWS]

You're using a news server that only allows connections from your IP.
If you're using a server that doesn't require a username or password then that is the problem.

[Mum]

Apart from just Yahoo groups, can you use myspace groups? Any other ideas what other ones you can use?

[silurius]

I am having trouble with scheduling this to run in a standard cron job.

Cron job:
/usr/local/php5/bin/php /home/myuser/mysite.com/forum/gateway.php

I believe init.php is designed to use a relative path on line 42 and if that fails it grabs the full path from config.php, as shown here:

// ################################################## ###########################
// fetch the core includes
require_once(CWD . '/includes/class_core.php');

// initialize the data registry
$vbulletin =& new vB_Registry();

// parse the configuration ini file
$vbulletin->fetch_config();

if (CWD == '.')
{
// getcwd() failed and so we need to be told the full forum path in config.php
if (!empty($vbulletin->config['Misc']['forumpath']))
{
define('DIR', $vbulletin->config['Misc']['forumpath']);
}
else
{
trigger_error('<strong>Configuration</strong>: You must insert a value for <strong>forumpath</strong> in config.php', E_USER_ERROR);
}
}
else
{
define('DIR', CWD);
}

if (!empty($vbulletin->config['Misc']['datastorepath']))
{
define('DATASTORE', $vbulletin->config['Misc']['datastorepath']);
}
else
{
define('DATASTORE', DIR . '/includes/datastore');
}

if (!$vbulletin->debug)
{
set_error_handler('vb_error_handler');
}Unfortunately, even when I specify the full path in config.php the cron job fails. And when I enter the full path in line 42 of init.php, my whole site breaks. I've highlighted the bad path here.

Error:
Warning: require_once(/home/.saint/user/includes/class_core.php): failed to open stream: No such file or directory in /home/.saint/user/mysite.com/forum/includes/init.php on line 42

Fatal error: require_once(): Failed opening required '/home/.saint/user/includes/class_core.php' (include_path='.:/usr/local/php5/lib/php:/usr/local/lib/php') in /home/.saint/user/mysite.com/forum/includes/init.php on line 42

Anyone have ideas?

[joined]

Hi to all.

I have make some changes and fixes.



// Modified by Fabio Dell'Aria on 20-Sep-2007 (fabio.dellaria@gmail.com)
// Modified Files: nntp.php, functions_nntp.php
// ---------------------------------------------------------------------
// Changes:
// --------
// 1)...Added the ability to can works with local NNTP servers;
// 2)...Added the ability to can avoid duplicated NNTP messages;
// 3)...A little NNTP Send speed improvement;
// 4)...Increased the Debug details;
// 5)...Fix a bug that caused "Invalid From address" NNTP error;
// 6)...Fix a bug that caused "Duplicated Message" NNTP error.
// ---------------------------------------------------------------------

How I can includes them in the OFFICIAL release?

[arkadia]

hello,

i try to download this newsgroup: nzn.fr.delphi (server server.amis.tv) and i can't !! it's work for some time (sometime i can download 500 messages, sometime 3000), but i always have : Fatal error maximum execution time of 1800 seconds exceeded in nntp.php on line 121 !

where the problem is? what can i do?

thanks you by advance !
stéphane

[TMM-TT]

Fatal error maximum execution time of 1800 seconds exceeded in nntp.php on line 121 !

where the problem is? what can i do?


Large posts that makes the script time out maybe? I had a similar problem myself on some groups before, and that was unfortunately never solved for me..

[arkadia]

yes, but in fact it's impossible to make the script workin ... i always have this error !! this is on the line : $response = trim(fgets($this->fp, 128));

when i check on outlook express, the server work perfectly without any probleme ! you can try : nzn.fr.delphi (server server.amis.tv)

[dotcomguy]

hello,

i try to download this newsgroup: nzn.fr.delphi (server server.amis.tv) and i can't !! it's work for some time (sometime i can download 500 messages, sometime 3000), but i always have : Fatal error maximum execution time of 1800 seconds exceeded in nntp.php on line 121 !

where the problem is? what can i do?

thanks you by advance !
st?phane
I think I had the same problem with execution time when I first used this mod. However it was only a problem initially because I was downloading everything. I just had to keep running it automatically until it had everything downloaded. Now it runs frequently enough that it is able to download all the messages without any trouble.

[arkadia]

But now i have a BIG problem! it's sometime crash php-cgi.exe (with big windows error popup) !!!

[itdevil]

I am having similar issues, the mod put too much load on the server, i am 2x DualCore Xeon 2.8 with 2GB ram 100mbit.... Cron set to run every 24hrs or midnight :o... once the cron run the load shoots up to 6 - 20 till apache crashes....

even now the server is dead slow due the gateway.php cron

xyz 18394 11.2 3.6 86660 74972 ? Ss 00:00 25:39 /usr/bin/php /home/xyz/public_html/forum/gateway.php

i have also noticed, sometimes gateway file (but stop leeching from news server) never get killed automatically, on next cron another process is created...

This is a great hack, but someone need to update and optimize the code.

[KW802]

Most likely you're trying to grab too many threads (aka: articles) in one session resulting in the timeout. Before bringing in a new group check to see what the last article ID is and then retrieve only the last hundred or so articles. Do that and add one group at a time and see if there if a difference in your results.

[dotcomguy]

I am having similar issues, the mod put too much load on the server, i am 2x DualCore Xeon 2.8 with 2GB ram 100mbit.... Cron set to run every 24hrs or midnight :o... once the cron run the load shoots up to 6 - 20 till apache crashes....

even now the server is dead slow due the gateway.php cron

xyz 18394 11.2 3.6 86660 74972 ? Ss 00:00 25:39 /usr/bin/php /home/xyz/public_html/forum/gateway.php

i have also noticed, sometimes gateway file (but stop leeching from news server) never get killed automatically, on next cron another process is created...

This is a great hack, but someone need to update and optimize the code.
I haven't had the same problem, but I would recommend running it more than once a day. I run it every 10 minutes. Otherwise you'll have to download a significantly more amount of data

[krohnathlonman]

the largest bottlekneck in large scale archiving is the hard drive speed. If you don't have high speed SCSI drives don't ever try to archive millions of posts! Run things manually and NOT through cron if you're weeks behind in the posts.

[arkadia]

i build a new nttp gateway from zero using php, C++ and Delphi. it's works very more efficiently than the actual nntp gateway (i test it on a 100.000 articles newsgroup without any problem) and also i inprouve the charset encoding/decoding and correct some litle bug i found in gateway.php

exemple : after inserting an article
$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET firstpostid = $postid
WHERE threadid = $threadid
");

that look false and must be

$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET lastpostid = $postid
WHERE threadid = $threadid
");

:)

i need now some volonters to test it and to gave me they comments !

[TMM-TT]

i build a new nttp gateway from zero using php, C++ and Delphi. it's works very more efficiently than the actual nntp gateway

By the way - don't forget the handshake part between server<->client. The "MODE READER" (that should be placed before the AUTHINFO) is quite important to use, to identify the client as a client. :)

[arkadia]

i not understand exactly what you mean? off course i handle the AUTHINFO and all the neccessary nntp protocol !

[TMM-TT]

i not understand exactly what you mean? off course i handle the AUTHINFO and all the neccessary nntp protocol !
In the current version of the gateway the client "identification" is missing. Since I run a usenet-server that communicates internally with another news-server, the gateway can't handshake correctly without the MODE-string.

A "normal" usenet-client sends "MODE READER" to a server before the authorization, and a usenet-server sends "MODE SERVER" to another server, because of the different article-transfering modes. This usually is based on IP, so if a server is configured to handle feed-traffic, it also treats a local client as a server - which is wrong.

It doesn't happen often, but it happens and may be important for some. :)

[arkadia]

ok, i do it ! i send a mode reader just after the connection

[scoutz]

wow this mod seem to work fine in 3.6.8 as well as 3.6.7 just needs a few very small query fixes. thanks for this mod.

[mccollin]

Would you share the fixes?

[solokron]

This is what I am receiving when I run gateway.php

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
<title>Domain - Forum Database Error</title>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" />
<style type="text/css">
<!--
body { background-color: white; color: black; }
#container { width: 400px; }
#message { width: 400px; color: black; background-color: #FFFFCC; }
#bodytitle { font: 13pt/15pt verdana, arial, sans-serif; height: 35px; vertical-align: top; }
.bodytext { font: 8pt/11pt verdana, arial, sans-serif; }
a:link { font: 8pt/11pt verdana, arial, sans-serif; color: red; }
a:visited { font: 8pt/11pt verdana, arial, sans-serif; color: #4e4e4e; }
-->
</style>
</head>
<body>
<table cellpadding="3" cellspacing="5" id="container">
<tr>
<td><img src="http://www.mydomain.com/forum/image.php?type=dberror" alt="Database Error" width="48" height="48" /></td>
<td id="bodytitle" width="100%">Database error</td>
</tr>
<tr>
<td class="bodytext" colspan="2">The Domain - Forum database has encountered a problem.</td>
</tr>
<tr>
<td colspan="2"><hr /></td>
</tr>
<tr>
<td class="bodytext" colspan="2">
Please try the following:
<ul>
<li>Load the page again by clicking the <a href="#" onclick="window.location = window.location;">Refresh</a> button in your web browser.</li>
<li>Open the <a href="/">www.mydomain.com</a> home page, then try to open another page.</li>
<li>Click the <a href="javascript:history.back(1)">Back</a> button to try another link.</li>
</ul>
</td>
</tr>
<tr>
<td class="bodytext" colspan="2">The www.mydomain.com forum technical staff have been notified of the error, though you may <a href="mailto:admin@mydomain.com">contact them</a> if the problem persists.</td>
</tr>
<tr>
<td class="bodytext" colspan="2">&nbsp;<br />We apologise for any inconvenience.</td>
</tr>
</table>

<form><textarea rows="15" cols="70" wrap="off" id="message">Database error in vBulletin 3.6.8:

Invalid SQL:

SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle, post.visible AS postvisible,
thread.visible AS threadvisible
FROM post as post LEFT JOIN thread as thread
ON (thread.threadid = post.threadid
AND post.userid = thread.postuserid
AND post.postid = thread.firstpostid)
WHERE post.isusenetpost = 0
AND post.postid &gt;
AND thread.forumid = 27;

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND thread.forumid = 27' at line 11
Error Number : 1064
Date : Monday, December 10th 2007 @ 07:00:51 PM
Script : http://www.mydomain.com/forum/gateway.php?debug=1
Referrer :
IP Address : 000.000.000.000
Username : admin
Classname : vB_Database</textarea></form>

</body>
</html>

[b00k]

-----------

[dodgez]

Anyone know the fix to run it on 3.7?



Database error in vBulletin 3.7.0 Beta 3:

Invalid SQL:

INSERT INTO vb3_cronlog
(varname, dateline, description, type)
VALUES
('gateway14',
1198943338,
'Ran NNTP Gateway<br />',
0);

MySQL Error : Duplicate entry '64376' for key 1
Error Number : 1062

[Amenothep]

I have been reading through this thread and haven't seen an answer for this yet.



I'd check postcount .. :)

[AWS]

Anyone know the fix to run it on 3.7?



Database error in vBulletin 3.7.0 Beta 3:

Invalid SQL:

INSERT INTO vb3_cronlog
(varname, dateline, description, type)
VALUES
('gateway14',
1198943338,
'Ran NNTP Gateway<br />',
0);

MySQL Error : Duplicate entry '64376' for key 1
Error Number : 1062
Evidently something changed with the cron scripts in vbulletin. In your cron task uncheck the option to write to log.

[b00k]

-----------

[solokron]

Why would I receive the following:

<form><textarea rows="15" cols="70" wrap="off" id="message">Database error in vBulletin 3.6.8:

Invalid SQL:

SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle, post.visible AS postvisible,
thread.visible AS threadvisible
FROM post as post LEFT JOIN thread as thread
ON (thread.threadid = post.threadid
AND post.userid = thread.postuserid
AND post.postid = thread.firstpostid)
WHERE post.isusenetpost = 0
AND post.postid &gt;
AND thread.forumid = 27;

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND thread.forumid = 27' at line 11
Error Number : 1064
Date : Monday, December 10th 2007 @ 07:00:51 PM
Script : http://www.mydomain.com/forum/gateway.php?debug=1
Referrer :
IP Address : 000.000.000.000
Username : admin
Classname : vB_Database</textarea></form>

forumid = 27 is a valid id and is writable.

[KW802]

Why would I receive the following:

...
AND post.postid &gt;
AND thread.forumid = 27;
... If you check ACP => NNTP Gateway Options => NNTP Gateway Settings, is there a value for the "The max postid in post table when the script was last run." settings?

[mindfrost82]

I just installed this and so far it is running good, with the exception of 1 problem.

Certain messages will generate a "PHP has encountered a Stack overflow" error. The only way I can figure out to get around it is to go in and manually change the last downloaded message number so that it skips that message.

Any ideas on how to fix this? I'm running a Windows 2003 server. I have already tried increasing the memory limit in PHP and it doesn't seem to have helped.

[solokron]

It's empty.

If you check ACP => NNTP Gateway Options => NNTP Gateway Settings, is there a value for the "The max postid in post table when the script was last run." settings?

[KW802]

It's empty.Find the last post ID in your vBulletin and then insert that ID as the value.

[solokron]

Of course. Thank you. I have it working now.

Find the last post ID in your vBulletin and then insert that ID as the value.

[dodgez]

Evidently something changed with the cron scripts in vbulletin. In your cron task uncheck the option to write to log.


Still getting the errors

[AWS]

Don't know what the problem may be then.

I have it running here (http://extremetechsupport.com/forum) on 3.7 beta 3. I have modified it several times fixing things that change between vbulletin version so it possible my version has a fix that also fixes a problem with 3.7.

[dodgez]

Maybe it is because I use it for mailing lists?

[mindfrost82]

I have been reading through this thread and haven't seen an answer for this yet.

The hack has worked somewhat on vBulletin? Version 3.6.8

I ran it, it imported some threads from Giganews and it stopped working.

I re-ran it with /gateway.php?debug=1 and it just sits there. No threads are imported, no error message, nada. If i stop it and rerun it i get the usual:
Another instance of gateway.php is running, try again in 30 minutes if the script was crashed.

So, i launched it on my dev site and with a different newsgroup i ran into the same issue.

Anyhow, is there a way to resolve this ? I have increased my wait_timeout to 1200
Have you gotten this working at all? I just setup an account with Giganews to try it and I'm having the same issue. It hasn't imported ANY threads with Giganews, it just hangs. Sometimes I'll see:


Connecting to server, server says: 200 News.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com


...then it'll just hang. Then the next time I try running the script it just hangs and doesn't try pulling any posts, not even ones using the free Microsoft server.

Everything worked fine with a free server that doesn't require a login, my problem only started once I tried Giganews.

I can verify that my username/password works because I can login using Outlook Express just fine.

[AWS]

The connection is being established.
Try running setlastmsg.php and then running gateway.php after that. Wait a little while in between.

[mindfrost82]

The connection is being established.
Try running setlastmsg.php and then running gateway.php after that. Wait a little while in between.
Thanks, that seems to have worked!

[scottct1]

This works great for me on vBulletin 3.7.8 except for some reason new thread notification for those members who subscribe to new thread notification does not work. They are not sent an email that there has been a new post / thread.

Anyone know how I can get that to work?

[kermitklein]

I have installed and is running very well on my 3.6.8. My only problem is that the attach are not imported to threads... Anyone can help me?

[dermax]

Hello,

I face a problem with some groups. Randomly, the gateway.php run stops with the following message:


Connecting to server, server says: 200 Text.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com

Info for it.comp.macintosh at text.giganews.com: 211 678136 438957 1117092 it.comp.macintosh


In this case, the it.comp.macintosh newsgroup seems to be broken: I need to disable the group to make the script run again. I tried to import the messages from the group with a generic usenet client (GrabIt) and it worked absolutely flawless. Does anyone have an idea?

Thanks!

[nintendo]

If you ever get the

Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected

message via gateway.php?debug=1 it's a message from the NNTP-server...you're screwed...go look for another NNTP server. (For those of you in Rio Linda, they banned your server!!)

For a simple way to find out if you've been banned with a server, via SSH...

telnet SERVER 119

if you get...
Trying 202.177.16.121...
telnet: Unable to connect to remote host: Connection timed out


you're banned. If you get

telnet msnews.microsoft.com 119
Trying 207.46.248.16...
Connected to msnews.microsoft.com (207.46.248.16).
Escape character is '^]'.
200 NNTP Service 6.0.3790.1830 Version: 6.0.3790.1830 Posting Allowed

Guess what!!! You're not banned...atleast not *yet*!!! Quick, download a few million threads before they do ban you!!!

[kermitklein]

Still getting only this message:

Getting message number 339:
txt extensions are not accepted, as currently set up in the control panel.

I'm sure I have the permissions right... at least I think so... I can post this extensions using my username and password.

I'm trying to import a yahoo group at:

http://www.forexforums.org

Any ideas?

[squishi]

This mod sounds great. But I can't get it to work.
I am testing it with a free-to-use university usenet server.
I am sure the newsgroups I tried out exist. No typo or anything.

But I get this error:


Connecting to server, server says: 200 news.XXX.de InterNetNews NNRP server INN 2.3.5 ready (posting ok).

Info for alt.test at news.XXX.de: 411 No such group alt.test

[sdsvtdriver]

the news server does not have a group called 'alt.test'. pick one it does have

[squishi]

the news server does not have a group called 'alt.test'. pick one it does have

Negative. It DOES have this group! I can read it just fine in my newsreader.

I tried all kinds of combinations with this prefix that one can define in the settings, like prefix="alt.", group="test" etc.
But that did not work, either.

As I said, the group is there.

[tyler3]

Been going through this thread for awhile now and can't seem to figure this out...

I have the mod installed and setup.

I run 'gateway.php' and it just displays a blank page..

When I run 'setlastmsg.php' it gives the following error:

1 group(s) gatewayed. Could not connect to NNTP-server. ( (1)) Logging in to , group freetext.usenetserver.com: Not connectedNot connectedMax: Min: Not connected


I got the newsgroup from a post in this forum saying it was free to access, no login / pass require. I've also tried a couple of microsofts groups and the php.net group with no success (always gives me the same error as above). Anyone know what might cause this??

Thanks in advance!

[arkadia]

hello tyler

do you use microsoft or unix ?

[David_R]

Not connected
Not connected
Could not connect to NNTP-server. (php_network_getaddresses: getaddrinfo failed: Name or service not known (0))

We just moved our server and started getting this problem.

[TMM-TT]

telnet SERVER 119

if you get...

Trying 202.177.16.121...
telnet: Unable to connect to remote host: Connection timed out

you're banned


Umm.. No?

[HappyAndyK]

Hello,

I am trying to implement this under a b installation, running 3.7.1.

How ever I am stuck at the place where I am suppose to edit the admincp/index.php file as I am unable to locate the side piece of code,

construct_nav_option($vbphrase['vbulletin_options'], 'options.php?null=0', '<br />');

:(

can any one shade some light :)

thank you.

[bigtime]

Anyone try this on 3.7.2?

[HappyAndyK]

Anyone try this on 3.7.2?

It seems to work as one way communication as I see with my board running 3.7.2... It is importing data from NNTP however it is unable to send it, errors like,

[quote="error"]Result: 441 (629) Article Rejected -- Ill-formed message id '<User.Name.3bzrtn@nntp@mydomain.com>'</quote>

can any one help as how can the post posted in the forum can also be sent to the NNTP news group ??

[AWS]

Running on 2 3.7.2 installs with no problems.

[mrrijo]

Does this use Akismet moderation? I wanted to avoid spams getting downloaded from nntp servers.

Will this work along with Akismet module or any other spam blocking modules?

[t3nt3tion]

Unfortunately no. I`m still tweaking it for my own board, added alot of changes, and the akismet one is probably one that I will need to look into.

[mrrijo]

eagerly waiting....i hate spammers.

[onlinecc]

Wow, these are quite complicated installation instructions.. No possibility of using the otherwise nice "Upload Plugins" tool from the vbulletin adminCP? All this editing of index.php makes me uneasy.. Also, I don't see any mention of the /XML filder located in the includes folder - if it should be uploaded or not..

[t3nt3tion]

Well, you just upload files and that`s what. The xml you upload adds the menus required in admincp. What`s hard about that ? :)

[onlinecc]

Hehe, well if it's just that then... Just thought I saw some instructions telling me to edit code in files etc.. But ok, if it's just uploading files I'll stop being a cry-baby :)

[Rik Brown]

If you aren't already using it, I thought I'd mention that the new Automatic Thread Tagger (https://vborg.vbsupport.ru/showthread.php?t=185495) mod works great in conjunction with the NNTP Gateway mod and vb's tagging system.

The mod can go back and tag all of your prior Usenet messages plus automatically tag new incoming Usenet messages using vb's cron (not to mention RSS feeds as well).

Check it out as all your Usenet forums can now be auto tagged! -- Rik

[tpearl5]

I would be willing to pay to have this mod updated to work properly. (plus some additional features) Please contact me if interested.

[burmjohn]

Awesome mod. Thanks for those who created / modified this. I'd be interested in helping improve suggestions and whatever else is needed.

[sensimilla]

Awesome and still works for me on 3.7

[koon85]

Has anyone install this mod and it works fine with Google groups or Yahoo groups? I can't get it done :(!

[Simon Lloyd]

It seems to work as one way communication as I see with my board running 3.7.2... It is importing data from NNTP however it is unable to send it, errors like,

[quote="error"]Result: 441 (629) Article Rejected -- Ill-formed message id '<User.Name.3bzrtn@nntp@mydomain.com>'</quote>

can any one help as how can the post posted in the forum can also be sent to the NNTP news group ??
Yes i'm running 3.7.3 and get that message too!, i also need to know how to fix it.

[t3nt3tion]

Has anyone install this mod and it works fine with Google groups or Yahoo groups? I can't get it done :(!

This mod is intended to be used with newsgroups, not google or yahoo groups. Different things ....

[TMM-TT]

Result: 441 (629) Article Rejected -- Ill-formed message id '<User.Name.3bzrtn@nntp@mydomain.com>'The message-id is, at it says ill-formed. If I'm not completely wrong, you - that have problems with this - should look at the "Email address domain"-setting, in your configuration and skip the first part (the "@nntp"-thing).

[Simon Lloyd]

I dont get an error message when posting in the forum but when the cron job takes place none of the forum posts appear in the newsgroup (msnews.microsoft.com microsoft.public.excel.programming) do you know how to fix this?

[t3nt3tion]

This is an error everyone is waiting a fix for.

[TMM-TT]

I dont get an error message when posting in the forum but when the cron job takes place none of the forum posts appear in the newsgroup (msnews.microsoft.com microsoft.public.excel.programming) do you know how to fix this?

Exactly how does your configuration for microsoft.public.excel.programming looks like? Have you combined both the newsserver and the newsgroup on the same row?

[Simon Lloyd]

Server line: msnews.microsoft.com
and newsgroup/mailinglist email : microsoft.public.excel.programming nothing else!

[manuelsechi]

hi

can I supply a date and/or a number of messages to download a limited number of messages as start point? maybe modifing the setlastmessage.php file?

thank you

[TMM-TT]

Server line: msnews.microsoft.com
and newsgroup/mailinglist email : microsoft.public.excel.programming nothing else!

What happens if you run the script in debug-mode after a post?

hi

can I supply a date and/or a number of messages to download a limited number of messages as start point? maybe modifing the setlastmessage.php file?

thank you

If you can access the newsgroup with a newsreader, you can pick the "Xref"-header and set the startpoint by editing the "Last message"-setting in the config for vB. But I don't know if you can limit the number of messages to download. All messages after the startpoint you set will be downloaded, as I know it.

[Simon Lloyd]

What happens if you run the script in debug-mode after a post?
Tomas, thanks for taking the time to reply again, i have very little experience so could you tell me what you mean here?

[TMM-TT]

Tomas, thanks for taking the time to reply again, i have very little experience so could you tell me what you mean here?

Make a post and then use the gatewayscript like this - http://your-forum-url/gateway.php?debug=1 - and see what it says.

[keez]

when will the modification final developed ?

[Simon Lloyd]

Gateway version 2.3.2 17 group(s) gatewayed.

Connecting to server, server says: 200 NNTP Service 6.0.3790.1830 Version: 6.0.3790.1830 Posting Allowed

Info for microsoft.public.excel.programming at msnews.microsoft.com: 211 19098 984384 1003553 microsoft.public.excel.programming


Connecting to server, server says: 200 NNTP Service 6.0.3790.1830 Version: 6.0.3790.1830 Posting Allowed

Info for microsoft.public.excel.worksheet.functions at msnews.microsoft.com: 211 16719 516169 532928 microsoft.public.excel.worksheet.functionsThis is what it says for all of the ng's i have set to read, it says posting allowed but the post doesnt reach the newsgroup?

[Simon Lloyd]

Quick update on the debug, i ran it again and got this:
Gateway version 2.3.2 17 group(s) gatewayed.

Connecting to server, server says: 200 NNTP Service 6.0.3790.1830 Version: 6.0.3790.1830 Posting Allowed

Info for microsoft.public.excel.programming at msnews.microsoft.com: 211 19031 984528 1003628 microsoft.public.excel.programming

Can we post? server responded: 340 Continue posting - terminate with period
Posting Message 'Re: VBA code' from Simon Lloyd. Result: 441 (629) Article Rejected -- Ill-formed message id '<Simon.Lloyd.3hivaz@@thecodecage.com>' in field 'Message-ID:'

Connecting to server, server says: 200 NNTP Service 6.0.3790.1830 Version: 6.0.3790.1830 Posting Allowed

[Simon Lloyd]

I fixed the @@thecodecage.com but it still doesnt post to the newsgroups

[TMM-TT]

I fixed the @@thecodecage.com but it still doesnt post to the newsgroups
Do you get any errormessages or empty answers from the server? I tried to post a testmessage directly on the server and it took a little time before it was propagated.

[Simon Lloyd]

Do you get any errormessages or empty answers from the server? I tried to post a testmessage directly on the server and it took a little time before it was propagated.
The post just doesn't appear in the newsgroup!, i don't know how else to explain it.

[Simon Lloyd]

It seems i can post in the other microsoft newgroups from my forum but just not to microsoft.public.excel.programming any ideas on a work around?

[TMM-TT]

It seems i can post in the other microsoft newgroups from my forum but just not to microsoft.public.excel.programming any ideas on a work around?
What happens if you post with a newsreader? Does it work?

[Simon Lloyd]

What happens if you post with a newsreader? Does it work?
Yep i can post fine using outlook , its quite strange!

[Simon Lloyd]

Quick update: it seems that if i make a post in the forum whilst viewing the forum through firefox it posts to the microsoft newsgroup if i use IE it does not anyone know of how to tweak the code to accommodate both?

[cedivad]

Ehm.. question: does anybody get this working with the nntp server autentication?

Thank you very much! (and sorry for cross posting..)

[cedivad]

I have a serius problem: script post 2 time the same post in the same thread! What happens?

Thank you!!!

[cedivad]

I have a serius problem: script post 2 time the same post in the same thread! What happens?

Thank you!!!

Fixed. It was simple due the script run twice..

Does anybody have do a shell script to avoid this?
Thanks!

[Rik Brown]

Fixed. It was simple due the script run twice..

Does anybody have do a shell script to avoid this?
Thanks!

It should check whether or not its running via the is_running variable in the NNTP Settings of the control panel. However, I've had what you describe happen to me some years ago. What we have done since as a preventative measure is to do a quick restart of apache (just takes a second) and that makes sure any of its processes are killed.

-- Rik

[Rik Brown]

Via our killfile we have blocked all urls over the years to prevent spam. Since our killfile has grown so large and probably blocks most spam and trash anyway, we are considering removing the blockage of urls from it and then post-process posts via mysql to remove any "non-approved" urls. We're probably going to try this on one newsgroup initially.

Our idea is to tag approved messages with some sort of marker at the last byte of the post if the url is OK. If the marker is not there and a url exists, the post would later get deleted.

Is anyone doing something like this with the mod? If so, what are your thoughts on the the matter (or am I just opening up a floodgate of spam)?

Thanks. -- Rik

[Hemanth]

Willing to pay good money for this mod to be updated and optimized for vB 3.7.x & future versions.

If anybody interested please contact me.

[Rik Brown]

Willing to pay good money for this mod to be updated and optimized for vB 3.7.x & future versions.

If anybody interested please contact me.

Hemanth:

This looks like the best possibility at the moment: https://vborg.vbsupport.ru/showthread.php?t=184344 by Kerry-Anne. I hope that she is working on updating this mod. She always does a first-class job on her mods.

-- Rik

[Rik Brown]

Just thought I'd share this useful mysql query by Wayne Luke that I found on vbulletin.com that finds duplicate posts within each thread. You can then easily delete those dupes.

http://www.vbulletin.com/forum/showthread.php?t=285501

If you've ever had 2 instances of the NNTP importer running on the same newsgroups by mistake, you'll really benefit from this. But it works just as well finding dupe posts inadvertently posted by your own vb users or from Usenet users.

The first time I ran it yesterday, I found thousands of dupe messages that have built up over the years connecting to Usenet. Now that they are all cleaned out, I'm going to run it on an regular basis before each build of our sitemaps.

I hope that you find it useful in conjunction with this mod. -- Rik

[cedivad]

Thank you for the link, really usefull!

[LPH2004]

I've had this script working for a long time, changed the php version in December and it stopped. I finally figured out the time out was not set correctly in my.cnf and was able to get things rocking again.

Until today.

Today, the hosting company had to move my site to another box. The linux box the site was on was hacked. No problem. Except there has been nothing but problems with this new box.

I cannot get the script to run in the browser. I can, however, get the script to run at the CL when I ssh to the box. I can run #php /path/to/gateway.php and all is fine.

However, the scheduled task craps out as well as gateway.php?debug=1 in the browser.

Does anyone have any suggestions as to where to look?

[Simon Lloyd]

Maybe you have to run the setlastmessage.php again?

[LPH2004]

Maybe you have to run the setlastmessage.php again?

Thank you for the suggestion but the gateway script runs at the CL as root but not in the browser. After changing some of the messages in nntp.php file, I see fsockopen fails in the browser but works at the CL. However, I am unsure how to fix the permissions to get this to work.

Update: The above statement is probably not accurate. The fsockopen fails for the nntp.php script but works for a little test script I wrote:

http://technologyquestions.com/technology/testfp.php

Update: After 5 days of the hosting company insisting something was wrong the script - they found the problem. SELinux was blocking the port. The script now operates as it did for years ;)

[Simon Lloyd]

Glad you got your's going!, i still get db errors since the php upgrade and can't seem to get anyone to sort it! :(

[Rik Brown]

Has anyone made modifications to the Gateway so that embedded YouTube videos (specifically the AME Media Embedding mod code) can be converted to normal URLs to export on Usenet?

I assume it just requires simple additions to the array in the includes/functions_nntp.php file. But I'm not so sure how to accomplish it?

If anyone has done so, could you please share the code?

Thanks. -- Rik

[dicot]

Exactly what does the ISP need to do to get rid of the UTF-8 garbage in the user name? Edit the php.ini, install something? They say i need to edit my php file... when others asked y'all sat they need to do something.

[t3nt3tion]

Anyone have a list of killfile strings to stop spam and adult posts getting in ?

[dicot]

Exactly what does the ISP need to do to get rid of the UTF-8 garbage in the user name? Edit the php.ini, install something? They say i need to edit my php file... when others asked y'all sat they need to do something.

Finally figured it out - IMAP extension needed to be installed in PHP. In my testing, i reran setup to install that extension.

[toibs]

Aww rats!!

Have been running this on an older server of mine for the last year or so, with no problems at all. Only using the listing tho, not posting to the groups in question.

However preparing a new server, and setting this up again. Have installed the files etc etc no problems. However when i run http://www.xxx.com/gateway.php?debug=1 on the new system, i get a complete blank page. If i run setmessage.php on the new server, this connects and runs no problems.

It seems to be configured exactly the same as the old server. The only differences i can guess are that this runs on a newer version of centos/php, however i cant see that actually stopping this running.

anyone any advice on things to try??

Thanks
Paul

[toibs]

Update - All seems to be working now. However i seem to be getting double posts - not on the headers, however on the content!! Any advice on how i can either remove them or preferably not get them to start with??

Thanks
Paul

[Rik Brown]

Update - All seems to be working now. However i seem to be getting double posts - not on the headers, however on the content!! Any advice on how i can either remove them or preferably not get them to start with??

See post #643 of this same thread for a mysql query by Wayne Luke:

https://vborg.vbsupport.ru/showpost.php?p=1687449&postcount=643

I hope that helps. I use it often.

Regards. -- Rik

[TMM-TT]

Update - All seems to be working now. However i seem to be getting double posts - not on the headers, however on the content!! Any advice on how i can either remove them or preferably not get them to start with??

Thanks
Paul

Do you mean ALL POSTS or only the posts that are being sent to usenet?

The most common reason of dupes (those of being sent TO usenet) is the fact that something is misconfigured. Like the posting organization for example, in the sent message.

[Ricsca]

For vbulletin 3.8.x ????

Thanks

[Simon Lloyd]

For vbulletin 3.8.x ????

ThanksIt works fine "out of the box" for 3.8.x however there is some deprecated code in the scripts.

[toibs]

ahh - maybe a problem.

Just changed news hosts, and it seems like this doesnt want to retrieve posts any more.

This is what i get...


Gateway version 2.3.2 1 group(s) gatewayed.

Connecting to server, server says: 201 unlimited.newshosting.com (fe04.ams) NNRP Service Ready - support@newshosting.com (no posting)

Server responded after user name: 381 PASS required

Server responded after password: 281 Ok

Info for rec.games at news.ams.newshosting.com: 211 279425 1239508 1518932 rec.games


I dont spose something silly like the "last message" number will be different on different hosts would it?? Anything anyone could suggest?

Have verified this on two different boxes & installations...

thx
Paul

[TMM-TT]

ahh - maybe a problem.

Just changed news hosts, and it seems like this doesnt want to retrieve posts any more.

This is what i get...


Gateway version 2.3.2 1 group(s) gatewayed.

Connecting to server, server says: 201 unlimited.newshosting.com (fe04.ams) NNRP Service Ready - support@newshosting.com (no posting)

Server responded after user name: 381 PASS required

Server responded after password: 281 Ok

Info for rec.games at news.ams.newshosting.com: 211 279425 1239508 1518932 rec.games
I dont spose something silly like the "last message" number will be different on different hosts would it?? Anything anyone could suggest?

Have verified this on two different boxes & installations...

thx
Paul

Did you change to a totally new server, you have to check the last-message (or reset it). If you want to do it the hard way, look at the forum what post that was the last. Then open a regular newsreader for the new server, find that post and get Xref from the header. That's the id of the last message. If it's a crosspost, you should use the id for the group that you are importing. Then put that id into the settings for the group you want to continue fetch posts from.

[silverslik1]

Has anyone gotten this pluggin to work where it actually allows users to send replies back to the newsgroup?

I cant seem to get that to work? Should I try a reinstall. Much appreciated if someone can help me get this to work

[Simon Lloyd]

It works out of the box for me on 3.8.3 (although i paid someone to change some deprocated code).....search these forums for another version of this that has in brackets (mailing lists) i think!, don't forget that port 119 needs to be open on your server!

[toibs]

Oki,

Any highlighted in "blue" text is a quote from one of the previous messages in the thread. Has anyone any idea how to automatically remove this?

Thx
Paul

[Simon Lloyd]

i think its in the mime or pop files!

[hmspain]

I have everything working except for attachments coming from NNTP to the vBulletin system. Somehow, my installation is missing the /includes/functions_image.php file :-?.

Could someone share the file with me or point me in the right direction? I thought that it might have be renamed to class_image.php but I don't think so :-).

Please help!

Harrison Spain
spain@ugs.com

[Rik Brown]

Once anyone using this mod gets hold of the vBulletin 4.0 beta, it would be great to hear whether or not it works with the mod or if any revisions will be required.

I'm keeping my fingers crossed that since the mod works at the backend level, it should work with 4.0 with little or no changes.

Cheers! -- Rik

[Nacman]

How many of us running this MOD still have the problem of only the first attachment from a LOCAL post going to usenet showing up on usenet as a link.

My example - I use this MOD to post to a TEXT only usenet group, so any attachments my forum users attach in the forums get sent up as links to the usenet posting. If it is only one attachment all is well, but if it is 2 or more only 1 gets made into a link.

Anyone?

[Rik Brown]

I've started trying to move this mod to a brand new vb 4.0 beta installation.

First thing I've found is that I had to create the left-side menu options in the admincp by creating a file named cpnav_nntp_gateway.xml in the root/includes/xml directory with the following code:

<?xml version="1.0" encoding="ISO-8859-1"?>

<navgroups product="vbulletin">

<navgroup text="NNTP Gateway Options" hr="true">
<navoption>
<text>NNTP Gateway Settings</text>
<link>nntp_settings.php</link>
</navoption>
<navoption>
<text>NNTP Gateway Newsgroups</text>
<link>nntp_groups.php</link>
</navoption>
</navgroup>
</navgroups>

The permissions on the xml file should be 755. When you reload the root/admincp/index.php file, the menu options created above should appear.

Now, I haven't gone any further than this so far so I'm afraid I can't be of further assistance until I've taken a few more steps.

-- Rik

EDIT: Well, I see the ZIP file includes a similar XML file as I made above so you probably should use that.

[Nacman]

I've started trying to move this mod to a brand new vb 4.0 beta installation.

First thing I've found is that I had to create the left-side menu options in the admincp by creating a file named cpnav_nntp_gateway.xml in the root/includes/xml directory with the following code:

Now, I haven't gone any further than this so far so I'm afraid I can't be of further assistance until I've taken a few more steps.

-- Rik

Thanks for picking up the ball. I have incorporated many of the mods to this, and even done a few of my own. I am a just above a noob with php, but learning more and more each day. If I can help I will.
Nacman

[alis]

The script run in vb 4.0 beta 4, but with the Automatic Similar Thread Search activated i get this error:
Fatal error: Call to undefined function fetch_similar_threads() in gateway.php on line 547

Here below the line 547 or gateway.php

require_once(MY_DIR . '/includes/functions_search.php');
$similarthreads = fetch_similar_threads($subject);


Function fetch_similar_threads isn't included or has changed in the 4.0 vb version of /includes/functions_search.php

Please can someone tell how to correct this error?

[Rik Brown]

Thanks for picking up the ball. I have incorporated many of the mods to this, and even done a few of my own. I am a just above a noob with php, but learning more and more each day. If I can help I will.
Nacman

So far, I've gotten the script (without changes) to connect to the gateway server and try to import messages. If the messages are killfiled, it bypasses them as expected (as seen in debug mode). But once it comes to the first message it should truly import, it stops. I haven't figured out why.

I hope someone with more php experience than me has some ideas. -- Rik

[TMM-TT]

I forgot something in this post.

I'll be back with that...

[TMM-TT]

This is the story:

My newsservers motherboard died in november, so I had to order a new one. This server has now been reinstalled from scratch so the Xref, which gateway.php uses is reset. Lastmsg in the nntp_groups-table therefore got useless, especially when the server resynched older posts again.

Fetching all posts again after resetting the lastmsg will result in duplicate messages. This has been fixed with a small patch.


Find the row:
$message =& $nntp['message'];

Add this code after that row:


/*
* Find duplicates by message-id against the forumid!
*
* Matching against forumid, so crosspostings to other groups
* can proceed to posting.
*
*/
$isduplicate = false;
$msgid = $message['message-id'];
$msgdupe = $db->query_first("
SELECT p.postid,p.threadid,p.msgid,t.forumid FROM post AS p
INNER JOIN thread AS t
ON p.threadid = t.threadid
WHERE msgid = '".$db->escape_string($msgid)."'
");
if ($group['forum'] == $msgdupe['forumid'])
{
$isduplicate = true;
logging("Skip, Message-ID already found.");
// No continue should be used here, since the lastmsg-counter must work
//continue;
}


Then go find the row for the post-insert (it?s actually the thread-insert), it should look like this:

$postid = insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $date, $parentid);

Before that line, add?


// If this isn't a duplicate
if (!$isduplicate)
{




Then, after the line ?
exec_send_notification($threadid, "0", $postid);

Add this:

} // Isduplicate Finish Right after this, you have to add similar code to the insertcode for posts, so directly after

} else {


?add this again:


// If this isn't a duplicate
if (!$isduplicate)
{


?and finish this with an ending ?}? after the line

logging("'$subject' from ". $from_name . ". New thread.");


Now, the gateway should?nt post a message with the same Message-ID twice?

[Rik Brown]

The script run in vb 4.0 beta 4, but with the Automatic Similar Thread Search activated i get this error:
Fatal error: Call to undefined function fetch_similar_threads() in gateway.php on line 547

Sorry that I can't help with a Gateway code fix, but your message solved my importing problem mentioned above (message #671) by letting me know to turn off the similar thread search:
admincp -> Message Search Options

Now the Gateway is importing threads on a 4.0 installation. Thanks. -- Rik

EDIT: You can still display the similar threads:
admincp -> Thread Display Options (showthread) -> Show Similar Threads = Yes.

Since similar threads aren't being updated during the import, I expect (unless they are somehow updated as one of the scheduled tasks) one has to update them regularly with:
admincp -> Maintenance -> Updated Counters -> Rebuild Similar Threads

[someguyonline]

Anyone gotten this to work on 4.0 gold?

[toibs]

Guys,

Would just like to thank you for spending time getting this mod to work on V4. I havent installed yet, however i'm just setting up a test forum based under 4.0.2.

Just wondering if one of you guys might be able to assist by passing on current working files, with any patches added to allow it to work (esp under v4).

I could trawl through the thread and apply patches, however there's so many "try this" - then - "This doesnt work, this does it better" that i think i may be here for days/weeks just taking the original files and altering code! It's like re-inventing the wheel!!

If anyone could assist, it would be much appreciated :)

On a side note, i would be happy to contribute alongside others, to a developer polishing this up or rebuilding it to work under 4.0.2 nicely, as it did back in the early days!!

Paul

[scottct1]

This mod is the only thing holding me back from going to 4.0.

I have a few services which email me information and this is the only hack I know which can check mailboxes and if new messages are found post them to the forum as new threads.

If anyone know a program which does the same thing let me know. I dont really use the usenet portion of the program.

[Eric]

I've been messing around with this, and If I can get it cleaned up and working Ok (and can somehow get ahold of the developer, to request permission to take over) I'll see about releasing a 3.8 (and 4.0) version. That's IF I can get it worked out, and permission from the developer.

[toibs]

I've been messing around with this, and If I can get it cleaned up and working Ok (and can somehow get ahold of the developer, to request permission to take over) I'll see about releasing a 3.8 (and 4.0) version. That's IF I can get it worked out, and permission from the developer.

SecondV,

I posted this yesterday in Paid Mods.

https://vborg.vbsupport.ru/showthread.php?t=236324

Not sure, however i suspect there's a fair few people on the list that would contribute maybe $20 each to get a decent supported working version. I know i would!!

It's the one thing that (like others) is stopping my forum migrating!!!

Paul

[someguyonline]

id drop at least 60 bucks to see this happen on 4.0

[toibs]

to be fair i'd go to $50 (Was thinking of others that might not be willing to pay as much.

So there ya go SecondV, thats at least $110 before even thinking about it!! if that isnt enticement, i dont know what is :D

Paul

[TMM-TT]

It actually works for vb4. I've tested it yesterday. I haven't tested postfunctions yet, but at least importing new posts/threads worked just fine.

Adding the code below to gateway.php (with your own HTTP_HOST-setting) and the script will pass the evil "bug" found in v4.0 (that I'm unable to describe right now, but this bug was about parsing urls). I added the code just after the "headers-sent" rows...


// for vb4.0
if (!$_SERVER[HTTP_HOST]) {$_SERVER['HTTP_HOST'] = "forum.tornevall.net";}

[TMM-TT]

Here's a patchfile (in the bottom of the post) with all changes I've done, diffed with the current attached zip in this thread!

It supports vb 4.0 (and there's some functions to run from console, like debug and "release the script from a earlier lock). It also looks for duplicate posts. This means if the post has already been posted from usenet into the forum, the post will not get duplicated in case of changes in the xref-header (also know as last message in admincp). If I don't remember wrong, the patch also fixes the last-post-link in the threadlist. The old file-attachment-issue is included, etc etc.

And of course: The handshake part has been fixed. I've also been playing with the signatures in this patch by the way, since I did'nt like the footer...

You may want to take a look. If someone have more handy functions in their plugin, feel free to contact me. Collecting everything in the same place would be nice. :)

http://tmm.tornevall.net/php/nntp_tornevall_100312.diff

Maybe I should write a readme or something?

[toibs]

TMM-TT,

Many thanks - What do you use to apply the Patchfile??

Would love to apply this and see if i can get it fully functional!!

Paul

[d8tabyte]

is there any way to get the users from the newsgroup posts to count as registered users and have the forums generate a new account for each user with a null password value or something? just so i can keep track of the newsgroup users?

[toibs]

TMM-TT,

Many thanks - What do you use to apply the Patchfile??

Would love to apply this and see if i can get it fully functional!!

Paul

Right. Have tried with both "patch" and "tortoiseSVN" and cant get it to patch!!!

Any pointers would be good!!

Paul

[TMM-TT]

It usually is patch -p0 or something like that you use (the patch is recursive, so it's not just gateway.php that gets patched), but if that's not working, I will put up a complete archive too.

[toibs]

It usually is patch -p0 or something like that you use (the patch is recursive, so it's not just gateway.php that gets patched), but if that's not working, I will put up a complete archive too.

Hiya,

Nope tried on the server and it just sits there and does nothing. Checked and double checked path in the .diff file, and it doesnt want to play :(

if you could post the archive that would be great.

Just wanted to add, Many thanks for breathing new life into this Mod. There is nothing out there which will allow the same functionality, so thanks :)

Paul

[TMM-TT]

I tried to post this yesterday but the posting function was totally down...

But here it is, as promised:

http://tmm.tornevall.net/blog/2010/03/17/nntpgw_100312/

The source is on top of the blog, then there's a small changelog.
Questions and wishes will be answered if possible!

[jarosciak]

This is working nicely for USENET groups, but is there any way to import Google Groups?

[TMM-TT]

This is working nicely for USENET groups, but is there any way to import Google Groups?
Set up a mailsubscription for them and import them as regular mail :)

[toibs]

Would just like to second this - seems to work really well - thanks TMM-TT for your help in getting it up and working again in Vb4. I shall donate something as a way of saying thanks (when i get paid later in the week!) for breathing new life into this!!

As an add-on (as D8tabyte had previously mentioned) is there any way of adding the users that are "created" as guests in this process to maybe a new "group" so they are separate from the rest of the members?? This way, if anyone wanted to join the forum from the group that was downloaded, the option would be there just to change the group that the person was in, and also it would effectively "reserve" the username.

The only downside would be if the "username" already existed, in which case maybe a 1/2/3 etc could be added to the next username.

Just a thought. Obviously not essential to do this, just a nice to have :)

Finally. and maybe the biggest want i would have, is there any way to import yahoo groups to this? it would have to scrape the web page and d/l the text from the messages i guess, however not being a coder i dont know how possible this would be!

Many thanks again
Paul

[TMM-TT]

I've just discovered a bug: Attachments seem to not work properly...

[hmspain]

Me too :-).

I'm having problems with attachments not coming over from NNTP/DNEWS to vbulletin via the NNTP gateway properly. Suggestions would be very welcome!

[toibs]

I'm seeing an interesting problem. When i download files everything goes fine, however it gets so far and then stops.....


'Re: Adding service outlet to re-import' from Gregory J Gabborini. Thread found by References.
Getting message number 758459:
'Re: finder's fee philosophy' from frog. Thread found by References.
Getting message number 758460:
'Re: Who sells Pop Bumpers?' from Johnspinballnutz. Thread found by References.
Getting message number 758461:
'United's Blazer' from pinmass. New thread.


all stops. Have checked the original post and there's nothing encoded or unusual - just text. No attachments (tho i'm about to retry with attachments "disabled" to see if there's a difference!)....

I've skipped forward a hundred of messages, nothing, just the header. If i move back it works fine, (obviously comes up with "duplicate file" and ignores), however makes me wonder if there's some kind of limit in the code??!?!

I have created another forum, and tried importing to this one, and the same happens!!!

Anyone got any clues?!?!?

Thx
Paul

[TMM-TT]

A solution for attachments in vB4:

http://tmm.tornevall.net/dev/functions_nntp_vb4_100326_2217.tar

Details: http://tmm.tornevall.net/blog/2010/03/26/nntp-gateway-modification-vb4-compatibility-fix/

Source is included in the post. And here's the attachment-test made out of the code, from de.alt.dateien.misc:

http://forum.tornevall.net/showthread.php?343942-Eislutscher-am-Wiener-Heldenplatz

[KW802]

Speaking of attachments... any work-around for multi-part binaries?

[James T Brock]

This works out of the box on vbulletin 3.8 or do I have to install some additional files/hacks to get it to work?

[tpearl5]

Anyone planning a version of this for vb4?

[AWS]

With the fixes in this thread it works with vbulletin 4. I have it running on my test server on 4.0.3. Everything works fine with the exception of similar threads which I have a fix for. I just have to check and make sure it works. Once it does I'll post that in the thread and then this will be working 100% with 4.0.x.

I would expect this to work until the admincp is revamped. I don't expect the db layer to change much if any so the core should continue to work. If it does it would probably be minor changes to this script to fix things.

[trackpads]

Aws, any luck? Thanks!

-jason

[AWS]

I haven't tested it yet and probably will not now. I decided that I wasn't going to upgrade to vbulletin 4 on the site I use this on and instead have started on an IPB usenet plugin for when I convert the site.

[offline]

Hello,

I am trying to implement this under a b installation, running 3.7.1.

How ever I am stuck at the place where I am suppose to edit the admincp/index.php file as I am unable to locate the side piece of code,

construct_nav_option($vbphrase['vbulletin_options'], 'options.php?null=0', '<br />');

:(

can any one shade some light :)

thank you.

3.8.5.....


construct_nav_option($navoption['text'], $navoption['link']);
}
}

// have all the options, so do the group
construct_nav_group($navgroup_holder['group']['text'], $navgroup_holder['group']['nav_file']);

if ($navgroup_holder['group']['hr'] == 'true')
{
construct_nav_spacer();
}
}
}
construct_nav_option('NNTP Gateway Settings', 'nntp_settings.php', '<br />');
construct_nav_option('NNTP Gateway Newsgroups', 'nntp_groups.php', '<br />');

[toibs]

I haven't tested it yet and probably will not now. I decided that I wasn't going to upgrade to vbulletin 4 on the site I use this on and instead have started on an IPB usenet plugin for when I convert the site.

I may well be interested in this, since i'm seriously contemplating moving the whole thing lock stock to IPB - i would miss a couple of mods i have, however it is far outweighed by the costs and improvements i would get rather than Vb's new pricing structures...

Ta
Paul

[tms1791]

TMM-TT, you are my HERO.

I'm simply looking to archive some mailing lists, and thanks to your efforts, I was able to get this going (so far) with a minimum of bashing-head-on-desk.

I haven't given it an extensive testing yet, but I did notice a couple of issues (vb4).

1. If "show similar threads" (or whatever the option name in vb is) is turned on, gateway.php gronks. Apparently, the function being called to find that list (fetch_similar_threads()) doesn't exist in vb4.

2. There's a couple of TABLE_PREFIX's missing, in the query that starts on line 342.

Other than that, so far, so good. #2 was an easy fix, but I don't know about #1. I'd really like that feature, but can live without it if need be.

Thanks again!

[shempsall2009]

Hi all I have installed this on vb4... also patched it and followed the quick instructions above..

I have ran setlastmsg.php and deleted it but nothing seems to be happening... Please could someone point me in the right direction... Thanks

[AtomicAcidbath]

Fix for 'Show Similar Threads' problem in 4.x

File: gateway.php

At line 586, Find:

if ($vbulletin->options['similarthreadsearch'])
{
require_once(MY_DIR . '/includes/functions_search.php');
$similarthreads = fetch_similar_threads($subject);
}


Replace with:

if ($vbulletin->options['similarthreadsearch'])
{
require_once('./global.php');
require_once(DIR . '/vb/search/core.php');
$searchcontroller = vB_Search_Core::get_instance()->get_search_controller();
$stl = $searchcontroller->get_similar_threads($subject);
$similarthreads = implode(',', $stl);
}


Rename all occurrences of variable $foruminfo something else like $forum_info. It's in there 6 times.

[scottct1]

I just moved to vb4 and this mod is working but I am having a strange issue...

When it imports email it adds a ? to the first line of a post.

I can't figure out where this ? is coming from....

[Aylwin]

First of all, big thanks to TMM-TT for updating this plug-in for vB4! I'm a long time user but only just recently upgraded to version 4.

I thought I'd contributed a modification I've made that checks if the email sender is also a forum user. If yes, then the post is added as that user. The reason I've done this is because I want our mailing lists to be fully integrated with our forums. Anyway, it's implemented as an option so you can choose whether or not to enable it.

I've also fixed the TABLE_PREFIX issue pointed out by tms1791 and added the 'Show Similar Threads' fix shared by AtomicAcidbath.

Additionally, the admincp files (nntp_groups.php and nntp_settings.php) were giving me SQL errors so I had to change "TYPE=MyISAM" to "ENGINE=MyISAM".

By the way, I think someone should create a new thread for this plugin in the 4.0 forum. That way, we can focus on 4.0 discussions and perhaps further development over there.

DISCLAIMER: I'm not really a proper coder so I can't guarantee that my version works. All I know is that it seems to work on my site.

[CRDeveloper]

I didn't know this was possible, one of my members suggested yesterday haha.

I will try ASAP, running 4.1.0 Beta!

[kgroneman]

I'm running it on 4.0.7 currently and it works great except for attachments. Any time it comes across an attachment it hangs. Is anyone else seeing this?

[scottct1]

I don't know what happened. This has been working great for a long time. But yesterday I started getting SQL error when gateway.php runs.

Database error in vBulletin 4.1.0:

Invalid SQL:

SELECT filename,attachmentid
FROM attachment
WHERE postid= 2424829;

MySQL Error : Unknown column 'postid' in 'where clause'

Any ideas?

[SamirDarji]

omg, I've been looking for something like this for almost half a decade! I can't believe I didn't see this before. I just hope it works on v3.8.x.

[hmspain]

I'm running this nntp sync and find myself having to choose between turning on thumbnails in vBulletin, or sync'g attachments.

I get the impression I'm not the only one.

Has anyone found a solution?

[hmspain]

I'm running this nntp sync and find myself having to choose between turning on thumbnails in vBulletin, or sync'g attachments.

I get the impression I'm not the only one.

Has anyone found a solution?

I'm running vBulletin 3.8.6.

[John Diver]

Is there a free server I can use to test this? I have tried 4 or 5 now and none are working.

Thanks

[Rik Brown]

Since no one has come back with a free server suggestion, I might suggest a 14-day free trial with Giganews.com.

-- Rik

ps: used them for years!

[Rik Brown]

After upgrading from vb 4.1.6 to 4.1.7, I got the following error:

PHP Fatal error: Call to undefined function build_post_index() in /home/www/html/subs/forums/includes/functions_nntp.php on line 623

What I found is that in the forum's /include directory there is a file named functions_databuild.php that holds the build_post_index function. I see in the 4.1.6 code that the build_post_index function has all the code between the initiation of the function and the end of the function commented out. In 4.1.7, the function has been totally removed from the functions_databuild.php file.

Since the mod worked fine with vb 4.1.6, this would indicate that the function wasn't being utilized by this mod at all. So I commented out the call to the build_post_index function from within functions_nntp.php. And, as far as I can tell, that gets the mod back up and running again with vb 4.1.7.

[photoads]

Anyone got this still working? I get a connection timeout everytime I try to run it any help appreciated

[hmspain]

Anyone got this still working? I get a connection timeout everytime I try to run it any help appreciated

It is working for me, but with limitations. Don't expect it to be perfect :-). The connection timeout sounds like a firewall issue.

Your vBulletin server needs to talk to your news server via port 119. You should be able to test this from your vBulletin server using a command shell window and:

telnet <news server) 119

I'm responding since most of us running this add-on depend on it a great deal and yet posts to this thread are far too often unanswered. I'll answer what I can. I use and depend on this add-on every day to keep my vBulletin forums and DNEWS server in "sync".

[kgroneman]

We've got it working very nicely, but have had to modify the code a bit to get it working right. If anyone wants it, I'll send you the files I have. Currently running on 4.1.9 PL1 without problems at http://forums.novell.com and http://forums.opensuse.org (nntp servers: forums.novell.com and forums.opensuse.org) Email me at kgronemanREMOVE@REMOVEnovell.com (sentient beings will know what to take out of that email address to make it work) We're still playing trying to get the NNTP cancel command to work, but we'll figure it out. Cancel was working but with one of the recent forum upgrades, it stopped so we'll have to find out what changed.

[TMM-TT]

After upgrading from vb 4.1.6 to 4.1.7, I got the following error:

PHP Fatal error: Call to undefined function build_post_index() in /home/www/html/subs/forums/includes/functions_nntp.php on line 623

What I found is that in the forum's /include directory there is a file named functions_databuild.php that holds the build_post_index function. I see in the 4.1.6 code that the build_post_index function has all the code between the initiation of the function and the end of the function commented out. In 4.1.7, the function has been totally removed from the functions_databuild.php file.

Since the mod worked fine with vb 4.1.6, this would indicate that the function wasn't being utilized by this mod at all. So I commented out the call to the build_post_index function from within functions_nntp.php. And, as far as I can tell, that gets the mod back up and running again with vb 4.1.7.

I'd rather patch that one, for "compatibility":


// Index post for searching
if (function_exists('build_post_index')) build_post_index($postid, $foruminfo);

[toibs]

Strange,
For some reason cant seem to re-install this on 3.8.5 - i know ive had this working on this version before...

Anyhow, getting the following :


Invalid SQL:

INSERT INTO nntp_groups
(newsgroup, forum, prefix, lastmsg, server, username, password, sender, checkuser, enabled)
VALUES
('ukpinball', 4, '[ukpinball]',
0, 'text1', 'text2',
'pass1', '', 0, 1);

MySQL Error : Unknown column 'checkuser' in 'field list'
Error Number : 1054
Request Date : Thursday, April 5th 2012 @ 05:42:03 PM
Error Date : Thursday, April 5th 2012 @ 05:42:03 PM
Script : http://www.name.com/community/admincp/nntp_groups.php?do=insert
Referrer : http://www.name.com/community/admincp/nntp_groups.php?do=add
IP Address : 82.69.165.76
Username : Paul
Classname : vB_Database
MySQL Version : 5.1.60-rel13.1


Erm ... Help?!?!!

Cheers
Paul

[Rik Brown]

Planning to change Usenet providers. Any recommendation for a good/inexpensive provider that works well with this mod?

Thanks. -- Rik

[TMM-TT]

Planning to change Usenet providers. Any recommendation for a good/inexpensive provider that works well with this mod?

Thanks. -- Rik

All providers theoretically works, but many of them only allows one user per account. The problem is to find which provider that allows multiple users to send posts via their services.

news.tornevall.net is unfortunately down for the moment (I'm working on it) - that service allowed this.

[donholloway]

I am using Usenetserver.com and they seem to be pretty responsive. They have a 24x7 online chat, also respond to email.

Thanks to everyone here for contributing, it took me much longer than I would have liked to get things working again. I had used the mod before, had mail servers working, but was struggling with Usenet.

My shared hosting provider had blocked port 119, found that they and my news provider were both supporting port 25, then spent many hours trying to find the port references in the php files. I found one in gateway.php, one in nntp.php, and setlastmsp.php. Still couldn't connect and was getting a timeout and it wouldn't connect.

Finally overnight, my ISP relented and unblocked port 119. I went to Tornevall's files that he posted on his webpage a couple of years ago (Tack för utstationering på engelska), made one edit to change for my host and it is now connecting and working.

I have that nice rush that you briefly have when you finally solve a tough one.

It was brief enough, the test post that downloaded was yEnc encoded and looks like gibberish. On to the next challenge. Thanks to everyone for supporting this. I will try to help improve the documentation a little bit.

Will try to share my experiences as I encounter the next challenge.

[toibs]

Donholloway,

Any chance you could share your changes in total? I'm about to move back to vB and would love to be able to re-install this again with minimal pain :)

or that applies to anyone - would anyone be willing to share their files with all the edits and changes rather than go through it all again?

Cheers
Paul

[Rik Brown]

Would love to hear feedback on whether the mod works with vb5 as soon as someone has tested it.

[toibs]

I will as soon as they get their fingers out and make it easy to upgrade from 3.x to 5... :( Cant do it online... and i havent had chance to call them...

[tpearl5]

FYI, Mine was getting an error when there was an apostrophe in the e-mail.

I added

//remove apostrophe from email
$from_email = preg_replace('/\'/', '', $from_email);


before

//Check if the sender is a user

which simply strips an apostrophe if present

[tpearl5]

Also, since I can't get the attachment stuff to work anyway (it's looking for the old functions_images.php vbulletin file) I just disabled attachments all together by commenting out the first call to process_attachments($date, $postid, $threadid, $forumid); in functions_nntp.php

Most of the files posted on newsgroups is junk/viruses anyway.

[SamirDarji]

Awesome notes on the install tpearl5. Thank you! I've yet to fully play with my install, and I'm sure these notes will come in handy. :)

[tpearl5]

Welcome - fyi, I was using the file provided by Aylwin above (https://vborg.vbsupport.ru/showpost.php?p=2122039&postcount=709). More thanks should go to Aylwin and others for providing various fixes.

[tpearl5]

Also, on another install of this I was getting a mySQL error when running gateway.php. I had to update the query in that matches posts to existing threads to this:
WHERE p.msgid = '".$db->escape_string($msgid)."'

note the p. in front of msgid.

[Hossam Hossny]

Can anyone here post his working version? There are many great contributors to this Mod but no one has actually posted a patched working version. I wonder why.

Thanks for everyone who has supported this mod that I know of since 2005. Back then I had it on 3.x but I wish I can know if it would work on latest 4.2.

Cheers,
Hossam