NNTP Gateway for Usenet ( Newsgroups ), Mailing Lists

[toibs]

thats strange - i didnt have that problem at all... luckilly enough it worked "straight out the box" (Tho still have the above mentioned problems with email.... :( )

[dkendall]

thats strange - i didnt have that problem at all...

It may only show up if the first post in the newsgroup has index 1. Since old posts can get archived/removed, it's possible your newgroups started with a post higher than 1.

Also, the problem (mentioned long ago) with the gateway spewing HTML when used with FireFox and the scheduled task manager is caused by this:

if (!headers_sent())
{
header("Content-Type: text/plain");
}
... around line 35 of gateway.php. Just take (or comment) it out, and the problem goes away.

David

[dkendall]

Another issue I've run into...

If an NNTP user has not provided a display name (at least in Outlook Express), the "From" field in his posts is "<name@domain.tld>" and the from_name function in functions_nntp.php returns an empty string.

My workaround is to add this:
if(!$from_name)
$from_name = from_email($from_raw);
Just before this:
return $from_name;

Then I made the email obfusication mods described here:

https://vborg.vbsupport.ru/showpost.php?p=899284&postcount=209

... so that only the "name" portion of the e-mail address is displayed.

Hopefully, the maintainer of this code can add this to the official version :)

David

[dkendall]

Also, the problem (mentioned long ago) with the gateway spewing HTML when used with FireFox and the scheduled task manager is caused by this:

if (!headers_sent())
{
header("Content-Type: text/plain");
}
... around line 35 of gateway.php. Just take (or comment) it out, and the problem goes away.

Another thing... If you want to be able to run the gateway in debug mode via the Scheduled Task manager, replace function logging in functions_nntp.php with this:
function logging($text)
{
global $nntp;

if (isset($nntp['debug']) AND $nntp['debug'] == 1)
{
if (function_exists('log_cron_action'))
{
print_r($text . "<br>\r\n");
}
else
{
if (!headers_sent())
{
header("Content-Type: text/plain");
}
echo $text . "\r\n";
flush();
ob_flush();
}
}

if ($nntp['settings']['do_cron_log'])
{
$GLOBALS['nntp']['cron_log'] .= $text . "<br />";
}
}


This will stop it spewing raw HTML.

AFAIK, the scheduler won't allow you do specify the DEBUG parameter, so you'll also have to modify gateway.php and add this:$nntp['debug'] = 1;

David

[dkendall]

I found and fixed a few bugs:


When an article is imported to a sub-forum, the NNTP gateway was doubling thread and post counts in the parent forum(s).

It was updating the "latest post" fields, even if the forum (or parent forum) has newer posts.

The postings added by the gateway were not inheriting the icon of the thread to which they were added (if I understand this concept correctly).

I wanted imported NNTP messages to be linked to the vBulletin user profile of the poster. As the code is intertwined with the above bug fixes, I've included it here too.

To get the vBulletin user id from the NNTP post e-mail address, locate the following lines in gateway.php:


//Separate name and email address
$from_name = from_name($message['from']);
$from_email = from_email($message['from']);


Add the following lines immediately beneath:


//Get user id from email address
$from_userid = 0;
if ($from_email AND ($nntp_settings['associate_by_email'] == 1))
{
$user_data = $db->query_first("SELECT userid FROM "
. TABLE_PREFIX . "user WHERE email='" . addslashes($from_email) . "'");
if (!empty($user_data))
$from_userid = $user_data['userid'];
}


Note that we require each unique email addresses in vBulletin.

Fix up the SQL statements in the if ($threadid) / else block in gateway.php as follows:


// if the correct thread was found, insert it.
if ($threadid) {
$lasticonid = 0;
$icon_data = $db->query_first("SELECT iconid FROM " . TABLE_PREFIX . "thread WHERE threadid=$threadid");
if (!empty($icon_data))
$lasticonid = $icon_data['iconid'];

$postid = insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $from_userid, $date, $lasticonid, $parentid);

// update thread
$db->query("UPDATE " . TABLE_PREFIX . "thread
SET lastpost = '" . $date . "',
replycount = replycount + 1,
lastposter = '" . addslashes($from_name) . "'
WHERE threadid = $threadid
");

// update the forum counts
$db->query("UPDATE " . TABLE_PREFIX . "forum
SET replycount = replycount + 1
WHERE forumid = $forumid");

// update forum last-post data
$db->query("
UPDATE " . TABLE_PREFIX . "forum
SET lastpost = '" . $date . "',
lastposter = '" . addslashes($from_name) . "',
lasticonid = $lasticonid,
lastthreadid = $threadid,
lastthread = '" . addslashes($subject) . "'
WHERE forumid IN ({$foruminfo['parentlist']})
AND lastpost < '" . $date . "'
");


// send out email notices
exec_send_notification($threadid, "0", $postid);
} else {
//can only be here if no thread is found
//Needs to create new thread

// Create thread
if ($vbulletin->options['similarthreadsearch'])
{
require_once(MY_DIR . '/includes/functions_search.php');
$similarthreads = fetch_similar_threads($subject);
}
else
{
$similarthreads = '';
}

$db->query("INSERT INTO " . TABLE_PREFIX . "thread
(title, lastpost, forumid, open, replycount,
postusername, postuserid, lastposter, dateline, iconid,
visible, views, similar)
VALUES ('" . addslashes($subject) . "', '" . $date . "', $forumid, 1, 0,
'" . addslashes($from_name) . "', $from_userid,
'" . addslashes($from_name) . "', '" . $date . "', 0, 1, 0,
'" . $similarthreads . "')
");


$threadid = $db->insert_id();

//insert_post
$postid = insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $from_userid, $date);

// update the forum counts
$db->query("UPDATE " . TABLE_PREFIX . "forum
SET replycount = replycount + 1,
threadcount = threadcount + 1
WHERE forumid = $forumid");

// update forum last-post data
$db->query("UPDATE " . TABLE_PREFIX . "forum
SET lastpost = '" . $date . "',
lastposter = '" . addslashes($from_name) . "',
lasticonid = 0,
lastthread = '" . addslashes($subject) . "',
lastthreadid = $threadid
WHERE forumid IN ({$foruminfo['parentlist']})
AND lastpost < '" . $date . "'
");

logging("'$subject' from ". $from_name . ". New thread.");

} //new thread or not


In functions_nntp.php replace function insert_post with the following:


//Insert post and follow up

function insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $from_userid, $date, $iconid = 0, $parentid = 0)
{
global $db, $nntp;

$message =& $nntp['message'];

$db->query("INSERT INTO " . TABLE_PREFIX . "post
(postid, threadid, title, username, userid, dateline, pagetext,
allowsmilie, showsignature, ipaddress, iconid, visible,
isusenetpost, msgid, ref, parentid) VALUES
(NULL, $threadid, '". addslashes($subject) . "',
'" . addslashes($from_name) . "', $from_userid, '" . $date . "',
'" . addslashes($message['text']) . "', 1, 0,
'" . addslashes($from_email) . "', $iconid, 1, 1,
'" . addslashes($message['message-id']) . "',
'" . addslashes($message['references']) . "', "
. $parentid . ")");

$postid=$db->insert_id();

//So that thread preview works
$db->query("
UPDATE " . TABLE_PREFIX . "thread
SET firstpostid = $postid
WHERE threadid = $threadid
");

// ULTRASOFT HACK
if($from_userid != 0)
{
// update user's post count
$db->query("UPDATE " . TABLE_PREFIX . "user
SET lastpost = '" . $date . "',
lastactivity = '" . $date . "',
posts = posts + 1
WHERE userid = $from_userid
");
}

//save attachments if any
if ($message['attachments'])
{
process_attachments($date, $postid, $threadid, $forumid);
}

// Index post for searching
build_post_index($postid, $foruminfo);

return $postid;
}


In AdminCP go the to NNTP Gateway Settings, click the Add a New Setting button, and add this:


Title: Associate by Email
Varname: associate_by_email
Value: 1
Description: Use the email address to associate postings with the corresponding vBulletin user.


Hopefully, the owner/maintainer of this hack can add this to the "official" version.

David

[dkendall]

Here's another mod I've made to this hack.

In wrestling with getting it working, I needed to blow away my vBulletin threads/posts several times, and re-import everything from USENET. Since some of the postings originated in vBulletin, it was skipping them (which was mucking up the threading).

Locate the whole if (trim($message['user-agent'])... block of code in gateway.php and tweak it as follows:


$skip_post = 0;

if (trim($message['user-agent']) == trim($nntp_settings['useragent'])
AND $nntp_settings['organization_check'] == 0)
{
if ($nntp_settings['full_import_from_usenet'] == 0) {
logging("Skip, post was sent from our forum.");
$skip_post = 1;
}
}
else if (trim($message['user-agent']) == trim($nntp_settings['useragent'])
AND $nntp_settings['organization_check'] == 1
AND trim($message['organization']) == trim($nntp_settings['organization']))
{
//added organization so that we don't skip fellow gateway's posts
if ($nntp_settings['full_import_from_usenet'] == 0) {
logging("Skip, post was sent from our forum.");
$skip_post = 1;
}
}
else if (stristr(trim($message['x-no-archive']), 'yes') AND
isset($nntp_settings['honor_no-archive']) AND
$nntp_settings['honor_no-archive'] != 0)
{
logging("Skip, X-No-Archive headers is set.");
}
elseif ($nntp['grouptype'] == 'mail'
AND $group['prefix']
AND stristr($message['subject'], $group['prefix']) == false)
{
logging("Skip, not matching prefix: \"" . $group['prefix'] . "\"");
}

if(!$skip_post)
{
$kf = killfile_match();


In AdminCP go the to NNTP Gateway Settings, click the Add a New Setting button, and add this:


Title: Full import from USENET
Varname: full_import_from_usenet
Value: 0
Description: Override useragent and organization checks to import absolutely everything from USENET.


In order to use this feature, you have to:


Enable it by setting the full_import_from_usenet value to 1.
Delete everything from the thread and post tables in the vBulletin database (selectively, if you want to re-import only certain newsgroups or posts).
In AdminCP go to NNTP Gateway Newsgroups and set the Last message number to zero (or some appropriate value).
Run Update Counters and rebuild the thread and forum information, and Update Post Counts.
Run gateway.php script.
Reset the full_import_from_usenet value to zero.

David

[dkendall]

I found that the NNTP gateway was choking on some articles that had a period on a line by itself.

This seems to be used to indicate the end of the article, but in fact, there was more text to follow.

I worked around this by changing function get_article in nntp.php. I replaced this code:


while(!feof($this->fp)) {
//$line = trim(fgets($this->fp, 256));
//Took out the length, some lines are longer than that
$line = trim(fgets($this->fp));

if ($line == ".") {
break;
} else {
$post .= $line ."\n";
}
}


...with this:


while(!feof($this->fp)) {
$line = fgets($this->fp);

if ($line == ".\r\n") {
break;
} else {
$post .= rtrim($line, "\r\n") ."\n";
}
}


BTW, the original code appears to strip leading blanks from each line. This is probably wrong, so I made it strip only the cr/lf.

David

[Deepdog009]

Hey dkendall glad 2 C U be on the Ball with yo add ons. This hack has some po 10 show if it be enhanced and bug free.

I had a problem with it last month! Some groups dont like link backs to userid and forumid. I X-nessed all dat stuff ( signiture, id, links and anything dat might get da boot ) from my ex-posts. Usenet groups be picky when importing from the groups and exporting. All newbies trying diss out be careful when posting to Usenet. Dont put more than one group per forum!!! Mod yo posts.

Got a ???
Can U fix it so that can delete posts from a certain Usenet user in VBull??? When I try to delete a user from Usenet in VBull I get message user not found goback. Got posts by user but VBull says cant find user, what the hec.

Can U tell me how to fix???

[Grinler]

Wow...that took me about 10 reads to fully understand you. Usenet post are not posted as real vbulletin users..they are actually considered guest posts.

[dkendall]

Some groups dont like link backs to userid and forumid. I X-nessed all dat stuff ( signiture, id, links and anything dat might get da boot ) from my ex-posts.

I'm not sure what this means (I'm a vBulleting newbie myself). How do you put links to userid, forum id, etc., into a post? And how do they look when they're exported to the usenet group?

Can U fix it so that can delete posts from a certain Usenet user in VBull??? When I try to delete a user from Usenet in VBull I get message user not found goback.

With the patch I posted previously, imported usenet posts will be linked to VB profiles if their email addresses are found in VB. Otherwise, they're guest posts.

I suppose it would be possible to automatically register unknown email addresses in VB (perhaps using the email address as the username too), but I don't think that's a feature I would want to use.

David

[rmd708]

Has anyone gotten the footer striper to work. It doesn't work as far as I can tell.

[rmd708]

By the way, the thing about attachments not working. It may be because guests don't have permission to post attachments, and these posts are being put through as guests.

[dkendall]

I implemented the e-mail address masking feature mentioned earlier in this thread. But we use vBulletin for tech suport, and sometimes we post messages telling users to e-mail us directly.

So I've tweaked the masking feature so that it does not strip out addresses from certain domains.

I've also made it configurable in the NNTP Gateway Settings by adding these variables:

Title: Email Address Mask
Varname: email_address_mask
Value: \1 (AT) \2 (DOT) \3
Description: Mask for obfusicating email addresses in imported messages.

Title: Mask Email Exclude
Varname: mask_email_exclude
Value: onedomain.com;domaintwo.net;third.org
Description: Semicolon-delimited lowercase list of domains for which e-mail addresses should NOT be masked.


In gateway.php locate the line logging("Gateway version "... and insert this beneath it:

// Get email mask settings
$email_address_mask = $nntp_settings['email_address_mask'];
$mask_email_exclude = explode(';', $nntp_settings['mask_email_exclude']);


Locate the following block of code:

//Hide real email address for mailing lists
if ($nntp['grouptype'] == 'mail')
{
$message['text'] = preg_replace('/([-_.\\w]+)@([\\w]+[-\\w]+)\\./', '\\1 (AT) \\2 (DOT) ', $message['text']);
}


Replace it with this:

//Mask real email addresses in message text
if (isset($email_address_mask))
$message['text'] = preg_replace_callback('/(.+)@([a-zA-Z0-9\.]+)/', "mask_email_callback", $message['text']);


A few lines later, after the call to from_name, add the following code:

//clean up name
if (isset($email_address_mask))
$from_name = preg_replace('/(.+)@(.+)/', '\\1', $from_name);


NOTE: This handles the case where the display name in the nntp posting is an e-mail address. It simply changes my.name@domain.com to my.name, rather than using the email_address_mask value and without regard to the mask_email_exclude setting.

Finally, add this code to the end of functions_nntp.php:
function mask_email_callback($parts)
{
global $email_address_mask;
global $mask_email_exclude;

if (in_array(strtolower($parts[2]), $mask_email_exclude))
return $parts[0];

return preg_replace('/(.+)@(.+)\\.([^\\.]+)/', $email_address_mask, $parts[0]);
}


This will replace "my.name@my.domain.com" with "my.name (AT) my.domain (DOT) com". Of course, you can change the email_address_mask value to anything (e.g., "\1@..."). Leaving it blank will disable this feature.

David

[Deepdog009]

Oh, been away 4 awhile. Your ? about links is pertaining to the settings in admincp. I deleted the links coming back to forum. I deleted most info linking back and setup to only show post info without sig and footer.

dkendall when U export your posts do U experience sometimes some posts dont up to usenet? Why is that? Is there a fullproof way to verify that all posts were exported to usenet without problems.

I think it might have something to do with mod posts. Ive noticed that non mod posts work ok, but mod posts sometimes dont export.

Updated files and now getting this message on some posts:
Warning: Invalid argument supplied for foreach() in /includes/functions_threadedmode.php on line 202
Email mask setting changes page 18

Used updates from page 17, working beauts!

[tiffin100]

Hi

I am having a massive problem with duplicated threads being imported. I have even tried setting the time delay to 2 seconds between posts incase it was a slow server issue.

I have set the cron to run once an hour, which works fine.

I get no errors, I just get duplicate threads and lots of them, any ideas or help would be great..

[dkendall]

Hmmm. I answered this yesterday, but my reply hasn't shown up!

dkendall when U export your posts do U experience sometimes some posts dont up to usenet?

No, I'm not seeing that.

Updated files and now getting this message on some posts:
Warning: Invalid argument supplied for foreach() in /includes/functions_threadedmode.php on line 202
Email mask setting changes page 18

I looked at this code, and don't see how it can be related to the Email mask setting changes. It's probably caused by posts who's parents are missing from vBulletin.

When you start using the NNTP gateway, you have to make sure the "Last Message Index" in NNTP Gateway Newsgroups is set to zero.

Also, it won't import posts from NNTP that originated in vBulletin unless you use the full_import_from_usenet mod I posted earlier.

David

[Deepdog009]

Ok, testing your updates, working ok so far. I guess U be right about some posts showing up with error message if begining thread not found. That sucks.

Im still having a minor problem with Moderation. When posts arent moderated they successfully export to NNTP, but when posts are moderated sometimes they dont feed to NNTP. Does NEbody know why that is???:cross-eyed:

Grinler got one 4 ya!
U B O ta

[lierduh]

Hi

I am having a massive problem with duplicated threads being imported. I have even tried setting the time delay to 2 seconds between posts incase it was a slow server issue.

I have set the cron to run once an hour, which works fine.

I get no errors, I just get duplicate threads and lots of them, any ideas or help would be great..

This is due to having two instances of the scripts running at the same time. The script has ability to avoid such situation, but only check for scripts running within 30 minutes. So if you have an importation that is lasting for 1/2hour (eg. the very first import), don't set up/enble cron!

[dkendall]

When posts arent moderated they successfully export to NNTP, but when posts are moderated sometimes they dont feed to NNTP. Does NEbody know why that is???:cross-eyed:

I'm guessing here...

When the gateway copies posts to NNTP, it keeps track of the last message copied.

If you have a moderated post that is older than an unmoderated post, and the unmoderated post is copied over to NNTP, the gateway may never look at the older post.

David

[tiffin100]

This is due to having two instances of the scripts running at the same time. The script has ability to avoid such situation, but only check for scripts running within 30 minutes. So if you have an importation that is lasting for 1/2hour (eg. the very first import), don't set up/enble cron!


thanks lierduh for the quick response, this solved the problem.

[Deepdog009]

By george i C it now, tricky aint it

Keep up the good work:banana:

[hostingforum-it]

"dkendall" can you tell me please wich of your BugFix will fixe the double posting (double message count) ? I have running this script in a subforum and it double all the post (and counts) to the main forum.

Thanks

[hostingforum-it]

ok forget what i wrote below.... it works!!!. The newsserver nedded a few mins to show the new posts.
But i have a question:
How to modify the attached message that will be send to the newsgroup It currently send this:
----------
uername Profile: http://forums.yourdomain.com.au/member.php?userid=34
View this thread: http://forums.yourdomain.com.au/showthread.php?t=3247
----------





just curiouse... how does the script post to newsgroups? it seems to post on forum but Not to the newsgroup.
The news server i use allow to post (i are able to send mails via outlook express to the newsgroup)
Thanks

[athloni]

I installed this hack and I am currently getting this error (using GigaNews)


Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected

[athloni]

Someone please, I need some help. I have had some smart people look at my install and no one knows what is going on.

I am hosted at Asmallorange.com in shared hosting.



I have had a ticket escalated all the way to the admin:

I tested this out and it appears to be working. Although, I'm unfamiliar with this script. If you can do some more testing and also contact the developers of it to see what could be wrong with the server, we'd be more than happy to resolve this for you.

Regards,
Ryan



I am trying to connect to GigaNews. I have tried editing nttp.php several times to change the port to 23, 80, and 119, all of which GigaNews accepts. All of them returned the same error:

Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected

[hostingforum-it]

are you able to connect to the news server though another newsreader (eg. outlook express)?

[athloni]

are you able to connect to the news server though another newsreader (eg. outlook express)?



Yes.


and actually the thing 'magically' started working. Importing topics STILL :D

Fantastic hack.

[dkendall]

We found that some articles posted in vBulletin were not showing up in the NNTP newsgroups, and the following error was logged:

Posting Message 'Re: blah blah blah' from testuser. Result: 441 (615) Article Rejected -- Duplicate Message-ID <testuser.25r4mk@ultrasoft.com> (GLE=0)

This error is caused by the following code in sendnews in functions_nntp.php:
$msgid = $u . '.' . base_convert ($msgid_date, 10, 36) . '@'.$nntp_settings['email'];
$msgid_date++;


Note that $msgid_date is a local variable, initialized from the global $nntp['msgid_date']. The next time this function is called, it uses the same value of $nntp['msgid_date'].

Consequently, if the same user posts twice (or more) all their messages get the same Message-ID.

The fix is simple. Use the global variable instead, as follows:
$msgid = $u . '.' . base_convert ($nntp['msgid_date'], 10, 36) . '@'.$nntp_settings['email'];
$nntp['msgid_date']++;


You can also get rid of the earlier initialization of the $msgid_date local variable, as it's no longer needed.

While we don't use moderation, a similar bug would seem to be the cause of the problem Deepdog009 reported with moderated posts getting left behind when non-moderated posts are replicated (causing the last_postid to get incremented).

Again in function sendnews in functions_nntp.php, the $good_to_set_postid variable is local in scope. When a moderated post is replicated, the local variable is set to zero (false), rather than the $nntp['good_to_set_postid'] global variable.

Having done these fixes, I found an additional problem whereby replies to the posts that had failed to replicate were themselves being rejected (because their parent posts were missing in NNTP).

I worked around this by setting the last_postid value in the NNTP Gateway Settings page to zero, then re-running the gateway. It won't duplicate posts because those already sent to NNTP are flagged in the database.

Regards,
David

[athloni]

I the forums are updating at around 1 post per second. Is the the normal rate that everyone else is getting?

It seems like downloading a few hundred MB of text should take minutes... not a week. Is it because it has to put into the db and posted?

[newmomsforum]

Hey Guys, having some frustrating times with this, the latest posts are being pulled down and sync'd with my forum fine, however when I post I'm getting a 441 posting failed error:

The Full text from gateway.php?debug=1 is:

Gateway version 2.3.2 2 group(s) gatewayed.

Connecting to server, server says: 200 News.GigaNews.Com

Server responded after user name: 381 more authentication required

Server responded after password: 281 News.GigaNews.Com

Info for misc.kids.pregnancy at news.giganews.com: 211 146164 682173 828336 misc.kids.pregnancy

Can we post? server responded: 340 send article
Posting Message 'Re: Andrea update' from Claire. Result: 441 posting failed

Any ideas where I can start to look to get this sorted?

Thanks in advance

Claire.

[poliveira]

I am receiving this error:

Database error in vBulletin 3.5.4:

Invalid SQL:
UPDATE nntp_settings
SET value = WHERE varname = 'last_postid';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 2
Error Number : 1064
Date : Saturday, April 8th 2006 @ 10:18:32 PM
Script : http://.../forums/gateway.php
Referrer :
IP Address : ...
Username : ...
Classname : vB_Database

[phill2003]

This is great, It does exactly what it says on the tin.

Thanks for this it really is awesome...

[newmomsforum]

Wish it did for me Phill2003

Still getting the 441 posting failed errors as above. So near and yet so far.

Does anyone have any ideas where I can start to look for this?

Could it be a server issue? I run the site on a dedicated server so have no issues tweaking settings if required.

Are there any NTTP options I need enabled within Apache to enable me to post back?

Thanks anyone for any advice :)

Claire

[phill2003]

Are you sure the news server your using allows posting quite a lot of the free ones dont, I used a premium service for mine and it worked straight away?

[newmomsforum]

Hey phill2003, thanks for the response :)

Yeh I'm using a premium service too (giganews) and if you look at the error message I'm getting:

Can we post? server responded: 340 send article
Posting Message 'Re: Andrea update' from Claire. Result: 441 posting failed

The newserver is confirming we can post but when the post is being seent, its at this point that it fails. :(

[dwbro1]

Mine was working fine updating messages for days now and it was almost caught up to current in the two groups I have added and then it seems to have just stopped. I use giganews and it was up to about 79K post in just one of the groups when it has just stopped. Still about 6 more months worth of post to catch all the way up too.. :(.. When I run the task from the scheduler it says:

UseNet Update

Done

but the post counts are not climbing. I checked giganews and I have not used but a 1/4 of my monthly quota so thats not it. Any thoughts on what I can do to find out what is happening?

[dwbro1]

Any ideas?

[newmomsforum]

dwbro1, try manually increasing the Last Message count by by 1 or two.

Ie, if your current message count is 754545 change it to 754546 and run gateway.php again.

See if this works :)

Claire

[dwbro1]

Yeah I did and nada. I finally just jumped from 7-22-05 to the first msg on 1-1-06 and started again and it worked. Just lost 4 to 5 months worth but thats ok.

[newmomsforum]

Fair enough..glad you got it working dwbro1, minus a few thousand posts anyway ;)

Quick question if anyone has any ideas.

Many of the posts I'm pulling for one of my sites from a particular usenet newsgroup includes posts from what I guess can only be described as a competitor, made to the usenet group via a similar (if not the same) hack.

Thats not the problem, what is the problem is every post from said site has a nice big footer including the text:

"This post was made at http://www......XXXXX - Click here for the best information about" etc etc.....

I don't add anything similar to my footers and although I don't mind general links in any posts I 'grab'; I don't want to be actively driving potential new members to this other site.

I know this can be done with mailing lists but is their a way to strip out certain footers for usenet syncs?

I thought that I might have been able to use the turn signature off for guests option but as these footers form part of the main text thats not really an option.

I have also tried the censorship option , however you don't appear to be able to censor full phrases, just individual words which is of no help to me.

I'm still pretty new to VB so I may be missing something. Any ideas / help is really appreciated.

Cheers all

Claire :)

[trackpads]

How do you truncate the usernames? They are warping my threadview ugh :)

I posted the question here as well with a screenshot. It is some of the NNTP usernames that are doing it.

https://vborg.vbsupport.ru/showthread.php?p=957447#post957447

Thanks again,

-Jason

[trackpads]

Bump please!!

Thanks again!

-Jason

[trackpads]

Some ideas for improvements or the next version:

1. Ability to have a seperate Killfile for some newsgroups. A good example is this scenario. I have Bush, Kerry both in my killfile. I know that in any computer or sports groups these are definately spam. However in the politics groups they are ok.

2. Ability to truncate ultra long usernames from imported posts. I have a few doozies that are stretching my last post columns. Here is an example:

=?Windows-1252?Q?Rob_Giordano_\=28Crash_Gordon=AE\

3. Make the script smart so that if the load goes up to a max amount (10 for example) that it will cancel operation until the next cron. This would help when in some cases it is working on some new groups and the forums are allready busy with members.

4. Make a default news server with username and password defined. This is instead of posting them each and every time you add a new group. I just added 90 groups and that was a lot of copying and pasting!!


Thanks to lierduh for creating this and thanks again to dkendall for the awesome edits!!

Thanks,

-Jason

[TMM-TT]

I'm not sure if this has been discussed yet (I have 20 pages to look through), so I'm going to take the risk and ask anyway -

Is it possible, in any way, to handle cancel-posts so posts are deleted when they are canceled, or when you as a user delete your own posts, send a cancel to the group?

[rmd708]

Truncating user names needs to be done in the thread list file of vbuletin. I did it, but it was a while ago so I forgot what I did. It involved copying the vb code used to truncate subject lines to X characters, and rewriting it to do the same for user names. If it is really needed I can look it up.

[rmd708]

One problem I'm having is some messages (MANY) are being completely striped of the body. It says email footer match found, and then deletes the entire message body. When I check the original message and then my strip footer list, there is nothing at all that matches. So I have no idea why these messages are being erased like that.

[dwbro1]

Truncating user names needs to be done in the thread list file of vbuletin. I did it, but it was a while ago so I forgot what I did. It involved copying the vb code used to truncate subject lines to X characters, and rewriting it to do the same for user names. If it is really needed I can look it up.

Oh yea this would really be nice. I get some of the kooks out there that put entire sentences for their name then by tables are about 50/50

[rmd708]

In functions_forumdisplay.php I have added


$thread['trimnamezz'] = fetch_trimmed_title($thread['lastposter'], 15);


just above this:
$thread['lastpostdate'] = vbdate($vbulletin->options['dateformat'], $thread['lastpost'], true);
$thread['lastposttime'] = vbdate($vbulletin->options['timeformat'], $thread['lastpost']);


Then in the threadbit template, I use this:

<phrase 1="member.php?find=lastposter&amp;t=$thread[threadid]" 2="$thread[trimnamezz]">$vbphrase[by_x]</phrase> <a href="showthread.php?$session[sessionurl]goto=lastpost&amp;t=$thread[threadid]"><img class="inlineimg" src="$stylevar[imgdir_button]/lastpost.gif" alt="$vbphrase[go_to_last_post]" border="0" /></a>

I forget what the original had, but your default one should be the same minus the variable "trimnamezz" (i forget what the default variable name is).

That will get the forum display user name trimmed.

Now for the forums index, in functions_forumlist.php:

Find:
$lastpostinfo['lastpostdate'] = vbdate($vbulletin->options['dateformat'], $lastpostinfo['lastpost'], 1);
$lastpostinfo['lastposttime'] = vbdate($vbulletin->options['timeformat'], $lastpostinfo['lastpost']);
$lastpostinfo['trimthread'] = fetch_trimmed_title($lastpostinfo['lastthread']);
and add this below:
$lastpostinfo['trimnamez'] = fetch_trimmed_title($lastpostinfo['lastposter']);

Then open up the particular template for the forum index and use the variable trimnamez in place of the default one.

[rmd708]

The strip footer seems to have problems. For example I found using {Yahoo! Groups} in the strip footer would strip the body of countless messages. I assume the '!' is being read as some type of wildcard, but am not sure about it.

[trackpads]

Thanks, it works great!

The only one that is missing is the subforum bits, see my screenie. I have looked everywhere and cant find the right code, how did you solve this?

Thanks again!

[rmd708]

I don't use subforums, so I never tried to fix that one.

Try one of these two:

1) Try the same variable in the template in case it uses the same function file.

If that doesn't work...

2) Find out which function includes file deals with the subforum display, then add the same code as above to it and then reference the variable.

[WritersBeat]

Does this work for 3.5.4?

[newmomsforum]

Does this work for 3.5.4?

Yes, I'm running this on 3.5.4 :)

[newmomsforum]

Hi Guys

When I attach the URL of my site in my Default Footer using html, the display gets all screwed up, BB code has the same impact. Am I missing something?

Cheers :)

[intinig]

I am using this hach as a gateway to a yahoogroups mailing list. We're Italian, so we use accented letters like ??????. After mails with accented characters are imported in the system I get lots of rubbish instead of those characters (http://italianmagic.net/forum for examples). Any clue?

[deejayh]

First post after buying VBulletin - purchased it for this addon!

I have installed this, but cannot get a newsfeed - tried various microsoft and my own ISP though I believe I cannot connect to them directly from my server? (not allowed?):
Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected

Anyone know of a free newsfeed source to test to see if my installation works.

Is this add on / mod still supported??

Really need this for my forum.
regards,
Dave

[deejayh]

I guess from the non response that this mod is dead & nearly buried?

Anyone got this working?

[britannia]

We really need this too.

If this is not supported - Is there anotyher one similar to this?

thanx

[deejayh]

Hi britannia,

I think this is dead! Cannot find when Lierduh (https://vborg.vbsupport.ru/member.php?u=28088) was last online!
Really annoying as I have just had my license for 2 weeks, which I purchased specifically for just for adding this!

Looks like I may have to sell it and go for another forum script - fortunately I have not gone live - just used impex to transfer everything to test and to add this add on.

We live and learn!

best regards,
Dave

[ubuntu-geek]

Hi britannia,

I think this is dead! Cannot find when Lierduh (https://vborg.vbsupport.ru/member.php?u=28088) was last online!
Really annoying as I have just had my license for 2 weeks, which I purchased specifically for just for adding this!

Looks like I may have to sell it and go for another forum script - fortunately I have not gone live - just used impex to transfer everything to test and to add this add on.

We live and learn!

best regards,
Dave
You might make sure your server has the nntp port open. It appears that could be the issue.

[deejayh]

Thanks for that - I was wondering the same!

Contacted my hosts:
you cannot import these feeds directly to your vbulleting installation on the server. You may need to setup a local copy and then import the mysql to the server.

Damn it!

Anyone know how to setup a local copy and then import the mysql to the server?

thanks,
Dave

[deejayh]

Thinking about it, do I realy need my hosts server to have port 119 open?

Surely I am connecting to the selected SERVER - news server port 119!!

A general newsreader does not use 119 on your own PC -just connects to the news servers port 119?

or am I getting it wrong??

But.... I have tried loads of servers even the microsoft one suggested - still no connection!! maybe I am wrong!

Help.....

[deejayh]

bummpppp!
Thinking about it, do I really need my hosts server to have port 119 open?

Surely I am connecting to the selected SERVER - news server port 119!!

A general newsreader does not use 119 on your own PC -just connects to the news servers port 119?

or am I getting it wrong??


How does this script connect to the newserver?

thanks,
Dave

[TMM-TT]

How does this script connect to the newserver?



As a client connects to a server.


There's some read-only servers to try with. Or are you looking for servers that allows both read and post?

[deejayh]

Thanks for the reply TMM-TT,

I would prefer to recieve and send - but at the moment I just need to be able to connect and download news - to ensure this add-on works - before I pay for a newsfeed.

I have tried a number of newsfeeds but it keeps coming up with:
Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected
Then ubuntu-geek said:
You might make sure your server has the nntp port open. It appears that could be the issue.
So I spoke with my host who said that port 119 is blocked.
Then I thought, surely this add-on is not talking thro my host servers port 119 - but connecting to the selected new servers port 119. So I would not need to have port 119 open on my host? Maybe I am wrong?

Just need to get it working!

Thanks again TMM-TT:)

Regards,
Dave

[TMM-TT]

Thanks for the reply TMM-TT,

I would prefer to recieve and send - but at the moment I just need to be able to connect and download news - to ensure this add-on works - before I pay for a newsfeed.

[...]

So I spoke with my host who said that port 119 is blocked.

Then I thought, surely this add-on is not talking thro my host servers port 119 - but connecting to the selected new servers port 119. So I would not need to have port 119 open on my host? Maybe I am wrong?

No, as long as the outgoing port 119 is not blocked, it should work fine. Try connect to news.tornevall.net at that port and see what answer you get.

By the way - buying accounts can be a bit "bad", so be careful with the choice there. :)
Normally newsproviders only allows one user per account which means that sharing such account with others may not be allowed.

[deejayh]

Thanks TMM-TT,

What newsgroup shall I add?

regards,
Dave

[deejayh]

tried:
server: news.tornevall.net
Forum: testing
newsgroup:uk.news
Last Message: 0
left everything else blank

got:
Gateway version 2.3.2 2 group(s) gatewayed.
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected
Could not connect to NNTP-server. (Connection timed out (110))

Not connected
Not connected

[Aylwin]

Now the attachs work in my 3.5.3, for this i copy a functions_image.php file from another old vb instalation to my includes.

And i comment some lines in the file incliudes/functions_nntp.php


// if (!$forumperms)
// {
// logging("This forum does not allow attachments, therefore attachment ignored.");
// }
// else
// {
$forumperms2 = true;
// }


Does anyone know if attachments work with mailing lists?

Also, can someone please attach functions_image.php? I'd like to try to get this working. Thanks!

[theaonline]

My outgoing emails have their extra carraige returns/line breaks snipped. Does anybody know where I can fix this?

[ubuntu-geek]

Just a FYI this works on 3.6.0 beta4 without any known issues yet..

[Aylwin]

Finally found a functions_image.php file and got this mod working.

Has anyone noticed a problem with large attachments/mails? gateway.php seems to hang when it gets to a message larger than 2MB.

[Altec]

Hi everyone,

I've tried to install this hack, everything went well except when I run gateway.php?debug=1 all I get is a blank page. Although setlastmsg worked fine.

Thanks!

[bang]

I assume this hack is very very far from working on 3.6?

[TMM-TT]

I assume this hack is very very far from working on 3.6?
It works for me.. :)

[Rik Brown]

Just a FYI this works on 3.6.0 beta4 without any known issues yet..

I've upgraded to vb 3.6.0 but have been waiting to try out this mod. Have you fully upgraded to the 3.6.0 release version? Still no problems?

I also see that there are database changes involved. Hopefully, those don't touch the vb database but only create their own tables?

Would appreciate an update before I try it out.

Thanks. -- Rik

[WritersBeat]

How would I set this up for alt.books.reviews ? I have no idea how to do this.

[TMM-TT]

How would I set this up for alt.books.reviews ? I have no idea how to do this.


This is how I set up alt.games.the-sims - just change the data to your settings. :)

[KW802]

Anybody able to hack this yet for 3.6.0 gold?

[TMM-TT]

Anybody able to hack this yet for 3.6.0 gold?
I had some problems with attachments, but after some changes (after the above advices about functions_image.php, etc) that worked too. I actually just made it work ;)


(But I still have problems with showing thumbnails)

[esfron]

First of all, thanks for this wonderful hack. I've been using this successfully for months. However it doesn't work correctly with 3.6.

The biggest issue is when i click on the 'go to last post' button on forumhome, it says no thread specified and it leads to: /showthread.php?p=0#post0. I have noticed that last button code does not seem the same between 3.5x and 3.6: .../forum/showthread.php?goto=lastpost... and .../forum/showthread.php?p=...#post...

I didn't test mailing list, attachments, export to usenet.

Thanks for any feedback on this issue

[KW802]

Would appreciate any insight on this one...

vB 3.6.0 -- Incoming feeds are working. Outgoing feeds though are a problem. According to the logs everything is working fine... the outgoing posts are giving a status code of 240 and no errors are record.

The outgoing posts, though, never appear on the news server. :(

Thoughts? Anybody know of a free NNTP server that I can use for testing just to see if the problem is with my site or my news provider?

[TMM-TT]

Thoughts? Anybody know of a free NNTP server that I can use for testing just to see if the problem is with my site or my news provider?


I can set up an account for you at news.tornevall.net. Just send me a PM if you're interested. :)

[KW802]

TMM-TT,

Thanks for the offer. I seem to have solved my issue, though. I switched from Supernews (via Forte's Agent News provider) to Giganews and everything is working fine again.

Thanks, :cool:
Kevin

[TMM-TT]

Thanks for the offer. I seem to have solved my issue, though. I switched from Supernews (via Forte's Agent News provider) to Giganews and everything is working fine again.

Perfect. Just tell me if there's any interest of forum-usenet-accounts.. :)


Another question -

When posts are made into the forum, I sometimes can see duplicate posts; just after the post, the imported one repeats itself so it looks like duplicates in the forum. Does anyone know if there's any dupechecking somewhere, or is it like this for you too?

Edit: My little solution didn't work, so I wonder if there is any other to prevent dupes instead. :P

[dethfire]

I'm finding most free servers are supporting only NNRP, any way to implement this?

[TMM-TT]

I'm finding most free servers are supporting only NNRP, any way to implement this?

200 news.tornevall.net InterNetNews NNRP server INN 2.4.3 ready (posting ok).

NNRP is the NNTP server for reader clients, that comes with the INN server package, a news-server running under Linux.

[KidCharlemane]

Are there any decent free servers for text only? I have a Giganews account but they tie you to a single IP so if youre logged in with a client on your home machine, the server cant log in to process messages.

I was hoping using their text only server they might let it slide but alas, they dont.

[dethfire]

I can't connect to anything all of a sudden. Is there something on a server that would deny access? Some security thing?

[KidCharlemane]

Anyone gotten this to work with Google Groups? Not Google's Usenet archive but the Google email groups.

[Rik Brown]

First I want to thank lierduh for his work on developing this mod and krandall for his fixes. We've been importing 3-years of messages from a large newsgroup this week and all appears working well so far (haven't tried exporting yet).

While it would be best to filter out spam and other junk in the importation phase [killfile], one obviously doesn't know what and who to scan for at first. Has anyone developed a series of mySQL commands for the post-importation of messages to work on cleaning up the spam and bad users? If so, could you please advise a sample here that I and perhaps others can build on?

In my case, I would like to:

1) Posts

- delete any post with a URL in it (99% spam)
- delete any post with certain keywords in them
- delete any post by specified users
- delete any post with a URL in the User Name

2) Threads

- delete any thread with a URL in the title
- delete any thread with certain keywords in them
- delete any thread by specified users
- delete any thread with a URL in the User Name

Thanks. -- Rik

[KidCharlemane]

Unfortunately I keep getting an "out of memory" error when I run this hack. Ive used it before with success but for some reason now, it's just not happening. Anyone got any ideas how I can work around the problem? Here is the actual error:

"Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 3840 bytes) in /usr/local/psa/home/vhosts/trianglecycling.com/httpdocs/includes/functions_newpost.php(181) : regexp code on line 1"

Any ideas?

[Aylwin]

Unfortunately I keep getting an "out of memory" error when I run this hack.

Any ideas?
Have you tried allocating more memory to PHP? I can't remember the syntax right now but you can set it in php.ini or .htaccess.

[KidCharlemane]

I dont think I can.. the domain is on a Plesk based system and I dont have access to the administrator control panel.

This is happening on the initial import so I feel pretty sure if I could get all the messages imported, the recent postings between intervals wont cause this problem. I'm trying to import two groups tho with a lot of posts so I dont know how long it will take to actually get them all. It would be nice if you could import XX amount of posts, have the script terminate and then restart to get them all. I dont think memory usage was ever a consideration to the original author of the hack.

[TMM-TT]

Shouldn't a dedicated (?) server have much memory available?

[KidCharlemane]

Depends on how the server is configured. If you run a lot of forums on one machine, you kind of need to keep scripts in check so nothing goes out of control and slows everything else down.

The box I have the partiulcar forum Im testing this hack on isnt mine and the way its configured, PHP scripts are limited to 8mb of memory.

[TMM-TT]

The box I have the partiulcar forum Im testing this hack on isnt mine and the way its configured, PHP scripts are limited to 8mb of memory.


My php.ini contains this row:

memory_limit = 8M ; Maximum amount of memory a script may consume (8MB)


... and I don't have any problem with the gateway-script. Strange?

[Aylwin]

I dont think I can.. the domain is on a Plesk based system and I dont have access to the administrator control panel.
Doesn't Plesk support custom php.ini files? I believe version 7 did but maybe not newer versions. Or maybe the webhost admins need to enable/allow it. Perhaps you could try to check with them.

My webhost uses cPanel which used to support custom php.ini files. Unfortunately though, it doesn't work anymore after they upgraded to PHP5 (I guess cPanel was also upgraded).

[Aylwin]

How to check incoming posts for valid forum users

I use this mod as a mailing list gateway. Many of the people on our mailing lists are also forum members. So I thought it would be good to check for valid forum users. No guarantees whatsoever but if anyone's interested, here's how I did it.

In gateway.php, look for:
//Separate name and email address
$from_name = from_name($message['from']);
$from_email = from_email($message['from']);

Below, add:
//Check if the sender is a user
$userinfo = $db->query_first("SELECT userid,username FROM " . TABLE_PREFIX . "user WHERE email = '" . $from_email . "'");
if ($userinfo['userid']) { $userid = $userinfo['userid']; } else { $userid = 0; }
if ($userinfo['username']) { $from_name = $userinfo['username']; }

Look for both instances of:
$postid = insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $date, $parentid);
Replace both with:
$postid = insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $userid, $from_email, $date, $parentid);

In functions_nntp.php, look for:
function insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $from_email, $date, $parentid = 0)
Replace with:
function insert_post($threadid, $forumid, $foruminfo, $subject, $from_name, $userid, $from_email, $date, $parentid = 0)
Look for:
'" . stripslashes($from_name) . "', 0, '" . $date . "',
Replace with:
'" . stripslashes($from_name) . "', " . $userid . ", '" . $date . "',

[John Diver]

Hey,

I'm not getting pictures, can pictures be included in posts? Almost all posts I'm getting have images.

Here is my forum for the posts: http://www.deuceace.com/forumdisplay.php?f=24

It also doesn't seem to be showing much text, am I able to change this or is it a limitation of what I am allowed to take from the sites?

[ShawnV]

Will this work on 3.6.1?

_V

[mdevour]

When I bought vBulletin last February I was depending on the fact that this hack was working and supported, which it at least seemed to be at the time.

Lierduh appears to be busy on other things and hasn't posted to this thread since March. Dave Kendall and others have written many tweaks and bug fixes, but they're not yet integrated into the distribution files, requiring each new user to review 350-plus posts to collect them all.

Thankfully, frakman in post 67 gives me some guidance on my most critical requirement, which is importing a bunch of mbox format e-mail archives to initially populate two of my six new forums.

I knew going in that I'd have to build on this code to get my board and list server to work together the way I need them to, but I had hoped to start from a working code base.

We at least need to:


Integrate the current bug fixes
Add the new features that a couple of people have coded
Update and expand the installation instructions
Update the hacker's guide (or incorporate its features as options)
Test everything against 3.6.1 and upcoming releases


I'd also like to brainstorm if the mod ought to be upgraded to use the plug-in system, or whatever other ideas anybody has.

If any of this is going to happen either lierduh has to pick it up again or somebody else has to (assuming he agrees, of course...).

Naturally I wouldn't be talking like this if I wasn't prepared to offer myself as a candidate. The trouble with that is I'm not a coder. All my programming experience is 20 years out of date. Every time I pick up this project I have to haul myself a little further up the learning curves for vBulletin's structure, XML, modern HTML, CSS, and PHP. As far as learning curves are concerned, on a scale of one to ten -- where zero is your grandma -- I'm somewhere below two! :cross-eyed:

So, can a guy who's working hard just reading this stuff take what others produce, integrate it, organize it and release it? Project leader as glorified filing clerk? Is that even vaguely realistic?

Is there enough interest here to build a serious collaboration around?

Maybe even enough interest to finance some development work?

What say ye?

I know some of you will think this is a wonderful idea, since you want the hack working again as much as I do, but I'm especially interested in the advice of experienced coders and developers. If I'm hopelessly out of my depth you'll know it, and it'd be better to figure that out now...

Thanks for reading.

Mike Devour

[lierduh]

If anyone who is interested in further developing this hack and provide support for it. I will have no problem hand over the hack. ie. assuming whoever takes over do not turn this into a commercial product.

I have lost the interest to do any work on this for the reasons I might have mentioned before.

1) Hacks are no longer the way the hacks used to be. When I was first on the net around 95, hack means everyone contributes code to make something better. Nowadays hack means free software. I have no problem with that, the problem is most of the people using the hack do not even spend more than a few minutes to read the instruction, let along checking the code, hack the code, help others.

The result is coders start to turn good code into commercial products, creating their own forums to build up the traffic. Not that I blame them.

2) I am not a real programmer, I check php.net regularly for how to use functions etc. I started hacking this when I needed this for my own forums. So I am not interested making money out of this, because my time is much more valuable doing what I do.:)

[Aylwin]

We at least need to:


Integrate the current bug fixes
Add the new features that a couple of people have coded
Update and expand the installation instructions
Update the hacker's guide (or incorporate its features as options)
Test everything against 3.6.1 and upcoming releases


I'd also like to brainstorm if the mod ought to be upgraded to use the plug-in system, or whatever other ideas anybody has.

...

What say ye?


Mike,

Great initiative! I'm willing to help in any way I can.

-Aylwin-

[TMM-TT]

We at least need to:


I think one thing to do is add support for "handshaking", to forums that connects to peer-servers on the same connection. That thing can be hard to explain, specially in english, but I'll try.

Client connects to port 119 as usual, but so are servers (peers that transfer the news). The difference between a peer and a client is the handshake string (not the authinfo). When a server-to-server connection is established, the string "MODE STREAM" are sent to identify the connection as non-human. Clients are sending "MODE READER" to identify as human readers. This hack does not do that, and that gives me some problems since I'm peering locally from the same server as the forum. I also use a suck-feed for some local groups which automatically makes me identified as a peer. So for now I have to use a dummy ip to connect as a client to the server. I tried to fix that part, but failed.

If a server is configured as a clientserver the "MODE READER" isn't necessary for the handshake, so it says "Hi, I'm a clientserver" first - and that's why this hack works so good. :)

If you're a peer, you have to send MODE READER if you don't want to peer. If a server is configured as a peering server (both methods are made by ip detection), the server immediately acts as streamingserver instead and you have to enter MODE READER to identify as a client. This is where the problems start. When I'm trying to download posts to the forum as a client, the server identify me as a peer and the posts never arrive.

To be short - handshake identification is needed in this hack. :)


And I'm also willing to help. :)

[ullo]

When I bought vBulletin last February I was depending on the fact that this hack was working and supported, which it at least seemed to be at the time.

Lierduh appears to be busy on other things and hasn't posted to this thread since March. Dave Kendall and others have written many tweaks and bug fixes, but they're not yet integrated into the distribution files, requiring each new user to review 350-plus posts to collect them all.

Thankfully, frakman in post 67 gives me some guidance on my most critical requirement, which is importing a bunch of mbox format e-mail archives to initially populate two of my six new forums.

I knew going in that I'd have to build on this code to get my board and list server to work together the way I need them to, but I had hoped to start from a working code base.

We at least need to:


Integrate the current bug fixes
Add the new features that a couple of people have coded
Update and expand the installation instructions
Update the hacker's guide (or incorporate its features as options)
Test everything against 3.6.1 and upcoming releases


I'd also like to brainstorm if the mod ought to be upgraded to use the plug-in system, or whatever other ideas anybody has.

If any of this is going to happen either lierduh has to pick it up again or somebody else has to (assuming he agrees, of course...).

Naturally I wouldn't be talking like this if I wasn't prepared to offer myself as a candidate. The trouble with that is I'm not a coder. All my programming experience is 20 years out of date. Every time I pick up this project I have to haul myself a little further up the learning curves for vBulletin's structure, XML, modern HTML, CSS, and PHP. As far as learning curves are concerned, on a scale of one to ten -- where zero is your grandma -- I'm somewhere below two! :cross-eyed:

So, can a guy who's working hard just reading this stuff take what others produce, integrate it, organize it and release it? Project leader as glorified filing clerk? Is that even vaguely realistic?

Is there enough interest here to build a serious collaboration around?

Maybe even enough interest to finance some development work?

What say ye?

I know some of you will think this is a wonderful idea, since you want the hack working again as much as I do, but I'm especially interested in the advice of experienced coders and developers. If I'm hopelessly out of my depth you'll know it, and it'd be better to figure that out now...

Thanks for reading.

Mike Devour


I also have bought VBulletin only for this hack.

It work with 3.6.1 ?
For the moment i think to use 3.5.5.
Is this version :
https://vborg.vbsupport.ru/attachment.php?attachmentid=32237
the latest and most recent ?

[Aylwin]

It work with 3.6.1 ?
For the moment i think to use 3.5.5.
Is this version :
https://vborg.vbsupport.ru/attachment.php?attachmentid=32237
the latest and most recent ?
I'm using it with 3.6.0 with no apparent problems. But I think I had to tweak something in one of the files. I just can't remember right now. Ihaven't had time yet to upgrade to 3.6.1 but I guess it should still work.

And yes, I believe that's the current version.

[Aylwin]

I'd also like to brainstorm if the mod ought to be upgraded to use the plug-in system, or whatever other ideas anybody has.
My personal interest in this mod is on mailing list support. I've been using it as a mailing list gateway and archive. I'd also like to expand this mod to use it like an email interface to the forums (kinda like phpBB's Mail2Forum mod). I know this is not the original intention of the mod but I think the necessary components are already there. I also know that there's a vB Mail Reply mod but that hasn't had any new development since 3.0.x and the implementation is a bit too complicated.

Having said all that, here's my wishlist:
- IMAP support
- configurable "user check" (I think I can handle this)
- configurable "add to user's post count" (I think I can handle this too)
- multiple forum support instead of 1 list <--> 1 forum
- updated attachment support (no need for an old functions_image.php file)
- support for large attachments (at least I can't get it to work now)

[TMM-TT]

Another thing:

I experience duplicates on posts that goes to usenet. They bounce back to the forum and gets posted one more time, like it was from a usenet-user instead of the original member.

[ullo]

I'm using it with 3.6.0 with no apparent problems. But I think I had to tweak something in one of the files. I just can't remember right now. Ihaven't had time yet to upgrade to 3.6.1 but I guess it should still work.

And yes, I believe that's the current version.

Thank you very much.
Now i will install 3.6.1

[ShawnV]

I am using this as a mailing list gateway however I have one small problem, is there a simple way to set it to not mask emails and to just send to mailing list using the users Vbulletin email?

Thanks in advance for any help.

_V

[Aylwin]

...send to mailing list using the users Vbulletin email?
In the newsgroup settings:
Enter 'use_real_email' if you want the posts to use the real email addresses.

[ShawnV]

Doh, yeah I must have been up way to late I missed that in the readme file.

Thank You. :)

[ullo]

The Subject deal worked perfect for Mailinglist.

But attachments are ignored.
My settings: "Skip attachments no_attach 0"

"Thread found by Subject.
This forum does not allow attachments, therefore attachment ignored."

If I post in the forum I can post attachments fine. I use filesystem for my attachments by the way, not sure if that matters.

I HAVE THE SAME IDENTICAL PROBLEM. ATTACHMENT ARE NOT DOWNLOADED BY THE THE SCRIPT WHEN I LAUNCH
gateway.php?debug=1
and i have the error :
"This forum does not allow attachments, therefore attachment ignored."
But i have allowed all attachment for all forums. What is the solution ?

[ullo]

I had some problems with attachments, but after some changes (after the above advices about functions_image.php, etc) that worked too. I actually just made it work ;)


(But I still have problems with showing thumbnails)

Yes. I have the error above using 3.6.1.
If i use 3.5.5 attachment and thumbnails work good ?

[TMM-TT]

"This forum does not allow attachments, therefore attachment ignored."
But i have allowed all attachment for all forums. What is the solution ?

It has been described earlier somwhere. If it was here or a thread for a rescent version of vB or not, I don't know but I think that worked out for me.

You need an old functions_image.php


In functions_nntp.php, disable this text (around line 887 or something):

if (!$attachtypes[$extension] OR !$attachtypes[$extension]['enabled'])
{
logging($extension . " extensions are not accepted, as currently set up in the control panel.");
}
else
{
$extensionok = true;
}


After that text I also added this (I don't know if that's necessary):

// Always accepting attachments... [tornevall]
$extensionok = true;



I never got my thumbnails work with the default code so I hade to change saveintodb to:


function saveintodb($i, $thumbnail, $filesize, $date, $postid)
{
global $vbulletin, $db, $nntp;
$message =& $nntp['message'];

// save with thumbs into the db
$filesize = $filesize + 0;

if ($vbulletin->options['attachthumbs']) {$okthumb = 1;}
// Another modification [Tornevall]
$okthumb = 1;

if ($okthumb == 1)
{
$db->query("INSERT INTO " . TABLE_PREFIX . "attachment SET dateline = '$date',
filename = '" . addslashes($message['attachment' . $i]['headers']['filename']) . "',
filedata = '" . $db->escape_string($message['attachment' . $i]['body']) . "', visible = 1,
thumbnail = '" . $db->escape_string($thumbnail['filedata']) . "',
thumbnail_dateline = " . $thumbnail['dateline'] . ",
thumbnail_filesize = " . $thumbnail['filesize'] . ",
filesize = $filesize,
postid = $postid");
//logging("inserted with thumb");
}
else //and without
{
$db->query("INSERT INTO " . TABLE_PREFIX . "attachment SET dateline = '$date',
filename = '" . addslashes($message['attachment' . $i]['headers']['filename']) . "',
filedata = '" . $db->escape_string($message['attachment' . $i]['body']) . "', visible = 1,
filesize = $filesize,
postid = $postid");
//logging("inserted without thumb");
}
//logging("insertion complete");
}



... to always accept attachments.

Something like that. Those "Tornevall"-things is made for myself to find the code easier in case of patching, etc. :)

[ullo]

Can you tell me where is the functions_image.php that i have to replace ? I have not found it in the directory includes

[TMM-TT]

Can you tell me where is the functions_image.php that i have to replace ? I have not found it in the directory includes


I use one from 3.0.10.


<b>Parse error</b>: parse error, unexpected $ in <b>/home/httpd/vhosts/asian-99.com/httpdocs/groups/includes/functions_nntp.php</b> on line <b>1060</b><br />


Try the one I attached - that's the one I use and it should work.

[ullo]

Yes. Pardon My Terrible Stupidity.
Pardon Me If I Made Different Post.

When I Have A Problem I Lose My Mind And I Became Mad.

Now I Will Cancel All Post And I Try Your Script.

THANK YOU VERY MUCH.

[mdevour]

I've gotten some supportive replies to my message (https://vborg.vbsupport.ru/showpost.php?p=1081972&postcount=352) asking for the chance to help resurrect this mod. Lierduh has indicated he's willing to pass it on to somebody new, so long as it stays a no-cost hack. I agree with that!

I've printed out the php and xml files so I can start looking over the code and getting a feel for the scope of what we're working with. Fun... :tired:

What I need to know next is how you manage a modification thread here on vB.org? Is there a "Newbie's guide to publishing a hack?" I haven't found anything like it yet, except one sketchy announcement (https://vborg.vbsupport.ru/announcement.php?f=113&a=24) at the top of the 3.5 mods forum.

Otherwise, is there a developer out there who'd take a few minutes to talk me through the basics?

Do we start out in this thread or jump right in and start a new 3.6.1 thread and do all of our bootstrapping in plain view? Or launch a development thread someplace a little less obtrusive and wait 'til a solid beta is ready to put it up on the 3.6.1 forum? Has anyone seen how it's done?

As I said, I've got the time and interest to keep the files and documentation updated as you all fix bugs and add features. I can also work on the installation instructions, since I'll be doing a lot of installing and testing.

By the time we're ready to release for 3.6.1 (or .2) I'll probably be able to contribute my share of coding and support.

That much should let people start installing and using the hack again. Then we can step back and take a look at what we want to do next...

TMM-TT and Aylwin have both listed features they'd like to see added. I've my own list as well. If we can attract the interest of a couple more folks who've used or are using it, we may have enough momentum to make a go.

I hope so.

Answers, ideas, or suggestions anyone?

Thank you,

Mike D.

[TMM-TT]

TMM-TT and Aylwin have both listed features they'd like to see added. I've my own list as well. If we can attract the interest of a couple more folks who've used or are using it, we may have enough momentum to make a go.


Well, there's actually more to think over, like this:

If someone on your forum starts to abuse a newsgroup, you really like to remove posts from that user. The problem is that when a post has been sent to a newsgroup, you can't delete the post (only local), unless you send a cancel-message to the group. Deleting a post in a forum doesn't send any cancels, and it's too late to stop such message, as fast as it has reached usenet. I've been thinking of this before, how this would work if such function was added, that sends cancels to a group when something is deleted on the forum.

There's a few risks to think of too. Like admins that gone crazy and starts canceling everything in a group. But then, we have message id's. If something could identify the local post so cancels only can be made on posts that come from the forum, that could be possible to avoid. Cancels can always get bypassed, but if it can be made harder, moderating local users on usenet could be an idea too. Another problem is when someone on usenet reply a local "abuse", that cannot be moderated. Etc etc. :)

[mdevour]

There are similar challenges as you expand the mailing list gateway concept.

How do you keep spammers or trolls from forging the addresses of real members? And, of course, there is no way to cancel an e-mail once it has been sent out to your members... I hope there's a viable strategy with some encryption tricks and custom headers that will help. Lots to think about!

Still, what's the answer to any of these questions? Work through the logic of the problem, pick out whatever solutions might exist, if any, and present them to the user of the mod as options in Admin-CP! No problem! ;)

Have you found sources for the protocols for Usenet? I think we're each going to need to study the parts of this application we're personally interested in and share our research to help with the coding.

Be well,

Mike D.

[TMM-TT]

How do you keep spammers or trolls from forging the addresses of real members? And, of course, there is no way to cancel an e-mail once it has been sent out to your members... I hope there's a viable strategy with some encryption tricks and custom headers that will help. Lots to think about!

With canceling e-mails it's different. There's no actual solution for that as I know it. SMTP-servers only act on delivery, not on cancels. If that was what you meaned with the custom headers.. :)


Have you found sources for the protocols for Usenet? I think we're each going to need to study the parts of this application we're personally interested in and share our research to help with the coding.

Here's some of them. :)

Standard for Interchange of USENET Messages (http://www.w3.org/Protocols/rfc1036/rfc1036.html)
Network News Transfer Protocol (http://www.w3.org/Protocols/rfc977/rfc977)


I don't think everything in them are necessary, because some parts might be interesting only if you're building a new server (I haven't read it so I can't confirm on that)..

[John Diver]

Hey,

I'm not getting pictures, can pictures be included in posts? Almost all posts I'm getting have images.

Here is my forum for the posts: http://www.deuceace.com/forumdisplay.php?f=24

It also doesn't seem to be showing much text, am I able to change this or is it a limitation of what I am allowed to take from the sites?

[jruttenberg]

Just bought our license and got vb-2.6.1 running. Now I need to lure the old time mail and news junkies in. Can NNTP Gateway work with vb-2.6* If not now, when? Thanks.

[mdevour]

To John Diver:

There's the problem, John. This hack is dormant. The only support is from whatever other users there are that want to chime in. I don't have enough experience with the hack yet to know much of anything. The only thing to do, if nobody answers, is to search this thread and the one for the previous version and see if they say anything about a fix for your problem.

Me, I need the hack working, too. When I've had some more time to work on it I'll be of more help.

Jruttenberg:

The most recent version of the hack works on 3.5.0, mostly. You need to apply some bug fixes that are scattered through this thread but not included in the distribution files.

There are several threads for earlier versions that you could research and test. Just search the forum for 'NNTP gateway.'

Why are you working with the older version of vB? Server limitations? Too much customization to want to upgrade and rebuild, just now?

Be well,

Mike D.

[Unexplained.tv]

I am pretty sure he meant 3.6

[Aylwin]

Mike,

What's the status now? Have you been able to put anything together? Perhaps we could start a development thread somewhere and start contributing/beta-testing?

-Aylwin-

[esfron]

Mike,

What's the status now? Have you been able to put anything together? Perhaps we could start a development thread somewhere and start contributing/beta-testing?

-Aylwin-
The biggest issue in import is when clicking on the 'go to last post' button on forumhome, it says no thread specified and it leads to: /showthread.php?p=0#post0.

I didn't test export to usenet, mailing list, attachments.

[mdevour]

Dear Aylwin,

I've been trying to gain my sea legs asking about the process of posting hacks here. See this thread in vBulletin Questions. (https://vborg.vbsupport.ru/showthread.php?p=1090896)

I'm also working on another small hack as practice for some later parts of this project.

I will read over the code for this hack and make a first attempt at installing it soon. I'm ready for that now.

With that experience under my belt, I expect I'll be ready to get started here, and to ask for the help of as many of you as are willing, to gather all the code enhancements and bug fixes that are already in this thread and integrate them.

Maybe then it'll be time to put up a new thread in the 3.6.x hack forum, or else start a development thread in the vBulletin Questions forum to bring it up to 3.6.x compatibility or make the first stab at some new feature integration.

I've not abandoned the effort. I hope y'all will hold on while I get up to speed... Unless of course another leader would like to step up to the task! :laugh:

Be well,

Mike D.

[ullo]

I have tested the script with Attachments.
I have used the version changed by Torneval and i have also commented a line in functions_image (the line 224).
See my old post here : https://vborg.vbsupport.ru/showthread.php?t=92588&page=25

I don't seen great problems a part the types of attachment and the dimension of them. I hate to have images of great dimension and to have videos files (clips and similar). But at the moment i can't stop to download them. I have used an .htaccess file to not upload this type of file (i hate in particular video... ).
php_value post_max_size 180000
php_value upload_max_filesize 150000

But it not work.

I think that this problem with attachments depend on the type of hack that was written for an old version of vbulletin and to download images i need to use functions_image that was thought for the old version 3.0.

Using the old version is it possible to stop attachment of video and files of great dimension ?
The old hack , if i use it with vbulletion 3.0. worked good with attachment ???

[ullo]

Dear Aylwin,

I've been trying to gain my sea legs asking about the process of posting hacks here. See this thread in vBulletin Questions. (https://vborg.vbsupport.ru/showthread.php?p=1090896)

I'm also working on another small hack as practice for some later parts of this project.

I will read over the code for this hack and make a first attempt at installing it soon. I'm ready for that now.

Mike D.

I'm also interested in the project but i have not any experience in php. For me php code and a cat are the same...

[mdevour]

I'm also interested in the project but i have not any experience in php. For me php code and a cat are the same...

My php tends to resemble a little puppy, kinda fuzzy and likely to leak, but I feel about the same!! <grin>

Mike D.

[ullo]

Hey,

I'm not getting pictures, can pictures be included in posts? Almost all posts I'm getting have images.

Here is my forum for the posts: http://www.deuceace.com/forumdisplay.php?f=24

It also doesn't seem to be showing much text, am I able to change this or is it a limitation of what I am allowed to take from the sites?

Pictures can be included but it is not possible for me at the moment to apply the dimension limits fixed in vbulletin attachment setting. At the moment i have also the problem that the hack work only if you insert the photo in a database and not in a file.
An old post of KevinM contain some help for fix the dimension of the images but it not work for me now.
https://vborg.vbsupport.ru/showthread.php?t=65152&page=9

To include the photo you need to change functions_nntp.php by following the instructions of TMM-TT in the precedent page.

https://vborg.vbsupport.ru/attachment.php?attachmentid=53934&d=1159222924
and download the functions_image of an old version of vbulletin.

Then you need also to comment the line 224 of the functions_image.php file.

{
// $thumbnail['filedata'] = $DB_site->escape_string($thumbnail['filedata']);
}

[Aylwin]

I hate to have images of great dimension and to have videos files (clips and similar). But at the moment i can't stop to download them.
Wierd. I kinda have the opposite problem. I'm unable to get large attachments. If the email size is larger than around 2.5MB then gateway.php will hang. So I need to filter out all emails larger than 2.5MB to get this mod working properly on my site.

[ullo]

Wierd. I kinda have the opposite problem. I'm unable to get large attachments. If the email size is larger than around 2.5MB then gateway.php will hang. So I need to filter out all emails larger than 2.5MB to get this mod working properly on my site.

At the moment i think to exclude all attachment and use only "pure text", in fact my provider allow this setting. What system have you used to allow attachment ? In the original script attachment are not allowed. I have used the original functions_image and changed some code using Torneval instuctions.
https://vborg.vbsupport.ru/attachment.php?attachmentid=53934&d=1159222924

In any case i have not tested any attachments greater than 2.5 MB. Max is 1.5 MB.

Is it possible that your problem depend on php.ini ? Is it possible that depend on some vbulletin internal setting ? Have you contacted VBulletin support ?

[ullo]

Wierd. I kinda have the opposite problem. I'm unable to get large attachments. If the email size is larger than around 2.5MB then gateway.php will hang. So I need to filter out all emails larger than 2.5MB to get this mod working properly on my site.

Pardon my curiosity but. ... What system have you used to filter out email larger than 2.5 MB ? If you have use it , i can solve my problem. Have you used htaccess or changed some part of the code ?

[ullo]

The biggest issue in import is when clicking on the 'go to last post' button on forumhome, it says no thread specified and it leads to: /showthread.php?p=0#post0.

I didn't test export to usenet, mailing list, attachments.

Is it possible to change the template and avoid the problem, not showing this types of links in the template ? I think yes but i need to verify.



In Template Edit
Forum Home Templates - FORUMHOME: - forumhome_lastpostby
i have changed the link "goto" :

<a href="showthread.php?t=$lastpostinfo[lastthreadid]" title="<phrase 1="$lastpostinfo[lastthread]">$vbphrase[go_first_unread_in_thread_x]</phrase>"><strong>$lastpostinfo[trimthread]</strong></a>

Original :
<a href="showthread.php?$session[sessionurl]goto=newpost&amp;t=$lastpostinfo[lastthreadid]" title="<phrase 1="$lastpostinfo[lastthread]">$vbphrase[go_first_unread_in_thread_x]</phrase>"><strong>$lastpostinfo[trimthread]</strong></a>

[ullo]

My php tends to resemble a little puppy, kinda fuzzy and likely to leak, but I feel about the same!! <grin>

Mike D.

:p :p :p :up: :up: :up: :banana:

[KidCharlemane]

I seem to be having a problem with the hack only processing one group while the other sits empty. Has anyone else experienced this?

[mattpark]

Hello all.

Im having a problem with this mod at the moment.

Followed setup instructions to the T.

Ran the script.

All imported off the newsgroup, great!

A few hours later i go and setup the cron to run every 15mins.

It runs, and abou 95% of the messages are re-downloaded!

I've tried so many differnt ways of fixing this, but every time, when the script runs again after the initial sync it downloads most the messages again!

No errors, just hundreds of duplicates!

Stupmed? You can see for youself here - http://ukvoiptalk.com/forum/uk-telecom-voip/

Please help.

Matt

EDIT: Just an update to show you what my cron output was.

X-Powered-By: PHP/4.4.3
Content-Type: text/plain
Set-Cookie: bbsessionhash=0935c3032697dfb066954e0d894d42fa; path=/; HttpOnly
Set-Cookie: bblastvisit=1161047701; expires=Wed, 17 Oct 2007 01:15:01 GMT; path=/
Set-Cookie: bblastactivity=0; expires=Wed, 17 Oct 2007 01:15:01 GMT; path=/

[mattpark]

Pretty please guys, can someone give me a had with this?

Thanks
Matt

[mattpark]

Yep, still looking for a little pointer for this one if anyone is willing!?

[Aylwin]

Sorry, I have no idea what might cause this. Seeing as there are no replies so far, I'm guessing no one else has any idea either.

One thing you can check though. If you check NNTP Gateway Newsgroups from Admin Control Panel, what value do you have under Last Message? It should be the same as the last message you imported.

[mattpark]

yeah it is.

Well i deleted the forum and removed all synced posts.
Setup a new forum. Went in to admin CP and set the last post at 12000.

If i run gatway.php it starts importing from 0, and then sets the last imported post correctly. However, if i then run it again, it starts from 0 again even though the last post is incremented!

Thanks for you suggestion bud, but looks like im still stuck.

Matt

[Rik Brown]

Our killfile very often ends up killing the first message in a thread. In those cases, successive messages which have a "Re: " in the message title create new threads such as "Re: Here is my message...".

However, if there is no prior "Here is my message..." message on my system, I would rather NNTP_GW remove the "Re: " and just make the first message "Here is my message..." to which successive replies would append.

Has anyone already written a routine to accomplish this (I assume it would only take 5-10 lines of code) or is there an experienced PHP programmer willing to write it out in a few minutes time?

Thanks. -- Rik

PS:

Frankly, I would just assume remove the "Re: " from all messages as its not necessary for vb. One only needs it when replying back to Usenet.

Update: If you wish to remove the "Re: " from post titles and thread titles, below is a thread at vbulletin.com that shows how to accomplish this retroactively by updating the forum mysql database (backup first, of course):

http://www.vbulletin.com/forum/showthread.php?t=206982

Of course, if someone could adjust gateway.php to accomplish this during the import phase, that would be a better solution.

[babulgogoi]

Hi, am getting this error while running : gateway.php (I run it manually):

http://asiasrc.org/forum3.5/gateway.php

<br /><strong>Warning</strong>: fsockopen() [<a href='function.fsockopen'>function.fsockopen</a>]: php_network_getaddresses: getaddrinfo failed: Name or service not known in <strong>/includes/pop.php</strong> on line <strong>40</strong><br /><br /><strong>Warning</strong>: fsockopen() [<a href='function.fsockopen'>function.fsockopen</a>]: unable to connect to asiasrc.org:110 in <strong>/includes/pop.php</strong> on line <strong>40</strong><br />Success (0)
<br /><strong>Warning</strong>: fwrite(): supplied argument is not a valid stream resource in <strong>/includes/pop.php</strong> on line <strong>107</strong><br /><br /><strong>Warning</strong>: fgets(): supplied argument is not a valid stream resource in <strong>/includes/pop.php</strong> on line <strong>99</strong><br /><br /><strong>Warning</strong>: fwrite(): supplied argument is not a valid stream resource in <strong>/includes/pop.php</strong> on line <strong>107</strong><br /><br /><strong>Warning</strong>: fgets(): supplied argument is not a valid stream resource in <strong>/includes/pop.php</strong> on line <strong>99</strong><br />

any idea.. pls help.

[hostingforum-it]

hmm it stopped to work rgabbig fro newsgroup. If i try to run manyally
the getaway (as thi first time) it ask me to download the file...:alien:
Anyone know what happens?

[chris1979]

Does this work with version 3.6.2 or does anything similar exist for 3.6.2?

[ullo]

Does this work with version 3.6.2 or does anything similar exist for 3.6.2?

It work for me. A part allegated that not work for nothing (disable allegated)

[chris1979]

I've kind of got it working. Does anyone know how to customise the footer that goes on messages in the newsgroup when people from my forum reply?

[Domenico]

Well it stopped working for me after the upgrade from 3.5.4 to 3.6.2.

I now get the following error:

:/usr/bin/php -q /gateway.php

Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 92160 bytes) in /includes/class_bbcode.php on line 1045
Allowed memory size of 8388608 bytes exhausted (tried to allocate 0 bytes)

So any suggestions on how to fix this? Memory limit in php.ini is 48M

Thanks!

[deejayh]

damn, damn, damn!

Sorry that it is not working in 3.6+. Though it does look like a PHP memory problem!

[Domenico]

Please let someone pick it up. I would pay gladly for it!

Now it works but it keeps adding the same message over and over again. :(

[chris1979]

It's working fine for me in 3.6.2 now

[Domenico]

It's working fine for me in 3.6.2 now

How/what did you fix?

[chris1979]

I didn't change or fix anything - once I followed the instructions, it worked.

The only problem I'm having is getting the gateway.php script to run on schedule -- but I think that's a separate issue.

[ullo]

I didn't change or fix anything - once I followed the instructions, it worked.

The only problem I'm having is getting the gateway.php script to run on schedule -- but I think that's a separate issue.

Also for me the internal VBulletin Schedule system not work but if i use an external Scheduler (for example the scheduler of plesk) it seem to work.

[Domenico]

I didn't change or fix anything - once I followed the instructions, it worked.

The only problem I'm having is getting the gateway.php script to run on schedule -- but I think that's a separate issue.


What instructions?

[donnacha]

Okay, when people say that it's not working for them, can they please explain exactly what is and is not working.

I've had it installed for about 10 months and, in tests at that time, everything was working just fine with regard to both grabbing usenet groups AND posting. No, after upgrading to 3.6.4, a quick check reveals that posts made from within my forum are NOT actually getting posted to Usenet although they ARE appearing in my forum.

Is this a known problem with more recent versions of VB? For all I know, this might actually have started before I upgraded, I just don't know.

Anyone experiencing this particular problem?

[KW802]

Before I re-invent the wheel... does anybody have a little cleanup script to remove duplicate posts?

In the vB ACP there is the Remove Duplicate option to remove duplicate threads but it doesn't handle duplicate posts.

Anybody? :)

[Rik Brown]

I've had it installed for about 10 months and, in tests at that time, everything was working just fine with regard to both grabbing usenet groups AND posting. No, after upgrading to 3.6.4, a quick check reveals that posts made from within my forum are NOT actually getting posted to Usenet although they ARE appearing in my forum.

Is this a known problem with more recent versions of VB? For all I know, this might actually have started before I upgraded, I just don't know.

Anyone experiencing this particular problem?

Donnacha:

I just upgraded to vb 3.6.4 and exported messages in debug mode. It showed the messages being exported to my newsgroup server. I also logged onto the newsgroup server with my newsgroup reader and confirmed that my messages were there and that nothing looked strange in the headers, etc. Hopefully, it was just a temporary glitch for you?

Regards. -- Rik

[KW802]

Before I re-invent the wheel... does anybody have a little cleanup script to remove duplicate posts?

In the vB ACP there is the Remove Duplicate option to remove duplicate threads but it doesn't handle duplicate posts.

Anybody? :)Never mind. I ended up just creating a new "Remove Duplicate Posts" function in the ACP and that is doing the job fine. Taking forever to loop 1MM+ posts but it is working.

[donnacha]

Donnacha:

I just upgraded to vb 3.6.4 and exported messages in debug mode. It showed the messages being exported to my newsgroup server. I also logged onto the newsgroup server with my newsgroup reader and confirmed that my messages were there and that nothing looked strange in the headers, etc. Hopefully, it was just a temporary glitch for you?

Regards. -- Rik

Thanks for the feedback Rik, much appreciated.

Donnacha

[tpearl5]

Has anyone figured out a way to use regex or some other means to filter out certain spam? I'm getting a large amount of viruses and other crap coming through my feeds which is guming up the database. It takes a long time to delete these threads (1 - 5 min EACH)

Please help.

[mattpark]

Ladies and Gents,

I am offering $50 for the person who can resolve an issue with this mod, and update for 3.6

Please see this thread - https://vborg.vbsupport.ru/showthread.php?p=1145373#post1145373

[KW802]

Has anyone figured out a way to use regex or some other means to filter out certain spam? I'm getting a large amount of viruses and other crap coming through my feeds which is guming up the database. It takes a long time to delete these threads (1 - 5 min EACH)

Please help.Did you try the killfile settings? (ACP => NNTP Gateway Options => NNTP Gateway Settings => Global Killfile) I have a few entries in my and it seems to be working as expected.

Ladies and Gents,

I am offering $50 for the person who can resolve an issue with this mod, and update for 3.6

Please see this thread - https://vborg.vbsupport.ru/showthread.php?p=1145373#post1145373 Check your newgroup properties to see if the last message ID is getting correctly updated; if that that'll at least help narrow down where the problem is occuring. How many messages are you trying to download? There is a chance that if you're trying to bring in a new group that the cron job runs once, times out (which doesn't update the last message ID), run again, and tries to bring in all of the messages again.

[Rik Brown]

I'm guessing that about 95% of the messages that we import with the same title get properly merged into the prior thread of the same title. So perhaps 5% get orphaned and I have to merge them manually.

Is anyone getting a 100% proper merging of threads of the same thread title over time?

Thanks. -- Rik

PS:
I'm asking because if others are getting a 100% success rate, it may be a problem with our server (Although I've played around with setting delays between messages which doesn't, however, seem to affect things) and so it may be upgrade time.

[KW802]

Is anyone getting a 100% proper merging of threads of the same thread title or time?For the most part. The only problem I have yet is where if a new thread arrives with a subject of something like "RE: Whatever..." and "Whatever..." doesn't already exist then it seems like it keeps creating new "RE: Whatever..." threads with each import (as opposed to keeping them all together). I haven't looked at the code lately in an attempt to fix it.

[woodp]

I'm running vB3.6.4 and Mailman 2.1.9 and using this script to provide gateways between vB and email.

vB->email: Took a while to figure out all the Mailman idiosyncrasies but this part of the script is running perfectly. I'm certain my users will love it.

email->vB: Struggling with this. I can send an email to MailMan's pop account and it will a) get properly reflected to all users, and b) show up in the MailMan archive. Or I can "reply" to an email from the gateway and it will also a) get properly reflected to all users, and b) show up in the MailMan archive.

What isn't working is vB reading the POP3 account and posting the "email" or "reply" into the message board thread. I'm not certain what might be going on except that MailMan is immediately reading the POP3 account, then clearing it and vB has nothing to import? What else could be going on?

Any ideas out there?

(Noob here - How do I click INSTALL to indicate I'm running this script?)

[Rik Brown]

For the most part. The only problem I have yet is where if a new thread arrives with a subject of something like "RE: Whatever..." and "Whatever..." doesn't already exist then it seems like it keeps creating new "RE: Whatever..." threads with each import (as opposed to keeping them all together). I haven't looked at the code lately in an attempt to fix it.

Thanks for the feedback.

Regarding the "Re: " problem, I have that too. But I've been running a query against our forums after each mail exchange to remove the "Re: " in current vb thread titles before the next NNTP mail exchange. It seems to help somewhat in this respect, but not 100%.

If I get to understand PHP coding better, I'd like to alter the mod to remove the "Re: " part from all incoming titles as vb doesn't need them (and it looks better without them).

Thanks. -- Rik

[kjhkjh]

Donnacha:

I just upgraded to vb 3.6.4 and exported messages in debug mode. It showed the messages being exported to my newsgroup server. I also logged onto the newsgroup server with my newsgroup reader and confirmed that my messages were there and that nothing looked strange in the headers, etc. Hopefully, it was just a temporary glitch for you?

Regards. -- Rik

Have other people found this easy to install and get working with vb 3.6.4?

I think that I'm gonna try it now. Is it just the same file that I have to use as the 3.5 that is downloable from the top post?

Thanks.

[Rik Brown]

Have other people found this easy to install and get working with vb 3.6.4?

I think that I'm gonna try it now. Is it just the same file that I have to use as the 3.5 that is downloable from the top post?

Thanks.

It is easy to install. However, there are many mods that have been applied to this mod since the last release. You will likely want to go through the notes in this thread and apply most of the mods created by dkendall and a few others as well. It may well be that people here are operating many different versions of this mod.

As for vb 3.6.4, this mod basically works well for me but I only exchange Usenet newsgroups and do not utilize features for binary files or maillists. Others would have to give you feedback there.

Regards. -- Rik

ps: Its definitely worth the effort to get it going. We went from a few thousand messages to 450,000 online in one month.

[LPH2004]

If I use vbulletin cron then the scheduled tasks stop but using the plesk interface to setup the job fails with errors from line 1-6 about files not existing. I can manually run the gateway.php file from the browser and there is an import.

Any thoughts on the matter?

Update: The error messages:

full/path/of//gateway.php: line 1:
?php: No such file or directory
full/path/of//gateway.php: line 2: :
command not found
full/path/of//gateway.php: line 3: P:
command not found
full/path/of//gateway.php: line 4:
Release: command not found
full/path/of//gateway.php: line 5:
*/: No such file or directory
full/path/of//gateway.php: line 6:
/********************************************: No such file or directory
full/path/of//gateway.php: line 7:
syntax error near unexpected token `Usenet/Newsgroup'
full/path/of//gateway.php: line 7: `
NNTP (Usenet/Newsgroup) Gateway 1.9'

Update In Case Others See This:

The first line comes from <?php ...
The other lines are from comments in the gateway.php script. But the script does not seem to run so there must be something that I set wrong in the plesk interface.

[woodp]

full/path/of//gateway.php: line 1:
?php: No such file or directory

That sure doesn't look like a proper path ...

Shouldn't it be more like ./gateway.php assuming you dropped the scripts in your /forum/ subdirectory?

[LPH2004]

The cronjob is definitely running every 15 minutes but with the errors and so the php file does not execute. I updated my message with the ideas that the comments in the gateway.php file are causing the error. I just removed those comments from the file and will see what the next run through says. This is what was entered into the plesk interace. Do I also need to tell it about php?

0,15,30,45 * * * * /full/path/gateway.php

Thanks for the help.

Update: OK. After removing the comments from the script ... these are the 'new' errors

full/path/of//gateway.php: line 1:
?php: No such file or directory
full/path/of//gateway.php: line 2: :
command not found
full/path/of//gateway.php: line 3:
syntax error near unexpected token `E_ALL'
full/path/of//gateway.php: line 3:
`error_reporting(E_ALL & ~E_NOTICE);'

So - the php being interpretted. Hmm. Google search time

[woodp]

0,15,30,45 * * * * /full/path/gateway.php


Nope, there's no way a normal UNIX setup is going to place files at /full/path - Just ain't gonna happen.

Please provide the full path to your forums and the full path to your gateway.php script. If you can telnet, simply type 'pwd' from the forum subdirectory.

I'm guessing it's something like /home/{youruserid}/public_html/forum ...

[LPH2004]

Thank you. I figured out the correct line now. The php interpreter does need to be stated prior.

0,15,30,45 * * * * /full/path/to/php /full/path/gateway.php

The space between is important. Another way would be for the user to add the information into the gateway.php script. Something like

#!/usr/local/bin/php

as the first line.

Now, I am a happy camper :glasses:

Thank you for all of your help.

Oh - additional instructions in case someone else trips over this post...

If you need to find the path of your php interpreter then use the php info in the admincp.

[bashy]

Hi

What do i need to have this up and running, Does it have to be used for posting, can it just be used for downloading the binaries?

I am a little baffled with regards to setup requiering email addresses as so on...

In the server setup side of it, what do i put in here "Newsgroup/Mailing list E-mail" please?

Can i use NTL's free server for this news.ntlworld.com?

[growler]

I'm running 3.6.4,

What file do I add the two (construct_nav_option) lines from the installation instructions to? I've looked in a few files and don't see:

Edit ./admincp/index.php file

construct_nav_option($vbphrase['vbulletin_options'], 'options.php?null=0', '<br />');

-Thank you,
Growler

[kjhkjh]

ok I have my news server that I'm paying for: news.tigerusenet.com
I have a username and password for it.

I setup the forum and the nntp settings
I've uploaded the files but when I run the debug, I get the msg:

Gateway version 2.3.2 1 group(s) gatewayed.
Could not connect to NNTP-server. (Permission denied (13))

Not connected
Not connected

Now i thought that this might be because my hosting company is blocking the ports.
I spoke to them and they wanted the ip and port so that they can open it up.

Now I'm guessing that the IP is what news.tigerusenet.com translates to (66.250.146.196). Is this correct?

Then I told them port 119, and they said that they cannot open this up on shared hosting plans...

Can anyone help me get this up and running?

All I'm after is to be able to fill my forum up a little more with posts. So I don't even need forum members to be able to post to usenet.

It's so frustrating.

Please help

[KW802]

...
Now i thought that this might be because my hosting company is blocking the ports.
I spoke to them and they wanted the ip and port so that they can open it up.
...
Then I told them port 119, and they said that they cannot open this up on shared hosting plans...
...Wait... your hosting will or won't open up port 119 for you? If they won't open up port 119 then you'll need to see if your NNTP provider offers feeds on other ports like port 80 (and then you'll need to change the port # hard-coded in the PHP file but one step at a time).

[kjhkjh]

Thanks for your reply.

My host won't open up port 119.

I checked tigerusenet.com (the company I'm using to subscribe to the news server) and it says:

Port(s): 119, 80, 3128, 23, 7000, 8000, 9000

Now it doesn't say if I need to change options or anything there. These numbers are just in the corner so I guess that I can connect on any.

Do you know which is the best port to use there? port 80?

Do I just need to go through each of the php files and change 119 to, say 80?

Thanks for your help...

[kjhkjh]

...you'll need to see if your NNTP provider offers feeds on other ports like port 80 (and then you'll need to change the port # hard-coded in the PHP file but one step at a time).

I just changed the details in the php files from 119 to 80. ran the gateway.php script and got the same error message.


This is really strange.
I feel like this is an amazing hack and I'm so close just can't get it to work and am lost about the reason?!

Is it the files? Is it the news server (tigerusenet)? is it my hosting company? Is it me?

I have no idea...

[bashy]

If all you want to do is fill ya forum with posts, perhaps using the free Microsoft account, no user or pass required, theres a good few groups as well, This is what i have done for now :)

[kjhkjh]

I think the problem is my hosting company blocking port 119.

So even the microsoft news forums wouldn't work at this time. Unless I can get the hack to work on a different port.

Has anyone else had this same problem with their host? Any work around changing the ports in this script. I've kinda tried changin every occurance of 119 with 80 in the php files, but it still doesnt work?

Any advice

[KW802]

I think the problem is my hosting company blocking port 119.

So even the microsoft news forums wouldn't work at this time. Unless I can get the hack to work on a different port.

Has anyone else had this same problem with their host? Any work around changing the ports in this script. I've kinda tried changin every occurance of 119 with 80 in the php files, but it still doesnt work?

Any adviceIt's a double edged sword you're fighting here... on one side you have your hosting company who won't open up port 119 which is the standard NNTP port while on the other hand you have to find a NNTP provider who supports different ports like 80.

Not every NNTP provider supports port 80 so you need to find out from them what other ports, if any, they offer connections on and then you need to take that list to your hosting company to see which ports they have open. You may or may not find any common ports. You just can't blindly change the PHP file to different port numbers because (a) your hosting company may be blocking it but even if they aren't then (b) your NNTP provider may not offer that port as an option.

Take a step back with this...

- Go to your NNTP provider and tell them that your hosting company is blocking port 119 and then ask if they (the NNTP provider) offer connections on port 80 since that port is open for just about every hosting company.

- If your NNTP provider offer connections on port 80 then try to get the software up & running with a test newgroup first.

- If your NNTP provider does not offer connections on port 80 then try to find a free open access NNTP provider that offers connections on port 80. Most likely the newsgroup you want won't be available but at leat you can get the software up & running as a proof of concept and go from. After you have it up & running then you can shop around for a different NNTP provider.

[bashy]

What i meant was try the msnews.microsoft.com server not your hosting server, add that one as a new server and then add microsoft.public.pocketpc as a group and see if that works..

My newshosting account didnt work for me either so i tried the default server and forun a decent group in that server and hey presto it works...

I am referring to the server that you changed to yours when you 1st set it up, itshould be a microsoft one im sure, but anyway, use them details i have given you and it will work, let me know please, if ya dont quite understand what i mean let me know also and i will try to explain better :)

I think the problem is my hosting company blocking port 119.

So even the microsoft news forums wouldn't work at this time. Unless I can get the hack to work on a different port.

Has anyone else had this same problem with their host? Any work around changing the ports in this script. I've kinda tried changin every occurance of 119 with 80 in the php files, but it still doesnt work?

Any advice

[familyhistory]

I have exactly the same problem! My host will not allow NNTP ports to be open and block all ports!

As I can get them with a newsreader (outlook or others), could there be away of downloading them all every so often then exporting them somehow into mysql??!!

OR,
Maybe I could run a copy of Vb locally on my pc using WAMP, then copying the relevant database onto the live website??? Need to do this a lot though!

What do you reckon?

Thanks,:)
Dave

[kjhkjh]

It's a double edged sword you're fighting here... on one side you have your hosting company who won't open up port 119 which is the standard NNTP port while on the other hand you have to find a NNTP provider who supports different ports like 80.


Thanks KW802, I appreciate your help and comments... I'm well on my way with this now and think that when I have more time I can get it to work how I want.

What i meant was try the msnews.microsoft.com server not your hosting server, add that one as a new server and then add microsoft.public.pocketpc as a group and see if that works..


Thanks Bashy. I have found another way to get this working at the moment. I'm just not sure that what you suggest would work either though. Basically I'm guessing that this is the problem in easy terms (someone let me know if I'm wrong)... The script is being run from a php file on my website, it requests access to ports on a newsserver (that could be a free one or a pay for account), so long as it gets a connection (and authenticates - if its a pay for newsserver with login) then the messages are posted in the forum.

With my original problem being my web hosting not allowing connection to be opened for outgoing requests on port 119, this would affect the free and pay for newsservers connecting to usenet groups.

I have exactly the same problem! My host will not allow NNTP ports to be open and block all ports!

As I can get them with a newsreader (outlook or others), could there be away of downloading them all every so often then exporting them somehow into mysql??!!

OR,
Maybe I could run a copy of Vb locally on my pc using WAMP, then copying the relevant database onto the live website??? Need to do this a lot though!

What do you reckon?

Thanks,:)
Dave

Dave, I was trying to figure out a way to do exactly that - download the messages locally, then upload them to my forums. I'm not all that bothered with forum members interacting with usenet - I just want to fill my board with more posts. But I couldn't find a way to do this.

I did solve the problem of getting the connection working on a different port, and have managed to import messages. Only problem is with some of the titles, and also the links on the forumhome to the last message in each forum isn't working.

Anyways, 1 problem at a time. Here's how I got my web hosting company to open ports and connect to my newsserver. (as KW802 suggested):

I'm registered with http://www.tigerusenet.com/ (I think it's $7.50 a month - for 15GB / month)
This newsserver allows access on ports: 119, 80, 3128, 23, 7000, 8000, 9000

I had to then keep pushing my hosting company to open port 7000 (because they wouldn't open port 119 on a shared hosting plan) eventually they did it. It took 24 hours for them to process this change. They needed to know the ip address of the where the open port was connecting to so I did an ip lookup for news.tigerusenet.com

I then went through each of the installation files and replaced all instances of "119" with "7000" (only about 3 files needed changing, but i went through them all to make sure - use the easy edit>find>replace funtions of your text editor)

then after running the script (after putting in the settings) and it sucked the messages to my forum.

I still have some issues with it. (Like I haven't been able to repeat the task of sucking in the messages, but I haven't spent time trying to figure this out yet, plus I guess this has something to do with schedul tasks - just getting error messages) I will read through the posts here again.

But I'm optimistic because I sucked them in.

Give it ago, I'm not too sure how good tigerusenet is, but there is no contract, so if they end up being bad, I will cancel and change.

I know that if I was to allow 2 way posting to usenet, then I would probably have problems with this newsserver (maybe most are the same) - because each post to usenet would be from my account. And each instance of my account spamming usenet over tigernet is gonna get me a fine... so I disabled my forum members post from going on usenet (someone earlier in the thread suggested this).

Give it a go by seeing which ports you can use your newsserver on (eg. 7000?) then go to your webhost and ask to open port 7000 for ip (of your newsserver), then change all instances of 119 with 7000 in the install files (port 119 is the default).

Let me know how you go, good luck!

[KW802]

...
I'm registered with http://www.tigerusenet.com/ (I think it's $7.50 a month - for 15GB / month)
This newsserver allows access on ports: 119, 80, 3128, 23, 7000, 8000, 9000

I had to then keep pushing my hosting company to open port 7000 (because they wouldn't open port 119 on a shared hosting plan) eventually they did it. It took 24 hours for them to process this change. They needed to know the ip address of the where the open port was connecting to so I did an ip lookup for news.tigerusenet.com ...To make your life easier in the future I suggest using port 80 instead. Port 80 is the port that HTML (www.yoursite.com) goes over so it's almost a 'universal' port number. By using port 80 instead of 7000 then if you switch your hosting service to a different company in the future then you'll know that you don't have to go through this all over again with asking them to open up a different port number.

[bashy]

Hi

I have had port 119 enabled on my dedicated server but i still get
Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for uk.media.tv.misc at news.ntlworld.com:

Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for alt.binaries.movies.divx at news.ntlworld.com:

This happens whenever i try to connect to the news.ntlworld.com (free server)
Or when i try to connect to my paid newshosting server

Any thoughts on this please?

[TMM-TT]

Hi

I have had port 119 enabled on my dedicated server but i still get
Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for uk.media.tv.misc at news.ntlworld.com:

Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for alt.binaries.movies.divx at news.ntlworld.com:

This happens whenever i try to connect to the news.ntlworld.com (free server)
Or when i try to connect to my paid newshosting server

Any thoughts on this please?
Do you get such errors even if you login manually?

[bashy]

Sorry, can you elaborate, not quite sure what ya mean by login manually?

[KevinM]

Hi

I have had port 119 enabled on my dedicated server but i still get
Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for uk.media.tv.misc at news.ntlworld.com:

Connecting to server, server says: 502 Permission Denied - Permission Denied -- http://www.highwinds-software.com/ (Tornado v1.0.6.380)

Info for alt.binaries.movies.divx at news.ntlworld.com:

This happens whenever i try to connect to the news.ntlworld.com (free server)
Or when i try to connect to my paid newshosting server

Any thoughts on this please?

When you say free, do you mean the service which you get with your broadband provider on your home pc? If so, they probably perform an IP check and when it is not coming from a non-NTL line (which your ded server won't), it won't connect. The first time I got this hack I tried using the free BT service I got (and failed).

If you are getting this with a paid service (e.g. giganews, teranews etc), then I would guess the most common reason is username & password issues. Can you log in with these on a normal newsreader? The fact you are getting permission denied errors means the port is problably enabled correctly, or you wouldn't have got this far. What paid service have you gone for?

[bashy]

I am using newshosting, have been for about a year now, i copied and pasted the login details so theres no mistyping..... The only server i have managed to connect to is the free Microsoft 1

[TMM-TT]

Sorry, can you elaborate, not quite sure what ya mean by login manually?
Manually with, for example, a news client. I was thinking of the 502-message, that may indicate missing username/password.

[bashy]

ah right, yeah im downloading with grabit as we speak lol

[Rik Brown]

For aesthetic reasons, I prefer not to have the "Re :" in the subject line created by incoming Usenet threads/posts even if it is a reply. vBulletin doesn't use such formatting and any reply post leaving our system via the gateway will get the "Re: " pre-appended to the subject line for Usenet compatability.

If you likewise wish to remove the incoming "Re: " from the subject line, find the following line in gateway.php:

$subject = htmlspecialchars(trim($message['subject']));

After the above line, simply insert the following:

if (substr($subject, 0, 4) == 'Re: '){
$subject = substr_replace($subject, '', 0, 4);
}

If you wish to change your entire message database to remove any prior "Re: " in both threads and posts, just look for my prior post #396 in this thread which indicates the two mySQL queries to do so. Once done, all your threads/posts will be in this same format.

Again, this is only an aesthetic choice for me and shouldn't be considered a bug fix. I hope others might find it useful. If anyone sees any code conflict here, please advise.

Regards. -- Rik

ps: I'm not a PHP programmer so anything I say must be taken at your own risk. Backup your database first. But I've been using this change for several weeks now and all appears working fine.

After the above coding, NNTP_GW attempts to match the incoming post to vb threads "by reference" or "by subject." I'd prefer to make that a single lookup by subject at a future time as threading is already mangled if you are using a killfile that breaks thread sequences. Note, for example, that if the first message in a thread was deleted by your killfile, message #2 will now start a new thread without any "Re: " in it.

[TMM-TT]

if (substr($subject, 0, 4) == 'Re: '){
$subject = substr_replace($subject, '', 0, 4);
}


How about a single line?

$subject = preg_replace('[^Re: ]', '', $subject);

[Rik Brown]

How about a single line?

$subject = preg_replace('[^Re: ]', '', $subject);

Great! I'm just starting out with PHP and any tips are appreciated. Thanks. -- Rik

[TMM-TT]

Great! I'm just starting out with PHP and any tips are appreciated. Thanks. -- Rik
Another example that should fix all Re:'s no matter if it's in upper or lower-case. :)

$subject = preg_replace('/^re: /i', '', $subject);

[jgrtap]

Has anyone managed to get this to work using the internal scheduler in vBulletin? We're a small board on a shared server hosted at GoDaddy and I'm not sure we have access to an external scheduler...

[KW802]

Has anyone managed to get this to work using the internal scheduler in vBulletin? We're a small board on a shared server hosted at GoDaddy and I'm not sure we have access to an external scheduler...What kind of problem are you having?

[jgrtap]

What kind of problem are you having?For 15 hours yesterday and this morning, the scheduled task (retrieving messages forwarded to an email account from a Yahoo group) ran ONLY if I ran the task manually. Now, all of a sudden, it seems to have started working. Let me give it 24-48 hours to test more thoroughly and I'll be back... (sounds like a threat, doesn't it?)

Thanks...

-- jgr

[Rik Brown]

Ignore.

[TMM-TT]

For 15 hours yesterday and this morning, the scheduled task (retrieving messages forwarded to an email account from a Yahoo group) ran ONLY if I ran the task manually. Now, all of a sudden, it seems to have started working. Let me give it 24-48 hours to test more thoroughly and I'll be back... (sounds like a threat, doesn't it?)

Thanks...

-- jgr
I've had the same problem but not only for this hack - all scheduling seem to have stopped here too.

[jgrtap]

I've had the same problem but not only for this hack - all scheduling seem to have stopped here too.I'm totally baffled. The gateway works, no problem. But the scheduler is a disaster. It starts, it stops, no error messages, no nothing. I can't figure it out for the life of me.

Is there an external scheduler for a GoDaddy shared hosting account?

[Rik Brown]

I'm totally baffled. The gateway works, no problem. But the scheduler is a disaster. It starts, it stops, no error messages, no nothing. I can't figure it out for the life of me.

I hate to say it but the vb scheduler has worked flawlessly for us over the past several months that we have had this mod installed.

I have the log entries disabled and we run the gateway.php script hourly. I can't think of any reason it would operate manually but not operate under the scheduler except some conflict with the logging because almost all other options are simply for times/day, etc.

Sorry, I know that doesn't help much except to say that it can work flawlessly via the scheduler.

Regards. -- Rik :o

[jgrtap]

I hate to say it but the vb scheduler has worked flawlessly for us over the past several months that we have had this mod installed. I have the log entries disabled and we run the gateway.php script hourly. I can't think of any reason it would operate manually but not operate under the scheduler except some conflict with the logging because almost all other options are simply for times/day, etc. Sorry, I know that doesn't help much except to say that it can work flawlessly via the scheduler.As long as it CAN work, I'll keep plugging away, Rik, and I appreciate the info -- it gives me some hope!

[jgrtap]

I hate to say it but the vb scheduler has worked flawlessly for us over the past several months that we have had this mod installed. Rik, to double check, would you mind telling me exactly what the settings are that you have in nntp_groups, nntp_settings (substituting for any private info) and the scheduled task manager? I'd appreciate it.

[KW802]

I'm totally baffled. The gateway works, no problem. But the scheduler is a disaster. It starts, it stops, no error messages, no nothing. I can't figure it out for the life of me. By any chance, are you trying to bring in all of the articles in a particular group or groups? The script will time itself out after 30 minutes so if you're trying to bring in thousands of articles and the script is still running for greater than 30 minutes the next time the scheduled task runs then it'll kill the prior task and try over again.

[jgrtap]

By any chance, are you trying to bring in all of the articles in a particular group or groups? The script will time itself out after 30 minutes so if you're trying to bring in thousands of articles and the script is still running for greater than 30 minutes the next time the scheduled task runs then it'll kill the prior task and try over again.No, this is a very small email gateway for a private forum on one side and a small Yahoo group on the other. We might be talking about 200 messages in a busy week.

[familyhistory]

I have found a news supplier, who will accept port 80. I have changed the code in gateway.php
But I get the following errors:
Database error in vBulletin 3.6.3:

Invalid SQL:

SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle, post.visible AS postvisible,
thread.visible AS threadvisible
FROM vbpost as post LEFT JOIN vbthread as thread
ON (thread.threadid = post.threadid
AND post.userid = thread.postuserid
AND post.postid = thread.firstpostid)
WHERE post.isusenetpost = 0
AND post.postid > 966
AND thread.forumid = 67;

MySQL Error : Lost connection to MySQL server during query
Error Number : 2013
Date : Sunday, January 21st 2007 @ 02:08:15 PM
Script : http://www.forum.xx.com/gateway.php?debug=1
Referrer :
IP Address : 82.xxxxxx

Any help on this please,:)
Dave

[Rik Brown]

Rik, to double check, would you mind telling me exactly what the settings are that you have in nntp_groups, nntp_settings (substituting for any private info) and the scheduled task manager? I'd appreciate it.

I don't have a screen capture utility installed (a company no-no). But here is a sample setting for a newsgroup:


Newsgroup/Mailing list E-mail [alt.pandas ]
Forum [-- Pandas Forum ] <- vBulletin Forum
Prefix [ ]
Last Message [ ]
Server [news.yournewsserver.com ]
User name (only if required) [your_user_name_here ]
Password (only if required) [your_password_here ]
Enter 'use_real_email' if you want the [ ] <- I leave blank
posts to use the real email addresses.
Enter 'My Name <myemail@a_domain.com>' if you want all the posts use this name and email.
Please leave this empty if unsure!!!
Enabled Yes No <- Your choice


Except to change URLs to reflect our own site, we have used the defaults in the gateway settings except perhaps the following (set to 1 or zero):

Override useragent and organization checks to import absolutely everything from USENET. 1 = Full Import from oldest existing message, 0 = Ignore and start at next message.
Full import from USENET full_import_from_usenet 0

Use the email address to associate postings with the corresponding vBulletin user.
Associate by Email associate_by_email 1

Skip the message if the message has the X-No-Archive header set to yes. Google Group honours this Non-standard header. Set to 1 to honour. 0 to ignore.
Honour No-Archive honor_no-archive 0 [Edit]

When it is set to 1, no signatures will be sent with posts. When it is set to 0, users' signature selection at post is honoured.
Send no signature nosignature 1 [Edit]

Set to 1 : no profile/view thread footer will be sent. Set to 0 : attach user profile and view forum thread footer with each message posted.
Send no footer nofooter 1

Default = 0. Set to 1 : Time stamp for imported messages use the original post time. Set to 0 : Time stamp uses the time when the messages is imported
Use Post Time use_post_time 1

The vb scheduler looks like:


Varname gateway_varname
Title [Translations] NNTP Gateway
Description [Translations] Usenet newsgroups gateway mailer.
Log Phrase [Translations] NNTP Gateway
Day of the Week (Note: this *
overrides the 'day of the
month' option)

Day of the Month *

Hour *

Minute 30 - - - - -

Active yes
Yes No

Log Entries no
Yes No

Filename ./gateway.php

Product vBulletin


I hope the above helps. We are only importing newsgroups so we haven't tried binaries or mailinglists.

Regards. -- Rik

ps: Sorry, the gateway options order above may be different as we may have sorted them.

[jgrtap]

We copied our install onto a dedicated server so we could see the Apache error logs and the immediately obvious error was having gateway.php installed in includes/cron causing it to fail trying to require files from includes/cron/includes/cron thus causing cron.php to fail.


Filename ./gateway.php


We've corrected the task to use ./gateway.php -- and now the forum doesn't respond. (sigh)

[Rik Brown]

We copied our install onto a dedicated server so we could see the Apache error logs and the immediately obvious error was having gateway.php installed in includes/cron causing it to fail trying to require files from includes/cron/includes/cron thus causing cron.php to fail.

We've corrected the task to use ./gateway.php -- and now the forum doesn't respond. (sigh)

Forget cron for the time being. You need to have the gateway.php file installed in your vb forums root directory and functions_nntp.php, mime.php, and pop.php in the "includes" subdirectory beneath your forums root directory (and all other files in their correct directories per the install instructions).

Then run gateway.php in debug mode such as vb.com/forums/gateway.php?debug=1. You should see some sort of output that should help indicate if you have any further problems. But it just looks like you need to get the directory structure correct first. Then the vb scheduler should also work since ./gateway.php is referring to executing the gateway.php file in your root vb forum directory.

-- Rik

[gbechtel]

I am receiving this error:

Database error in vBulletin 3.5.4:

Invalid SQL:
UPDATE nntp_settings
SET value = WHERE varname = 'last_postid';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 2
Error Number : 1064
Date : Saturday, April 8th 2006 @ 10:18:32 PM
Script : http://.../forums/gateway.php
Referrer :
IP Address : ...
Username : ...
Classname : vB_Database


I am getting the same type of error message. I am not seeing the tables in my vb database

Mine...

Database error in vBulletin 3.5.4:

Invalid SQL:

SELECT post.*, thread.*,
post.dateline AS postdateline, post.msgid AS postmsgid,
thread.title AS threadtitle, post.visible AS postvisible,
thread.visible AS threadvisible
FROM vb3_post as post LEFT JOIN vb3_thread as thread
ON (thread.threadid = post.threadid
AND post.userid = thread.postuserid
AND post.postid = thread.firstpostid)
WHERE post.isusenetpost = 0
AND post.postid >
AND thread.forumid = 93;

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'AND thread.forumid = 93' at line 11 Error Number : 1064
Date : Tuesday, January 23rd 2007 @ 04:50:20 PM
Script : http://www.domain.com/forums/gateway.php?debug=1
Referrer :
IP Address : *.*.*.*
Username : *********
Classname : vb_database

[jgrtap]

Then run gateway.php in debug mode such as vb.com/forums/gateway.php?debug=1. You should see some sort of output that should help indicate if you have any further problems.We get the missing-value SQL error.
SET value = WHERE varname = 'last_postid'This message appears to be a consequence of having NO postings in the section.

[leeherron]

Jul 2005? What ever happened to this mod? Has anyone successfully installed it on 3.6.4?

[gbechtel]

I got mine running by putting the number 1 in the field for last_postid

Now to work on the killfile, does anyone have one they are willing to share. I love the mod but the spam on USENET makes it worthless unless I can filter most of it out.

[leeherron]

It's a great mod. I was just amazed to see it hadn't been updated since Jul 05! Working on updating my old setup (installed NNTP on that one back in '02) and would love to install this on my stage box but wasn't sure if it worked on 3.6.4 - thanks for the tip.

[KW802]

It's a great mod. I was just amazed to see it hadn't been updated since Jul 05! Working on updating my old setup (installed NNTP on that one back in '02) and would love to install this on my stage box but wasn't sure if it worked on 3.6.4 - thanks for the tip.Due to time constraints the original creator has had to give up support for it. It still works fine with the current versions of vBulletin but essentially it is now a user-to-user supported modification.

Lierduh has also given permission for people to create branch versions of the project so you may see some of them floating around as well.

[jarosciak]

...also given permission for people to create branch versions of the project so you may see some of them floating around as well.

Can you point me please to download or thread of one known to work with 3.6.4?
Thanks
Joe

[KW802]

Can you point me please to download or thread of one known to work with 3.6.4?
Thanks
JoeHave you tried the one in this thread? Seems to be working fine yet.

[Icehawk002]

Installed on 3.6.4 appears to work ok with the mailing lists although not tested the reply from forum feature as yet.

Two issues, when the showthread does not go to the specified thread but to no such thread, despite the link appearing to be a numbered showthread and correct.

Secondly sometimes the prefix is RE:[PREFIX] and is therefore excluded can you add some value or more than one prefix for a mail list.

Any help?

[Icehawk002]

I read some of the posts in this thread reference the RE: issue.

First I think I may have set the mailing list prefix wrong should this be a reference in the subject line.

emails are sent to the mailing list as either [group]topic or RE:[group]topic by setting the newsgroup prefix in admincp to [group] gateway.php ignores all RE:[group] messages.

hope thats clearer for understanding my problem.

[Icehawk002]

OK partially sorted problem now is that RE[group] starts a new thread from.

[tpearl5]

Did you try the killfile settings? (ACP => NNTP Gateway Options => NNTP Gateway Settings => Global Killfile) I have a few entries in my and it seems to be working as expected.


Ah sweet, I didn't even see that!

Second question: is anyone having issues with posts not being added to newsgroups threads if the user doesn't quote a reply? The instructions in Advanced_gateway_hacking_guide.txt are a little off. If you corrected this, how?

For example, here is a newsgroup thread looking through google groups: http://groups.google.com/group/alt.cellular.verizon/browse_thread/thread/5c33e3ac6addaf1c/e912874c7da47317?lnk=gst

Here it is on the forums: http://cellphoneforums.net/alt-cellular-verizon/t242610-vzw-data-plans.html

[tpearl5]

Oh, and where do you actually put the killfile? I don't have a form to put any values in.

[KW802]

Oh, and where do you actually put the killfile? I don't have a form to put any values in.
The name "killfile" is kind of a misnomer... any entries you put into that ACP option will actually have the affect of skipping the post, not actually moving it to a seperate forum or file.

[tpearl5]

The name "killfile" is kind of a misnomer... any entries you put into that ACP option will actually have the affect of skipping the post, not actually moving it to a seperate forum or file.

Right. I didn't realize there was a 'values' field. duhh

I just figured out how to get the quick reply and reply buttons to go away for newsgroup threads.

See attached Advanced gateway hacking guide for the steps I used. I have 3.5 but I would think it would work fine in 3.6 as well.

[Aylwin]

I've just noticed that forum subscription doesn't work (at least not for me). When new threads are created from the newsgroup/mailing list I don't receive any email notification.

Does anyone know how to code this? Or point me in the right direction?

[Rik Brown]

I'm looking for other sources to network this Gateway with beyond Usenet. Is anyone using this Gateway to network forums other than Usenet? It appears Microsoft pubic forums are a possibility. Is there anything else out there that people are using the Gateway with?

Thanks. -- Rik

[mpascal]

Just upgraded from 3.5.2 to 3.6.4

The instructions say:
2. How to run your AdminCP interface?

OPTION A (Edit a php file)

Edit ./admincp/index.php file
Find
construct_nav_option($vbphrase['vbulletin_options'], 'options.php?null=0', '<br />');

Below, add
construct_nav_option('NNTP Gateway Settings', 'nntp_settings.php', '<br />');
construct_nav_option('NNTP Gateway Newsgroups', 'nntp_groups.php', '<br />');

Above step adds two links to your Admin Control Panel for you to click.
================================================== =======================
That worked fine in 3.5.x but in 3.6 there is no such line.
I tried a couple of similar places but then the NNTP links would show up in every menu category.

Thanks
Marino

[tpearl5]

I found that if I put 'www' and 'http' in the killfile, there are a significant amount of posts that are not imported from usenet, sometimes causing threads to be not found by reference and split up. Has anyone else experienced this?

[dotcomguy]

Thank you for this hack. I currently have it setup and am importing posts from four newgroups. However, I am noticing a problem with bbcodes. The gateway script automatically adds color and adds [url] tags around URL's, however these bbcodes are not being parsed correctly when viewing the thread. The raw bbcode appears. I suppose I could rebuild the post cache to fix all those that I have already imported, but it would be nice if new posts added would have corresponding entries in the postparsed table.

Thanks

[Droll]

Installed this hack and finally got it running, I think I have applied all "updates" found in this thread, but I got a big problem.......

When downloading pictures from the news, some of them will be corrupted ( not all, pictures I have sent to the news group are fine :confused: , most of the other are distorted or just blank )

This one is downloaded with Thurderbird news client
http://tinyurl.com/376raf

This one is downloaded with this hack :confused:
http://tinyurl.com/3a2lma

Running vBulletin 3.6.5 on a ClarkConnect (http://www.clarkconnect.com/)( Linux Centios ) based server ( dual P3, 1GB ram )

Arne Kjetil

[panda1]

Will this NTTP Newsgroup hack work well with 3.6.5?

I coudn't locate any hack for 3.6.5!

Any body around have installed it successfully?

Thanks!

Panda1

[Rik Brown]

Will this NTTP Newsgroup hack work well with 3.6.5?

I only use the newsgroup option but that part works fine for me with vb 3.6.5. -- Rik

[panda1]

I only use the newsgroup option but that part works fine for me with vb 3.6.5. -- Rik


Rik,

Every think went fine with me but when trying to run gateway.php i received the following error:
------------------------------------------------
Database error in vBulletin 3.6.5:

Invalid SQL:
UPDATE nntp_settings
SET value = WHERE varname = 'last_postid';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 2
Error Number : 1064
Date : Wednesday, April 18th 2007 @ 04:59:06 PM
Script : http://forums.networkingland.com/admincp/cronadmin.php?do=runcron&cronid=19
Referrer : http://forums.networkingland.com/admincp/index.php
IP Address : 212.35.77.232
Username : admin
Classname : vB_Database

--------------------------------------------------

any help to fix this

Panda

[manuelsechi]

hi all
what's happen if i have to change NNTP server?
I done it and I was not able to receive any new news message...

thanx a lot for any help

[manuelsechi]

please help

I was using a NNTP server and everything went fine
I decided to change server, and now I'm not able to receive any message.
The new NNTP server works well, because i use it for another service
so I think now it doesn't work because of the last message id number

I can reset that number but I don't want duplicate messages from usenet in my board...

any idea?

thank u

[Rik Brown]

Rik,

Every think went fine with me but when trying to run gateway.php i received the following error:
------------------------------------------------
Database error in vBulletin 3.6.5:

Invalid SQL:
UPDATE nntp_settings
SET value = WHERE varname = 'last_postid';

MySQL Error : You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE varname = 'last_postid'' at line 2
Error Number : 1064
Date : Wednesday, April 18th 2007 @ 04:59:06 PM
Script : http://forums.networkingland.com/admincp/cronadmin.php?do=runcron&cronid=19
Referrer : http://forums.networkingland.com/admincp/index.php
IP Address : 212.35.77.232
Username : admin
Classname : vB_Database

--------------------------------------------------

any help to fix this

Panda

Panda:

Sorry, I'm not much of help with mySQL. But it looks to me that a variable is not set since the
line "SET value = (some value should be here) WHERE varname = 'last_postid';" is missing a value which should be in your nntp settings. See the "Last Max postid" variable in your NNTP Gateway Settings.

I'd make sure you have the defaults all set there and leave nothing blank.

-- Rik

[panda1]

Panda:

Sorry, I'm not much of help with mySQL. But it looks to me that a variable is not set since the
line "SET value = (some value should be here) WHERE varname = 'last_postid';" is missing a value which should be in your nntp settings. See the "Last Max postid" variable in your NNTP Gateway Settings.

I'd make sure you have the defaults all set there and leave nothing blank.

-- Rik

Thanks Rik!

You are right, this is a testing forum and I have no messages yet and the 'last_postid' should be some thing greater than zero.

I did created one message in my testing board and the problem was fixed, but I still have a problem connecting to google Usenet groups servers.... and I don't know if I am providing the right newsgroups sever URL, username and password!!!

I keep getting the following error message:
----------------------------------------------------
Gateway version 2.3.2 4 group(s) gatewayed.
Could not connect to NNTP-servegoggler. (A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
(10060))

Not connected
----------------------------------------------------

Any help will be appreciated


Panda1:cool:

[panda1]

Just upgraded from 3.5.2 to 3.6.4

The instructions say:

That worked fine in 3.5.x but in 3.6 there is no such line.
I tried a couple of similar places but then the NNTP links would show up in every menu category.

Thanks
Marino


Marino,

Go to next installation option
---------------------------------------------------------
OPTION B (No need to edit php file)

Just run your AdminCP interface from:

http://forums.yourdomain.com/admincp/nntp_settings.php
http://fourms.yourdomain.com/admincp/nntp_groups.php
---------------------------------------------------------------------------

It worked well with me, but don't forget to copy all hack files to your forum not only the ones in the readme file!!


good luck!

Panda1

[silurius]

I'm running vB3.6.4 and Mailman 2.1.9 and using this script to provide gateways between vB and email.

vB->email: Took a while to figure out all the Mailman idiosyncrasies but this part of the script is running perfectly. I'm certain my users will love it.

email->vB: Struggling with this. I can send an email to MailMan's pop account and it will a) get properly reflected to all users, and b) show up in the MailMan archive. Or I can "reply" to an email from the gateway and it will also a) get properly reflected to all users, and b) show up in the MailMan archive.

What isn't working is vB reading the POP3 account and posting the "email" or "reply" into the message board thread. I'm not certain what might be going on except that MailMan is immediately reading the POP3 account, then clearing it and vB has nothing to import? What else could be going on?

Any ideas out there?

(Noob here - How do I click INSTALL to indicate I'm running this script?)Did I miss any replies to this question? I'm also attempting to get Mailman->vB working.

[silurius]

Did I miss any replies to this question? I'm also attempting to get Mailman->vB working.Never mind. Just filling in a few more fields in the list management vB console and subscribing the appropriate address in Mailman seemed to take care of it.

[amaarvell]

Hi

when i do all the tests on the gateway they show as working and open, but when the cron does a scheduled start, I get n error message as below
.................................................. .................................................. .......
Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator, carl@whereinhull.com and inform them of the time the error occurred, and anything you might have done that may have caused the error.

More information about this error may be available in the server error log.


--------------------------------------------------------------------------------

Apache (Secured-VIPER) Server at www.vistareadygames.com Port 80

.................................................. .................................................. ........

I have checked the logs and its ahowing p as

[Tue Apr 24 23:27:33 2007] [error] [client 83.100.189.61] Premature end of script headers: php-script, referer: http://www.vistareadygames.com/admincp/index.php

Can you shed any light on this

Cheers

[bkaul]

<a href="https://vborg.vbsupport.ru/attachment.php?attachmentid=63669&d=1177602990" target="_blank" title="Name: 1.jpgViews: 34Size: 40.0 KB">1.jpg</a>

The link is to showthread.php?p=0#post0 rather than the actual post number, when the post is one that is generated by the gateway. Any ideas on what modifications would be necessary to fix this bug?