No.
You should not have this problem if no scripts have been modified.

I've read through this entire thread and surprised that no one else has seen this error.

This happens on the same message number repeadedly. I've removed the forum, created a new one, and initiated a download. It always stops at this message with this error:

Getting message number 567216: PHP has encountered a Stack overflow

When I adjust the message number to skip it via admin CP, it works fine and then again I get the message:

Getting message number 567237: PHP has encountered a Stack overflow

Any suggestions?

On a seperate note:

I set up the gateway on another board. Instead of the attachments getting decoded, I'm getting the raw data inserted into posts.

for example:

=ybegin line=128 size=111070 name=image-4.jpg
))=J*:tpsp*+++*r*r**)*m*-,,-,,----.--./2//../411026466545578<:78;855:@:;=}>???69AB@>B<>?>)*m+-.././3//3>757>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>)?*;2,?+?-+L*,;+-;+)?*F**,-++++*********-.,/0*1+2)?*E+*,-+++**********+,*-./01)*6-+*,:-:**+?5ո

The image import works fine on the other board, but for the life of me I can't figure out why this isn't working on board #2.

Ideas?

Is there any way to make this script use multiple connections, as it can be slow at importing large Newsgroups?

Yep, it would certainly speed up the import process on my end, too. But I'm not sure it's easily feasable :rolleyes:

How about the ability to mark the currently ported newsgroup as "locked". This way, one can run multiple instances of gateway.php, and each instance would import a different newsgroup (skipping the currently locked newsgroups).

I know, I know. It's not what you're asking, but it will help if you're importing several ng at once, no?

I would like this options also. Sometimes some of my newsgroups gets alot of post all of a suden and then i don't get download from all of the groups because it takes so long to download the new groups.

Is it normal for the time of the imported messages to be the time they are actually imported into the local forum rather than the time they are posted on USENET? I just installed the hack, and 3,008 messages were uploaded with the same time on each one.

Yes, this is normal since the times displayed are the times the post was posted to your forums. If you have 3800 messages, most likely that took a while so the older messages will be a couple minutes or more older than the newest messages.

There is an option to set post time to the time it was posted to usenet. Option number 11, I beleive, on usenet_settings.php.

anyone else have issues with pictures?

I pull down pictures from a drag racing usenet forum. They come across fine as attachments in the forums, but they pictures appear to have been decoded improperly. There are colorbars and distortion in the images. If I pull the pictures down with a newsreader, they look fine.

Ideas?

example:
http://www.domesticmuscle.com/dmforu...read.php?t=663

AWS, thanks for the fix with the post time.

Is there a way to completely disable the forum thread and user's profile from being attached to the end of a message? I just deleted these variables completely, but I haven't tested it yet.

Also, when someone creates a new thread on my board, why is it imported back to my board after it's exported out? For example, here is the original thread from my board:

http://www.hdtv-info.org/forum/showthread.php?t=483

And here is the very next thread being duplicated after the import:

http://www.hdtv-info.org/forum/showthread.php?t=484

There is also an option to not send the thread and profile link. Option 7 or 8 I think. I have that disabled and then define a default footer to attach which is the second to last option.
I've never had a problem with threads from the board being duplicated.
Do you have more than one feed going into one forum?

Thanks for your help!

I only have one HDTV newsgroup going into my forum, but I changed the "No Organization Check" variable, and it stopped the duplicate messages. Are there any negatives to changing this variable?

Okay last question. How do I add a carriage return inside my default footer? I tried \n but this didn't work. Right now when someone creates a post, it puts the default footer on the same line as the username. For example, my outgoing posts look like this:

--
hdtvnutThis post originated from http://www.hdtv-info.org/forum/

Please, go through settings at least once.

It is not that the connection takes so much time, it is the time it needs to insert the imported posts into the forum system. It needs to build index, find the similar threads... Normal user posts message one by one, we are 'posting' perhaps hundreds of messages in one go. If your server is slow, then it will take a long time.

In
includes/functions_nntp.php
find
$signature .= $nntp_settings['default_footer'] . "\n";
change it to
$signature .= "\n" . $nntp_settings['default_footer'] . "\n";

should do the trick.

Ok, cheers :)

I do have a wierd problem with the script at the moment, it just stops importing certain messages and it exits the while script. It does this every time it runs and I have to tell it to skip manually. Do you know why it might be doing this, or how the script can just skip it if it can't import it?

Hi

I´m trying to remove the "reply buttons as told in the "hacking_guide" but can't find the showthread template....anyone who can tell me where to find it?

im running 3.0.7

/peter

What php version are you running ? Take a look at https://vborg.vbsupport.ru/showpost....&postcount=801
.
.

So If I understood you correctly, if I comment this line in functions_nntp.php

and then use these functions (in Admin CP -> Import & Maintenance):

-> Rebuild Forum Information
-> Rebuild Similar Threads

(or perhaps Rebuild Search Index)

that will speed up the import process since the rebuild function will be called just once (at the end), is that correct?

(imported newsgroups are shown to visitors only after the import process is finished)

Well, I don't think I need this anymore...

For those wishing to speed up the import process, you can setup a private test forum, setup the NNTP hack, import newsgroups and then use impex to copy the imported data from the private forum to the real one.

Not sure if this is a good idea, I'm going to try it right away...

hmm .. not sure if this has been mentioned before ..

Hacks been working brilliantly and there's only one thing I've noticed which bugs me ..

When looking at the list of threads in a forum that is of USENET posts and you hover your mouse over a thread title instead of getting the first post wording pop up you seem to get the last post in the thread.

This is not the normal way the forums work. I've looked in the settings and I can't see anything that would effect this. Is there anyway I can get the first post to show instead of the last post?

As a side note, I did have it set to show time a post was imported into the forums .. I've now changed that to show post made on USENET .. not sure if that will make a difference ..

Any ideas?

Just spotted this thread ..

https://vborg.vbsupport.ru/showpost....&postcount=220

Will this solve my problem? .. It's from last year so I would of thought it would of been incorporated into the hack .. has it?

I don't want to add it to my gateway.php if it's already been done somewhere else in the code ..

I'm using 4.3.x so it should work :(

does this idea works for you ?
please post your comments if you are able to speed up this SLOW process anyhow.

thanks.

Can't confirm, working fine with 5.0.4 :)

Is this hack being supported anymore? Haven't seen much input from the programmers lately...

he just posted on may 13

Ahh, perhaps my questions were just overlooked :)

Hi,

Great MOD you have, but i have some problems;

why does a titel like change to You can see it here http://www.dacor.dk/forums/forumdisplay.php?f=9

some have told me it can be a MIME problem, but is there a solution for this?

/Peter

Because your PHP is compiled without imap which is needed for imap_mime_header_decode to work in gateway.php. Furthermore this hack doesn't handle utf-7/8, for that to work gatway.php needs to be modified further (for instance with one of the comments mentioned under imap_mime_header_decode in the PHP manual).

David_R,

(Sorry for the late reply)

I didn't try it (not yet :)). I'm not sure, I'm afraid it will crash my server.

I installed a private test forum and setup this hack on it (total of 3 different forums with their respective gateway.php instance running every hour), this way I could simulate multi-threading & hopefully speed up the import process. The msg imported into the test forum would be ported to the live one using impex (very fast, since both forums are located within the same machine)

I don't have enough data to make any conclusion, but as far as I can see, the speed is almost the same.

Perhaps the script is already running at maximum speed, or perhaps as the author suggested, the import is slow because of the similair threads thing, I don't know (I have a dedicated box with 100Mbps, my nntp provider doesn't limit download speed).

In my case, getting a dedicated box and switching to a better nntp provider helped me speed up the process.

HTH,
-k

I have tried with 3 nntp providers on a Dual Xeon with 8 GB RAM dedicated box.
But no improvements. It takes more then 12 hours to import about 2,00,000 posts :(

Did you mean 2,000 or 2,000,000 (millions)?

2 Millions in 12h is great IMO. I have a dedicated box (3Ghz, 1GB RAM) and I'm averaging 200,000 *REAL* msg/day (I'm using GigaNews).

But, if you meant 2,000 (or 20,000)...then yes, it *is* slow.

Can you confirm?

Perhaps you should check with your nntp provider if they're limiting the speed at wich you can download msg.

-k

Hello

I'm using this hack with some high-volume groups. Most of the time it works ok, however there are some problems.

1) not all newsgroup posts are downloaded to forums.
i debugged this thing and i know why but don't know how to fix it

the post numbers on nntp server are not in correct order, replies have earlier numbers than the original post which starts a thread. This causes the replies to be inserted first as new thread and then the thread starting post to be inserted as another new thread. I can't think of anything else then running a script from time to time which would solve such things using msgids and references. Any other idea?

for example one of the threads have the following server post numbers:

and the result is two threads,
one
and second
2) some of replies (on forums) to downloaded posts are not posted to newsgroup. Even though they have "isusenetpost=1" they do not appear on newsgroup. When i make "isusenetpost=0" in DB and change $nntp_settings['last_postid'] to include this post and run gateway.php the post is uploaded correctly. The problem is only for some of the posts not all and i did not find out yet anything that would give me a clue about this problem. Msgids and references are ok

Thanks

How often does your script run? If a reply arrives at your NNTP server before the original message, then threading could be wrong. The chance for this to happen will be reduced greatly if the time between the running of the script is increased. Also you can try the option to thread your messages by subject instead of Msgids. There is an option in the settings. I can't remember how I implemented this, but it is worth a try.

I can't explain the second problem. You can turn on the logging by editing the setting at the beginning of gateway.php. It will email you the debug message each time the script is run. If for any reason the server rejects the message, the debug message should show the error message.

Thank you for your reply. Ny script runs once every hour. I don't think that changing to thread by subjects would help. Also searching the DB for subjects is too slow when there are nearly 400 000 posts. The problem is when you get messages from the server from $lastmsg to $max, replies get downloaded first and inserted as new thread, then the thread starting post is downloaded and inserted as another new thread. I don't think that threading by messages would delete the first thread and link it to the second thread.

Thanks

What exactly are the isusenetpost and msgid fields needed for? They are indexes in the post thread, but I think it might be slowing down inserting new posts for the rest of the forum?

Has anyone figured out or modded the hack to exclude certain forums from sending posts to the Usenet? I have some forums I'd like the users to be able to post to the Usenet on, but other's I wouldn't.

This is an awesome script, BTW. My issue is that I have one forum with around 200,000 posts and it is SOOO SLOW for it to sync up that database. It even stops the entire forum from responding while it's doing it's thing. Once it gets to importanting, it is fine, but the initial scan or whatever it's doing, is dog slow.

Any ideas or solutions?

Also, when it comes to filtering, it doesn't seem to filter SUBJECTS. I'd really like for it to do that, doing a H or B for the filter doesn't seem to work for subjects.