Because your PHP is compiled without imap which is needed for imap_mime_header_decode to work in gateway.php. Furthermore this hack doesn't handle utf-7/8, for that to work gatway.php needs to be modified further (for instance with one of the comments mentioned under imap_mime_header_decode in the PHP manual).

David_R,

(Sorry for the late reply)

I didn't try it (not yet :)). I'm not sure, I'm afraid it will crash my server.

I installed a private test forum and setup this hack on it (total of 3 different forums with their respective gateway.php instance running every hour), this way I could simulate multi-threading & hopefully speed up the import process. The msg imported into the test forum would be ported to the live one using impex (very fast, since both forums are located within the same machine)

I don't have enough data to make any conclusion, but as far as I can see, the speed is almost the same.

Perhaps the script is already running at maximum speed, or perhaps as the author suggested, the import is slow because of the similair threads thing, I don't know (I have a dedicated box with 100Mbps, my nntp provider doesn't limit download speed).

In my case, getting a dedicated box and switching to a better nntp provider helped me speed up the process.

HTH,
-k

I have tried with 3 nntp providers on a Dual Xeon with 8 GB RAM dedicated box.
But no improvements. It takes more then 12 hours to import about 2,00,000 posts :(

Did you mean 2,000 or 2,000,000 (millions)?

2 Millions in 12h is great IMO. I have a dedicated box (3Ghz, 1GB RAM) and I'm averaging 200,000 *REAL* msg/day (I'm using GigaNews).

But, if you meant 2,000 (or 20,000)...then yes, it *is* slow.

Can you confirm?

Perhaps you should check with your nntp provider if they're limiting the speed at wich you can download msg.

-k

Hello

I'm using this hack with some high-volume groups. Most of the time it works ok, however there are some problems.

1) not all newsgroup posts are downloaded to forums.
i debugged this thing and i know why but don't know how to fix it

the post numbers on nntp server are not in correct order, replies have earlier numbers than the original post which starts a thread. This causes the replies to be inserted first as new thread and then the thread starting post to be inserted as another new thread. I can't think of anything else then running a script from time to time which would solve such things using msgids and references. Any other idea?

for example one of the threads have the following server post numbers:

and the result is two threads,
one
and second
2) some of replies (on forums) to downloaded posts are not posted to newsgroup. Even though they have "isusenetpost=1" they do not appear on newsgroup. When i make "isusenetpost=0" in DB and change $nntp_settings['last_postid'] to include this post and run gateway.php the post is uploaded correctly. The problem is only for some of the posts not all and i did not find out yet anything that would give me a clue about this problem. Msgids and references are ok

Thanks

How often does your script run? If a reply arrives at your NNTP server before the original message, then threading could be wrong. The chance for this to happen will be reduced greatly if the time between the running of the script is increased. Also you can try the option to thread your messages by subject instead of Msgids. There is an option in the settings. I can't remember how I implemented this, but it is worth a try.

I can't explain the second problem. You can turn on the logging by editing the setting at the beginning of gateway.php. It will email you the debug message each time the script is run. If for any reason the server rejects the message, the debug message should show the error message.

Thank you for your reply. Ny script runs once every hour. I don't think that changing to thread by subjects would help. Also searching the DB for subjects is too slow when there are nearly 400 000 posts. The problem is when you get messages from the server from $lastmsg to $max, replies get downloaded first and inserted as new thread, then the thread starting post is downloaded and inserted as another new thread. I don't think that threading by messages would delete the first thread and link it to the second thread.

Thanks

What exactly are the isusenetpost and msgid fields needed for? They are indexes in the post thread, but I think it might be slowing down inserting new posts for the rest of the forum?

Has anyone figured out or modded the hack to exclude certain forums from sending posts to the Usenet? I have some forums I'd like the users to be able to post to the Usenet on, but other's I wouldn't.

This is an awesome script, BTW. My issue is that I have one forum with around 200,000 posts and it is SOOO SLOW for it to sync up that database. It even stops the entire forum from responding while it's doing it's thing. Once it gets to importanting, it is fine, but the initial scan or whatever it's doing, is dog slow.

Any ideas or solutions?

Also, when it comes to filtering, it doesn't seem to filter SUBJECTS. I'd really like for it to do that, doing a H or B for the filter doesn't seem to work for subjects.