Hmmm.. this sounds like a tricky one :(

If you are getting the headers correctly, then we know it's talking to the INN server correctly and understands how to retrieve header information by message number.

The next thing the script does is to run those headers through the spam filter and if it decides it's an article we want, it requests the body from the news server using the message number and the 'body()' method from perl NNTP module. Assuming you have no replacements set, this must be the bit that is failing to retrieve the article from the server.

Why it's failing, I have no idea. Have you got another news server you can try. That way you can determine if it's an INN specific thing. You might want to make sure you have the latest NNTP module from CPAN aswell.

Yeah, I installed perl 5.6.1 and all the modules fresh, as a precaution. Unfortunately that machine can't get out to the net easily, so it doesn't really have access to another news server I can test against.

Thats not a major problem though. I was simply wondering if anyone had bumped into this one before.. I'm going to hack some debugging into your code and see what it *thinks* it's inserting into the DB =). A look at the database would probably be helpful, too.

BTW, have you thought about using a newsfeed instead of a poll method to populate the database? I wrote a perl script (still have it if you want it) that plugged into an INN server and pushed postings into a mysql database. It's spawned by INN when it starts, and INN pushes it an id of every message that INN gets sent (whether by an external feed, or a client) and then the script uses an INN supplied command to turn that id into a path to the physical location of the posting. The script reads it into an NNTP::Article structure and gets all the headers/body and dumps it into relevant fields in a MySQL database.

It's a bit broken atm, no checking for SQL statements in the body/headers etc :eek: but it works.

I was originally going to turn it into a vB clone, but then I found your hack. :D Much nicer to get vB to do all the hard work thats already been done, eh?

The reason I did it this way was that most people don't have either a full newsfeed or their own news server. Most of us are using a personal news account such as Supernews. There are issues of missing posts and problems with different character sets that could probably be handled more easily if the news server was dumping things raw into a database for us to process later so I would be very interested in your script if you'd like to share it :)

http://www.stupendous.net/news2mysql-0.1.tar.gz

I hope you find it useful.

I notice that you don't use News::Article for parsing of the articles? You'd probably find you would have better luck using that to handle all the character set issues and to avoid using home-grown regex's. I'm sure it uses regex's internally (not bothered to look) but why reinvent the wheel, eh?

How do Supernews feel about you polling their servers every 30 minutes? :)

Hrm...

The postings are definately in the database, and your newnews.pl is definately pulling out the body fine, its just that vB is not displaying it at all. Very strange :confused:

Feh.

Do you know if there is any debugging I can turn on in vB that will show extended information about the postings when viewing threads? its almost as if the posting isn't being associated with the thread properly.

In fact I described my problem incorrectly. The threads are being created but individual posts are not showing up in the threads - it's not the body of the message at all - Doh...

For some reason newnews.pl isn't assigning a threadid to any of the posts that it inserts into the DB. More investigation needed I think =)

Your DBI or DBD module is out of date. Read back a few pages and you will see other references to this. Older versions of these modules do handle the mysql_insertid() function correctly.

Here's is a news server that I found.

ccnews.thu.edu.tw

Joey

DBI-1.18
Msql-Mysql-modules-1.2216
perl 5.6.1

those are the latest afaics

What versions are you using to develop with?

I'm using those versions.

The problem you are seeing is due to something not handling the mysql_insertid() function.

Others had the same problem and solved it by upgrading their modules. Perhaps it's a MySQL issue. I'm not sure as I have never encountered the problem.