[RELEASE] Usenet Gateway for vB 2.01

Description:

This hack provides a gateway to selected usenet groups. The entire usenet group is mirrored locally. Posts submitted locally to mirrored newsgroups will also be sent out to usenet. A single perl script is executed via cron that collects news and posts any outgoing articles. Incoming messages are parsed against customizable spam filters and threaded correctly before being placed in your forums. By default, no personal information is sent to usenet other than the posters username and whatever you have globally configured as a footer and organization. The option does exist for a user to include a custom email to use for usenet posts. After initial installation (which involves creating some new tables, adding a few columns to existing vB tables, and entering the database connection information to the news script) configuration is completed via the vB control panel.

• See it in action at dBforums.com (usenet forums are at the bottom)
• Download latest version for vB 2.2.4

Current version is 20020323 for vB2.xx

Current feature list:

• correct threading of outgoing posts on usenet
• full control panel integration
• email notification to usenet replies
• emoticon translation into vb icons
• vb code removal or conversion prior to posting to usenet
• hyperlinked urls in messages
• color coded, italicized & indented quotes
• vB style quote to usenet style quote conversions for outgoing posts
• logging of outgoing posts
• support for multiple news servers
• support for seperate footers per forum in outgoing posts
• multi-language (selectable) handling of quoted MIME printable headers (for all those funny foreign characters)
• handling of mailing archive groups
• flexible spam control and replacement variable options for incoming and outgoing messages
• configurable auto-expire option
• option for users to show email address in their outgoing usenet posts (may be different than normal one in profile
• option to enable/disable user signatures on outgoing posts.
• fully compatible with vB moderation functions

Still to be done:

• canceling of messages after they have been sent to usenet via control message in accordance with RFC1036.
• binary attachment support
• email to PM gateway

Screeshots:
Main Options
More Options
Configuration of Newsgroups
Add New Newsgroups
Spam Control
What an imported usenet article looks like in vB

The latest version of this hack will always be in this first post of the thread.

Before asking questions in this thread, please read all the posts in this thread. If your question goes unanswered, it will probably be because the question has already been answered countless times in this thread.

This hack was created for use on my forums and will only be supported as time permits.


Links to required Perl Modules

• News::NNTPClient
• Net::SMTP
• MIME::WordDecoder
• Text::AutoFormat
• Mail::Address
• URI::Find
• DBI
• DBD::MySQL
• Date::Parse
• HTML::Entities

[george_proost]

The new code is performing wonderfully. Thankyou very much.

However, a small parsing/dellimeting error :

Query failed:
INSERT IGNORE INTO usenet_outgoing(poster,email,signature,newsgroup,subject,body,threadid,postid)
VALUES ('Mic Murph','xxx@xxx.com',,'ibmpub.java.os390','Access
to Environment Variables from JNI native code','A while back, I developed a Java & C/C++ native wrapper for our company\'s
data management product (tableBASE). We\'re running on
OS/390 MVS V2R8 using JDK 1.1.8 under USS. \r\n\r\nThe Java 
wrapper statically loads the C/C++ class when it is instantiated. 
\r\n\r\nIn the C/C++ native routine, I used a "fetch()" library call 
to locate the function pointer to a HLASM routine, which is stored 
in a pre-existing MVS namespace in a PDS(not PDSE) link-editted load library. \r\n\r\nI used the "STEPLIB" environment variable in my BASH script to refer to the two load libraries that had the 
HLASM stub and the additional overlays that it calls. \r\n\r\nNote that this results in a HLASM call wrapped in a C/C++ JNI native
 method wrapped in a Java class. We used this approach since 
the HLASM interface is well known by our customer base. \r\n\r\nI set up an RMI-distribution for the Java wrapper using a BASH 
script to set/export various environment variables. We\'ve been 
experimenting with different client architectures, such as 
servlet/JSP, standalone application over RMI etc. \r\n\r\nThe 
setup works and is pretty speedy, considering we\'re still using 
the May 2000 edition of the 1.1.8 JDK. \r\n\r\nNow, we\'re 
starting to scale up for productionalize by running the JNI 
wrapper from within a servlet in WebSphere 3.02. And when we 
run the servlet, the Java wrapper class is able to load the C/C++ 
native wrapper OK. \r\n\r\nThe problem is that the C 
program\'s "fetch" of our HLASM load library fails, as if it\'s not finding our STEPLIB environment variable. \r\n\r\nMy understanding is that WAS inherits any environment variable 
settings from the HTTP 5.2 server config (in /etc/httpd.envvars). \r\nThis is where we\'ve put both our LIBPATH entry (for our 
native .SO DLL which is being found by the loader) and our 
STEPLIB variable (for our HLASM load library which apparently 
ISN\'T being found). \r\n\r\nI\'d prefer not to have to use 
the "setenv()" library calls to set/get the environment variables 
from my C/C++ native routine.\r\n\r\nAny suggestions? \r\n\r\nThanks, Michael Murphy \r\n\r\nP.S. This is actually a cross-
post from the new forum/portal "www.mainframeforum.com".  
Check it out!',30239,83853)


DBD::mysql::db do failed: You have an error in your SQL syntax 
near ''ibmpub.java.os390','Access to Environment Variables from 
JNI native code','A wh' at line 1 at www/admin/newnews.pl line 
596, <SOCK3> line 2.

welcome back

[webhost]

I really need some help here please

Need some help everything was working great than all of sudden my usenet posts where being added to my board post's . I had but the option in to avoid this but for some reason it quit working, when I purge usenet groups the board and the thread count along with usenet counts decrease. This is how I have it in forums/index.php file, also why would purging timeout before finishing? here;s what I have Found it had to remove lines in bold below.

// get total posts
$countposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post');
$totalposts=$countposts['posts'];
if ($totalposts=='') {
$totalposts=0;
}

// START USENET HACK //
$countthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread WHERE isusenetpost=0');
// END USENET HACK //

$countthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread');
$totalthreads=$countthreads['threads'];
if ($totalthreads=='') {
$totalthreads=0;
}

// START USENET HACK //
$countusenetposts=$DB_site->query_first('SELECT COUNT(*) AS posts FROM post WHERE isusenetpost=1');
$totalusenetposts=$countusenetposts['posts'];
if ($totalusenetposts=='') {
$totalusenetposts=0;}
$countusenetthreads=$DB_site->query_first('SELECT COUNT(*) AS threads FROM thread WHERE isusenetpost=1');
$totalusenetthreads=$countusenetthreads['threads'];
if ($totalusenetthreads=='') {
$totalusenetthreads=0;}
// END USENET HACK
// Total Thread Views

// get newest member

[webhost]

I will clean this up and show what I have done and where I am at.

First I still have first hack installed, Haven't upgraded yet.

Hack had been working fine until today.

I had earlier set up the option where the usenet posts would not count against the board total all had been going well for the last couple of months. Today I look and the board post count had the usenet count added to it also the thread count. I looked over script placement and it was correct according to the instructions. I went to my database and sure enough all board post and usenet posts showed up in the post table all 23,000 of them. So I went to the control panel and first I purged all of the newsgroups; post's came down when I did this both for board post and also for usenet post's. After this was complete I updated all counters. I still had some newsgroups showing contents even though I had just purged all of them. So I went to that forum and pruned them and them updated counters. I went back to the usenet section of the control panel and all newsgroups showed zero. I went back to board and even though I had just zeroed out everything in the usenet control panel; On the board it was still showing 181 posts and the board total was high by that number 181. I went back to the database post table and sure enough there were 181 usergroup post's still there. I then went and ran a test on 3 newsgroups and reactivated them, Information came in and same thing they are adding to my main board total.
First any ideas what is doing this?
Second what is the easiest way to remove those 181 post's in the post table?

Any help would be Welcomed. Have working again just need to know why the posts would not delete when the rest did and how to accomplish removing them?

Thanks Joey

[James Cridland]

This hack looks just incredible, and I'm excited about getting it to work.

I'm having problems installing the perl modules, reporting an error...

Checking if your kit is complete...
Looks good
Writing Makefile for Text::Autoformat
-- OK
Running make test
'test' is not recognized as an internal or external command,
operable program or batch file.
test -- NOT OK
Running make install
make test had returned bad status, won't install without force

I'm using Windows 2000 running ActivePerl - should I stop now, or should I persevere?

[george_proost]

I run on linux ... the hack is really great.

Keep on going at it till you get it right.
USENET-Gateway : is the most important hack for me.

Second hack is the overgrow archive hack... although I have my reservations about the effectiveness. I have made some changes which I'll release when I'm done, it'll be called <overgrows positively enhanced referral attractor> still needs a bit of final display work.

archive.mainframeforum.com

Some enhancements:
-Displays only levels with content allowing spiders to navigate faster, cleaner and always with something to digest.
-Employs text sizing, highlighting etc.. to attract spiders. They see the important stuff right away.

[fastforward]

Originally posted by george_proost
The new code is performing wonderfully. Thankyou very much.

However, a small parsing/dellimeting error :

Query failed:
INSERT IGNORE INTO usenet_outgoing(poster,email,signature,newsgroup,subject,body,threadid,postid)
VALUES ('Mic Murph','xxx@xxx.com',,'ibmpub.java.os390','Access
to Environment Variables from JNI native code','A while back, I developed a Java & C/C++ native wrapper for our company\'s
data management product (tableBASE). We\'re running on
OS/390 MVS V2R8 using JDK 1.1.8 under USS. \r\n\r\nThe Java 
wrapper statically loads the C/C++ class when it is instantiated. 
\r\n\r\nIn the C/C++ native routine, I used a "fetch()" library call 
to locate the function pointer to a HLASM routine, which is stored 
in a pre-existing MVS namespace in a PDS(not PDSE) link-editted load library. \r\n\r\nI used the "STEPLIB" environment variable in my BASH script to refer to the two load libraries that had the 
HLASM stub and the additional overlays that it calls. \r\n\r\nNote that this results in a HLASM call wrapped in a C/C++ JNI native
 method wrapped in a Java class. We used this approach since 
the HLASM interface is well known by our customer base. \r\n\r\nI set up an RMI-distribution for the Java wrapper using a BASH 
script to set/export various environment variables. We\'ve been 
experimenting with different client architectures, such as 
servlet/JSP, standalone application over RMI etc. \r\n\r\nThe 
setup works and is pretty speedy, considering we\'re still using 
the May 2000 edition of the 1.1.8 JDK. \r\n\r\nNow, we\'re 
starting to scale up for productionalize by running the JNI 
wrapper from within a servlet in WebSphere 3.02. And when we 
run the servlet, the Java wrapper class is able to load the C/C++ 
native wrapper OK. \r\n\r\nThe problem is that the C 
program\'s "fetch" of our HLASM load library fails, as if it\'s not finding our STEPLIB environment variable. \r\n\r\nMy understanding is that WAS inherits any environment variable 
settings from the HTTP 5.2 server config (in /etc/httpd.envvars). \r\nThis is where we\'ve put both our LIBPATH entry (for our 
native .SO DLL which is being found by the loader) and our 
STEPLIB variable (for our HLASM load library which apparently 
ISN\'T being found). \r\n\r\nI\'d prefer not to have to use 
the "setenv()" library calls to set/get the environment variables 
from my C/C++ native routine.\r\n\r\nAny suggestions? \r\n\r\nThanks, Michael Murphy \r\n\r\nP.S. This is actually a cross-
post from the new forum/portal "www.mainframeforum.com".  
Check it out!',30239,83853)


DBD::mysql::db do failed: You have an error in your SQL syntax 
near ''ibmpub.java.os390','Access to Environment Variables from 
JNI native code','A wh' at line 1 at www/admin/newnews.pl line 
596, <SOCK3> line 2.

welcome back

I've just encountered this error myself. It happens when a post is made that has the 'show signature' option unchecked. I'll post a fix later tonight.

to say thank you so very much, fastforward. this hack is extremely valuable, and I am thrilled to have it working on my board.

i look forward to your future contributions.

best wishes,
J

[dwh]

Originally posted by JenniferS
to say thank you so very much, fastforward. this hack is extremely valuable, and I am thrilled to have it working on my board.

i look forward to your future contributions.

best wishes,
J

Jennifer, I checked out your site, looks interesting tho' I'm still not clear on the main topic or purpose of the site...but I got I guess it was probably javascript error on the first thing I clicked.

You also may like to know that putting the cursor over your ip icon displays <!--Template IP code etc.. --> probably bad alt tag or else a missing quote somewhere..

i still have the system spitting out template identification comments which may or may not have been responsible for your javascript dead end.

which is to say, sorry you couldn't get in, and, my purpose is: knowledge accumulation.

[dwh]

I don't think it was javascript as I looked back. I think you have a quotes problem. Either you didn't close a quote, or maybe you have some code in phpinclude and mistakenly had something like

$variable="<font "Times New Roman">Words</font>"

and that messed up the code....if that's the case, to fix it try

$variable="<font 'Times New Roman'>Words</font>"

instead.