ok, the good thing about the spam is they can't just decide to charge us, one of the things i like about it.

i may work on a template set then,

cheers for the hack have been looking forward so long to get it installed! my mods are sick of me talking about it :)

possible bug

i started a new thread in a usenet forum, while newnews.pl was running (while it was collecting threads). Anyway, the thread now sites in usenet_outgoing and won't get sent even when i run newnews.pl again.

I'm sorry for being such a pain but even after fiddling around with replacement variables i can't get rid of the
<font size="1"> <i><font color="blue<br>">>
bit.
For the time being I had to do upside down gimnastics and create two replacements:
{usenetquotecolor} is replaced by #031AAB">
and
}">> is replaced by >
just to make it into valid html (but with <br> is still at the end of every single line, which makes all quotes twice as long). I simply can't find where is that <br> coming from. Any ideas?

I looked and couldn't find anything wrong, and I the same as fastforward don't seem to have the problem.

This is a long shot... check your vB replacement set
and make certain you don't have a carriage return after blue.

It will not show on the CP listing
{usenetquotecolor} is replaced by blue [edit] [remove]
you need to go into edit and press delete several times at the end of the word blue.

I hope this makes sense?

Cheers!

That was one of the first things i've checked. All the articles look ok in database:
so I guess whatever adds the <br> tags is between db and showthread.php.
I also have no custom replacement variables except for the usenetquotecolor ones....
:(

fast forward,
I have spent some 4 hrs reading with interest all the posts today in this thread with amazement.

You realy unleashed a can of worms with this hack didnt you :)

I must praise you and the other members who have been with you on this from the start for your response times and dedication to resolving the problems.

Well done.

After reading all this i have a few questions i would like to aske before i install it on my m8s board to "Try".

1) Exactly which perl modules need to be installed for this script to run?
2) How can i find out which modules are already installed?
3) How do i find out what CRON is and if that is installed or usable or whatever it needs to be?

Sorry for the dumb Q's but im prety sure they havnt been asked/answered before and i know absolutly nothing about perl.

Thanks for the hard work and effort once again.

Bundle::DBI - DBI-1.16.tar.gz (should already be installed)
Bundle::DBD::mysql - part of the Msql-Mysql-modules package (should already be installed)
HTML::Entities - part of the HTML-Parser package (should already be installed)
MailTools - MailTools-1.15.tar.gz (should already be installed)
Net::NNTP & Net::SMTP - both part of the libnet package (probably already installed but older versions will not have the NNTP module)
MIME::WordDecoder - part of the MIME-tools package (MIME-tools is probably already installed but older versions do not have the WordDecoder module)
Date::Parse - part of the TimeDate package (maybe installed)
Text-Autoformat - Text-Autoformat-1.04.tar.gz (probably not installed)
Email-Find - Email-Find-0.03.tar.gz (probably not installed)
Email::Valid - Email-Valid-0.13.tar.gz (probably not installed)
URI::Find - URI-Find-0.04.tar.gz (probably not installed)

The easiest way is to run the script. It will tell you if it can't find a module.

Cron is installed along with operating system in every unix and linux flavour I have come across. Type: 'crontab -l' at the command prompt. As long as it doesn't give you a permission denied error, you are ok.

I find the following script handy for the listing installed perl modules. I posted this a while back but for some reason the page that it is on is generating a vB parse error. So here it is again.

Download:
http://www.makeitsimple.com/download...=geninfo&did=6

Working example:
http://www.makeitsimple.com/cgi-bin/modules.cgi

I just do a find(Ctrl+F) on the page for whatever module is in question.

Cheers!

wow thanks for the quick responses guys much appreciated

OK... I'm moving along with the usenet gateway and getting closer. (For those of you who are new to perl and hacking, this is not a hack for the light of heart!) The problem I had installing the perl modules is that I am hosted by a virtual host who does not technically support perl modules, some of which require superuser access. However, I was able to get around it a bit by installing the modules and perl in a directory off of my root directory. Typing in usr/bin/perl or whatever would not work because I don't have control to the absolute root of the server, only mine. Hope this helps and I'm back at it...

Ditto!

Be afraid, be very afraid! :)

No, seriously, you need your own test environment, with PHP and Perl to try things out first, and have backups.

This in no way reflects in Fastforwards skills, in my book this IS the best and most elegant vb hack I ever installed. The reason you should be careful is that this hack requires significant changes in you database since it essentially adds a functionality VB never was designed to do. And it does it very well. It doesn't add a little thing here and there like most hacks, it adds a LOT.

So, if you know your stuff, go ahead. If you're not sure, ask first.

I couldnt agree more which is why i asked ;)

But by the souds of it id be best advised not to install and test on a working board with NO perl knowledge/ experience then ill leave alone and hope that maybe something similar will be written in PHP ( if its possible, but i thought that php could do everything that perl can do and more)

thanks for the forewarnings and all the advice though people it IS really apreciated

I'd be careful even if it was written in PHP! :)

You've got that the wrong way round. Having said that, this particular app could be written in PHP but you wouldn't have all the modules to help you out. Plus the fact that this is executed in the background using cron. Not many hosts have the PHP executable installed that allows you to run php scripts standalone. In fact many people don't even know that you can run php scripts as standalone programs.

Time to get serious about a usenet news feed. Up until now I've been all smiles, getting the gateway setup has been a blast.

Now however, reality hits and hits hard. I've had my head so buried in getting the gateway installed that I hadn't given much thought to tuning it on for mainstream use. Until now I've had all my usenet forums locked to read only, then I started thinking about the spam issue. So I stared reading TOS agreements at a bunch of the usual less than $25/mo services. Oh-oh time... I know I read a few posts here of people wondering about the same issue and at the time I just blew it off thinking ah heck, I'll worry about that when I get there.

Well I'm there/here and I'm worried. Not only is it against the TOS of just about any service to pull/suck news... most of them specifically state they are for individual use only. Not to mention the up to $200/hr spam cleanup charges. Several renegade BBS users could bring a world of hurt on the web master. Especially when we would be breaking the TOS even before getting caught for spam problems. Not a good picture.

How about a few of us group together and pre-pay for one *real* feed that we could all use? This would be cheaper than doing it alone.

Here's a few snips of what I found so far...

Usenet NewsServer Club
Account Usage - UNSC accounts are single-user accounts. This means that only one login from your account is allowed at a time, and the owner of the account (or their immediate family) is the only person allowed to use that account.
http://www.alibis.com/

NewsGroups.com
I will not sell, redistribute, or copy in any way the contents of this site. I will not allow anyone to use my password or post SPAM of any type. If I do post, I do so at my own risk.
http://www.newsgroups.com/

1usenet.com
INTENDED USE - Accounts are intended for personal use only, commercial use is prohibited without prior written authorization

We have ZERO tolerance for Spammers or Flooders. Posting is a privilege, not a right, if you post SPAM and or flood any group, your account or posting privileges may be revoked WITHOUT PRIOR WARNING, and you may be billed (at the discretion of the System Administrator) for the cleanup, at $200.00 (US) per hour, plus any Collections and or Attorney's fee's incurred in collection of this fee. 30 day auto billing for 1 year
http://www.1usenet.com/

Usenet-X.com = popup crap - Don't even bother
SPAM AND FLOODING - We have ZERO tolerance for Spammers or Flooders. Posting is a privilege, not a right, if you post SPAM and or flood any group, your account or posting privileges may be revoked WITHOUT PRIOR WARNING, and you may be billed (at the discretion of the System Administrator) for the cleanup, at $200.00 (US) per hour, plus any Collections and or Attorney's fee's incurred in collection of this fee.
Posting Footer - usenet-x.com reservers the right to attach an advertisement to the bottom any posts you make through our service. If you object to this you have the option of posting through spamkiller.usenet-x.com, which does not attach the footer.
INTENDED USE - Accounts are intended for personal use only, commercial use is prohibited without prior written authorization from usenet-x.com.

Here's a couple possible candidates.

Ball and Ball, L.P
$59.95/month Pull Newsfeed - includes accepting a return feed of your local posts. You are expected to enforce a strong anti-spam policy. (limited to 250 Megabytes of downloads per day)

$99.95/month Pull Newsfeed - includes accepting a return feed of your local posts. You are expected to enforce a strong anti-spam policy. (limited to 500 Megabytes of downloads per day)

$149.95 /month Pull Newsfeed - includes accepting a return feed of your local posts. You are expected to enforce a strong anti-spam policy. (limited to 1000 Megabytes of downloads per day)

Usenet Acceptable use Policy: We have a strong anti-spam policy. If you spam Usenet through our servers, we will terminate your account and charge you for any costs we incur in dealing with complaints and cleaning up your mess. Posting of illegal content is NOT allowed on our servers and is grounds for account termination without a refund. Groups carried are subject to change without notice.

For our Usenet service, we use the Dnews server software. Our service is aimed at the user who wishes to participate in Usenet discussion groups. If it's porn, warez, or mp3 files you're looking for, we're not the choice for you. If it's a fast server with decent retention and good spam filtering, we're the choice for you. We try to make our service family friendly.
http://www.bnb-lp.com/Usenet.html

Eskimo.com Usenet Services
For Organizations - We provide NNTP read/post access to other sites, ISPs, businesses, BBSs, and other organizations, for $600/year, or $180/quarter, or $70/month per domain. If you're looking for a way to provide Usenet News services to your customers without the expense of maintaining your own news server and you are looking for a solution that doesn't impose a per subscriber or per megabyte charge consider using our news service as an option.


NOT a candidate just more of what we don't want!

NewsGuy.com
Newsguy are intended for the use of its subscribers and is not intended for re-distribution, or as an unauthorized news feed to another service. Downloading article quantities indicative of pulling a newsfeed from either service is unacceptable. Subscribers connecting to the service with news retrieval software which is designed to copy the contents of large quantities of newsgroups, or sustain multiple news connections, will have their service terminated. *snipped*

The cost for a commercial newsfeed is $400 per month. The Net.* hierarchy of newsgroups requires extra setup at the customer's site to insure compliance with their "sound site" policy. Recommended minimum equipment to fully utilize our newsfeeds:
A Pentium server, 400 Mhz or faster, 500 Mbs of RAM, 64 Gbytes of disk storage, INN news server software. A bused pair of T1 with combined 3 mbps available bandwidth.
Newsguy Newsfeeds cannot be propagated outbound to servers or resold. Only newsfeeds back to Newsguy's servers are allowed. As Usenet news is always evolving, Newsguy cannot guarantee the availability of any group(s), and we strongly urge you to review our service policy prior to signing up for our newsfeed.

What I've done is installed a news server on my machine that pulls from all the other servers I'm interested in and then I just connect to my local server to get the news. You will never be able trap 100% of all spam, but with the hack spam filters and the DNews filters you should be quite safe.

As for the 'no pulling' clause. You will be pulling an insignificant amount of news in comparison to anyone who happens to download a few mp3s or a couple of binary pics each day. You won't even register on their meters.

The commercial clause is debatable. Unless you are charging subscriptions for access to your usenet forums you could argue your case.

But the bottom line is; the amount of news you are actually downloading really is tiny. As long as you stay away from the binary groups I wouldn't worry too much.

NgtCrwlr, how many newsgroups do you plan on pulling? If it's just a few, then you should have no problem with most of the cheaper ones that are meant for individual use.

Right now I have only 9 groups and don't plan to have more than 25. I know this is tiny and the bandwidth of even a history pull would not register on a meter. It still doesn't make me comfortable.

What about when/if any of us get popular and you end up with 500 to 2000 posts or more per day going out to usenet? Hmm one password and 1000K outgoing posts each day, I think that one might trip a red flag or two.

I think the biggest reason for my fear is that I'm not on a dedicated box yet and can't do some of the cool things that you dedicated guys can. If I had a usenet feed service send a spam alert to my current host I could effectively have my sites totally shut down, not just the news feed. Even if the problem was with some trouble maker on the BBS it still filters down to me.

So I guess it comes down to a few choices for me either leave the usenet forums read only and use free servers, pay the money for either a dedicated server, or pay the higher buck news service that understands and accepts what we are doing.

I'm still searching, who knows I might come up with a service that will welcome us all with open arms. :D

That Ball and Ball account seems very cheap. I was paying $30 for my newsfeeds account before I left them, and that was just a normal personal account for home use. I might take a closer look at them myself :)

Keep us updated if you come across anymore bargains.

Hi together,
I installed the script and everything works fine, my users love it -
now some of the usenet - users told me they will sue me if I copy their post to my forum!
I thought usenet was public???
Is there any chance that I can use X-NoArchive: Yes or exlude by posters mail adress?

BTW, I use the gateway read only to avoid problems with the newsgroup - I just don't get it...
The newsgroup deals with cycling, the other with mountainbiking...

a shocked Tom

Once they submitted their post to usenet they submitted it to the public domain. Threaten to counter sue for wasting your time :)
You think Google and the bazillion other usenet/mailing list archives are paying royalties to everyone?

It is could manners to comply with the x-noarchive flag though (even though this isn't required as part of the usenet RFC). I'll add the option to the script.

bringing back the usenet discussion.

At the moment i am using spamkillers.usenet.com

The main reason is it seems to work ok but secondly the company does NOT have my credit card. Some credit card intermediary does, ibill i think and having dealt with these people for my script they do everything and i mean everything in the consumer's favour. If you emailed them after using your service for probably 2 months and said "i don't like this, it is not what i expected, i want a refund" they would give you one. No kidding.

What is the point, the point is that if something ***did*** go wrong there is nothing that this company could do about it. They couldn't just charge you some crazy hourly charge for their clean up operation. Of course this should not stop you making it happen but it does reduce your liability. The only thing they can really do is email your isp/webhost provider about it. They are likely to be more understanding.

Despite this we are of course all breaking the TOS. If anyone wants to poll a commercial feed then i am definitely up to contribute. If we can get one for $100 a month and 10 of us pay $10 we are doing well. Our bandwidth is going to be insignificant with most people having less than 25 newsgroups, i would imagine on average that is what, 10MB a day absolute maximum. It would require one person to sign up and the rest of us to send in $10 a month. (although whether you are allowed to resale is another point!).

I am so nearly there :)

last thing. What would be the code i need to get this to run via cron but to create a log file of the output of the shell script (so i can check it is running properly).

i can work out the cron time bit but do i just add > /path/to/log.txt sort of thing?

also, do you think you could release some standard spam filters?

To generate a log from your cron job use something like this:
If you want the output emailed to you, just leave the redirection bits off completely. The output will be mailed to the owner of the cron job.

where can i download the newest version of this hack für vB 2.0 final?

The latest version is always in the first post of this thread, and also at http://britishexpats.com/download/usenet_gateway.tar.gz

ok, thanks, just thought, that this version is for the beta, not for the final

Can you already estimate when a next version will be released?
Thanks in advance

I've been really short of free time lately but I hope to get a new version out this weekend.

Any hope that binary attachments will be in the new update. I am very excited about being able to add this feature to the bulletin board.
:D

I plan on having a very early test of it available. It will be an option in the CP. If you turn it on, I wouldn't risk using it on a production site just yet. :)

Glad to hear you are already adding the option. I can imagine how it will have to be beta-runned with the alt.binaries groups though. Some of the groups have a ton of pictures and that would fill up the database really fast.
Thanks again for adding the option:D

I'm setting up all usenet forums as moderated posting only so I can keep a close eye on the spam issue.

I've found that if I set a forum to moderated posting, newnews.pl does not check the moderator validated flag and will post to usenet even though the post is still pending validation.

Also, I am having problems with local reply posts not making it to usenet. I'm wondering if this could be related to the moderated posting only setting. I didn't have this problem when I was testing without moderated posting on previous version.

The threads MIS Test 1 and MIS Test 2 show local replies but the replies do not show on usenet, and MIS Test 3 is a post waiting validation and dose not show locally, but does show up on usenet.

http://www.mis-forums.com/vbbs/forum...php?forumid=67

news:alt.test.test

I hope you're making progress on the x-no-archive switch. That switch and moderated posting are the last pieces I need before opening the doors to usenet posting on our forums.

Howdy,

I'm having a problem .. I can't figure out what it's supposed to do

I've installed it exactly like the instructions said..

I've added a newsgroup .. and directed to a forum .. but nothing

it says in the Use Net Options 61 posts . then I goto the main forum and nothing is there..

Hi Sinema,

A link to your BBS may help in figuring this out.

Does the forum that you setup for your usenet posts show up empty, or not at all? Double check all of that forums CP settings and make sure that you have forum active checked yes.

That's about all I can come up with right now.

<a href="http://kickme.to/buffybb/" target="_blank">http://kickme.to/buffybb/</a>

See it doesn't do anything...

I've created a forum called

alt.tv.buffy-v-slayer

and directed that newsgroup to go there..

but the forum you can see .. anything else is gone.

and I have no errors anywhere

I'm hoping I can get a set up like dBforums.com has .. now that is awesome :-)

except of course mine would be the alt.tv areas lol but that place is nice!

Also I'm curious

Do I put my ISP info into the username and password slot to log into a news server?

I placed my ISP's news server in the default slot because I had no idea what to put there..

After you installed this hack have you been able to display *any* vBulletin pages? If not you have to go back and figure out why... somewhere along the line you made a mistake that is not allowing vBulletin to work.

Even without posts in your usenet forum you should still be able to reach other forums and it looks like you can't so this is a more sever error you will have to backtrack to find.

For the main news server you can try your ISP and if your ISP requires login and password you must supply that too. The news server and password fields under the individual groups are for times when you want or need to pull news from different servers for specific groups that may not be available on your main news server. They can be left blank until you have a need.

Hope this helps... I'm out for the day, I need some sleep. Good luck!

Cheers!