Fixing vBulletin and bringing the hacker community back together.

[buro9]

I want to gauge an opinion here, and preferably with some input from users of everythingVb, vBulletin.nl, and all of the other hack sites. However I shall only post here solely to allow the discussion to exist in one place.

There are essentially two things that I would like to ponder:

1. Turning vBulleting into an Enterprise product and adding a developer API.
2. Giving the hack community a single, searchable breakdown of all hacks, template changes and modifications.

Turning vBulleting into an Enterprise product and adding a developer API.

As I see it vBulletin is quite simply the best forum software there is, I've been using it for ages and hacking the hell out of mine for almost as long.

Now, I work as a Senior Developer and in the past few years have worked for British Telecom and a few other FTSE100 companies... all of which had a need for forum software, none of which purchased vBulletin no matter how much I put the case forward.

To my mind, vBulletin has never cracked the Enterprise market... and in turn we lesser users haven't benefited from the improvements that an Enterprise version would bring us (improved stability, normalised database, external user database, database abstraction layer, choice of scripting language, increase scalability).

When I worked at Premium TV (we did football sites for the UK football clubs), we could expect 40,000 people online on a Saturday afternoon at one time. Nowhere could we find evidence that vBulletin could cope with this, and table locking in MySql suggested to us that it simply could not. Further... with 80 clubs to run sites for we would've purchased 80 licenses, however we would only have desired 1 codebase to maintain.



So here is my proposal: That the community of hackers take it upon themselves to offer a major hack to produce an Enterprise version of vBulletin.

My experience would say that these were some of the requirements of this:

• Database Normalisation: Do away with MySql specific features or MySql based tricks (such as overloaded ID columns where a join would have been better).
• Database Abstraction Layer: To have a plugin file hold the queries and take advantage of specific database features (the Oracle plugin would use Stored Procs and bind variables for example, and maybe even the Intermedia cartridge for searching), this is dependant on normalisation.
• External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.
• Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.
• A developer API for performing common tasks, such as sending PM's, E-mails, etc. This would need to be for each scripting language and would assist the companies in developing rapidly their own extensions without having to delve deeply into the vBulletin codebase.
• vBulletin Engine: Add a site flag and a site_hosts table to map domains to configurations. In effect to allow one codebase to power many sites (Each URL would have to be a seperate paid license of course).

Now that is an enormous amount of work... and something would be daunting for anyone. It's tantamount to a re-write of the ways in which vBulletin does things, almost a core re-write to accomodate some things (like database normalisation and abstraction).

The question would be asked whether this is an extension (major) to vBulletin or a seperate product (vBulletin Enterprise?), and in the case of the latter the relationship that would have to Jelsoft (would they sell it and take upon the additional staff, etc?) or whether it would be legit to create a totally seperate product, open source or paid... etc, etc. But for now the question is whether this could be achieved in the first place... the best way to go about it can be answered later.

But the benefits are huge, not least the choice of many databases very easily and the advantages each offers (row locking, permissioning, stored procs, tree-walking, transactions, searching, etc). But also for those with Windows boxes to be able to pick ASP .Net and C# and those with UNIX boxes to select a Java Tomcat version.

Organisations of this size already have Oracle DBA's, or Sql Server DBA's, they aren't especially willing to take on MySql.
They already have server farms and load balancing and redundancy for their Windows or UNIX web servers... they aren't willing to install PHP and manage that.
They already have their user database and a single-signon policy, they aren't willing to make their users register again.

So the above proposals are to help solve all of those and in turn give the vBulletin community an Enterprise branch of code that will be able to scale to tens of thousands online simultaneously (of course dependant on your hardware, but you can scale through multiple web servers with load balancing in front, caching for non-dynamic things and using Oracle clustering or a similar technology at the back).


Giving the hack community a single, searchable directory of all hacks, template changes and modifications.

Years of watching the hacker community splinter in ego driven tugs of war and petulance. Well I would love to say don't go there, but we have and the damage has already been done.

So what I propose here is not a forum! It's a Wiki... a simple web-site changeable by all, to be non-promoting of each community site and to act as a factual and searchable repository of all of the vBulletin hacks and extensions that exist, those that are paid for and those that are given back to the community for free.

It would NOT be a place to publish hacks, it would simply be a place to find out if a hack exists and where it can be found.

So this proposal is simply to use a Wiki to create a searchable directory of vBulletin hacks, modifications, additions, extensions, template changes, etc.

These would not be categorised by the type of change (template, code, new file), but by what they achieve... so the blink on new PM hack AND template change would both exist in the same place: Forum Home Enhancements.

No non-factual information would be stored... There would be none of the "This is the bestest and greatest hack for... " type thing... it would be just a factual description only.

As such it would act as a gateway to all of the communities, and the glue between them.

Now a fair few of the heavy users would argue that there is no need for this, that we know where to find things... yes, we do... but newbies and less frequent visitors do not, and remain lost in all of these sites and communities not understanding the personal politics that exist between some and the strange policies that each site has about how hacks, etc are published.

So that is the vBulletin Directory.


Summary
So there were the two proposals: One is an enormous piece of work to open vBulletin up to corporate customers and the other a way to help vBulletin administrators locate all changes across all of the developer communities and glue it all back together.

These are only proposals.

I think it's good to discuss such stuff, especially since the latter one shouldn't be driven by any particular owner or ego.

So please feedback what you think of them.

Cheers

David K

[Floris]

Originally Posted by buro9

I want to gauge an opinion here, and preferably with some input from users of everythingVb, vBulletin.nl, and all of the other hack sites. However I shall only post here solely to allow the discussion to exist in one place.

There are essentially two things that I would like to ponder:

1. Turning vBulleting into an Enterprise product and adding a developer API.
2. Giving the hack community a single, searchable breakdown of all hacks, template changes and modifications.

Turning vBulleting into an Enterprise product and adding a developer API.

As I see it vBulletin is quite simply the best forum software there is, I've been using it for ages and hacking the hell out of mine for almost as long.

Now, I work as a Senior Developer and in the past few years have worked for British Telecom and a few other FTSE100 companies... all of which had a need for forum software, none of which purchased vBulletin no matter how much I put the case forward.

To my mind, vBulletin has never cracked the Enterprise market... and in turn we lesser users haven't benefited from the improvements that an Enterprise version would bring us (improved stability, normalised database, external user database, database abstraction layer, choice of scripting language, increase scalability).

When I worked at Premium TV (we did football sites for the UK football clubs), we could expect 40,000 people online on a Saturday afternoon at one time. Nowhere could we find evidence that vBulletin could cope with this, and table locking in MySql suggested to us that it simply could not. Further... with 80 clubs to run sites for we would've purchased 80 licenses, however we would only have desired 1 codebase to maintain.



So here is my proposal: That the community of hackers take it upon themselves to offer a major hack to produce an Enterprise version of vBulletin.

My experience would say that these were some of the requirements of this:

• Database Normalisation: Do away with MySql specific features or MySql based tricks (such as overloaded ID columns where a join would have been better).
• Database Abstraction Layer: To have a plugin file hold the queries and take advantage of specific database features (the Oracle plugin would use Stored Procs and bind variables for example, and maybe even the Intermedia cartridge for searching), this is dependant on normalisation.
• External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.
• Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.
• A developer API for performing common tasks, such as sending PM's, E-mails, etc. This would need to be for each scripting language and would assist the companies in developing rapidly their own extensions without having to delve deeply into the vBulletin codebase.
• vBulletin Engine: Add a site flag and a site_hosts table to map domains to configurations. In effect to allow one codebase to power many sites (Each URL would have to be a seperate paid license of course).

Now that is an enormous amount of work... and something would be daunting for anyone. It's tantamount to a re-write of the ways in which vBulletin does things, almost a core re-write to accomodate some things (like database normalisation and abstraction).

The question would be asked whether this is an extension (major) to vBulletin or a seperate product (vBulletin Enterprise?), and in the case of the latter the relationship that would have to Jelsoft (would they sell it and take upon the additional staff, etc?) or whether it would be legit to create a totally seperate product, open source or paid... etc, etc. But for now the question is whether this could be achieved in the first place... the best way to go about it can be answered later.

But the benefits are huge, not least the choice of many databases very easily and the advantages each offers (row locking, permissioning, stored procs, tree-walking, transactions, searching, etc). But also for those with Windows boxes to be able to pick ASP .Net and C# and those with UNIX boxes to select a Java Tomcat version.

Organisations of this size already have Oracle DBA's, or Sql Server DBA's, they aren't especially willing to take on MySql.
They already have server farms and load balancing and redundancy for their Windows or UNIX web servers... they aren't willing to install PHP and manage that.
They already have their user database and a single-signon policy, they aren't willing to make their users register again.

So the above proposals are to help solve all of those and in turn give the vBulletin community an Enterprise branch of code that will be able to scale to tens of thousands online simultaneously (of course dependant on your hardware, but you can scale through multiple web servers with load balancing in front, caching for non-dynamic things and using Oracle clustering or a similar technology at the back).


Giving the hack community a single, searchable directory of all hacks, template changes and modifications.

Years of watching the hacker community splinter in ego driven tugs of war and petulance. Well I would love to say don't go there, but we have and the damage has already been done.

So what I propose here is not a forum! It's a Wiki... a simple web-site changeable by all, to be non-promoting of each community site and to act as a factual and searchable repository of all of the vBulletin hacks and extensions that exist, those that are paid for and those that are given back to the community for free.

It would NOT be a place to publish hacks, it would simply be a place to find out if a hack exists and where it can be found.

So this proposal is simply to use a Wiki to create a searchable directory of vBulletin hacks, modifications, additions, extensions, template changes, etc.

These would not be categorised by the type of change (template, code, new file), but by what they achieve... so the blink on new PM hack AND template change would both exist in the same place: Forum Home Enhancements.

No non-factual information would be stored... There would be none of the "This is the bestest and greatest hack for... " type thing... it would be just a factual description only.

As such it would act as a gateway to all of the communities, and the glue between them.

Now a fair few of the heavy users would argue that there is no need for this, that we know where to find things... yes, we do... but newbies and less frequent visitors do not, and remain lost in all of these sites and communities not understanding the personal politics that exist between some and the strange policies that each site has about how hacks, etc are published.

So that is the vBulletin Directory.


Summary
So there were the two proposals: One is an enormous piece of work to open vBulletin up to corporate customers and the other a way to help vBulletin administrators locate all changes across all of the developer communities and glue it all back together.

These are only proposals.

I think it's good to discuss such stuff, especially since the latter one shouldn't be driven by any particular owner or ego.

So please feedback what you think of them.

Cheers

David K

vBulletin.nl is not a hack site, it is a vBulletin fan site. There is a source code section yes, but that doesn't mean it is a hack site. Don't pull my site into your quests. Thank you.

[MindTrix]

On your first suggestion wouldnt that be nearly re-making vB which is up to the vB team not us? And we wouldnt be able to release the files would we, mearly write down all the changes and from what you said, would be alot.

On the second point, i dont see whats difficult about going to vb.org for hacks and vb.org for templates and using their search engines :P:P

Just my 2 cents

[13th_Disciple]

Originally Posted by floris

Don't pull my site into your quests. Thank you.

I don't see where your site was specifically mentioned in his post.. It is an opinion about how to bring together all for a singular common good. I see nothing bad about that and don't see it as a "quest" of any kind..

Nice post, buro9.. i for one think it is a good idea. But I doubt there is much, if any formal support for either of your suggestions.. Also, the open source idea, it generally doesn't fly around here as everyone "has to get paid" for their services or it isn't worth their time..

i also don't believe some of the things you are looking for out of MySQL are available yet.. Isn't it ver. 5 that will finally support stored procedures? t may do it now, but I thought it was 5 that will support it.. Also, the external user information would make it easy to develop a ton of other things.. as well as be able to pass that information across different authentication protocols..

i really like the vBulletin software, but I really can't stand some of the attitudes from some of the people involved in the heirarchy of it all.. It makes it seem as if you must be loyal to them or worthy of their time before anyone anywhere invlves themselves in anything someone may ask.. there have been a few helpful folks, but most of them are few and far between anymore.. and with sites spreading out, people not caring and no one in the upper eschelon paying attention, it will only get worse..

that is also my 2 cents.. but I am not the only one here who thinks things are falling apart. and aparently, offering ideas to pull things together such as buro9 suggested, already get the drop kick right from the get go.. gg folks..

[sabret00the]

not a bad idea but have you actually spoken to anyone over at .com about this?

[Dean C]

Let me just pick up on two points:

# External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.

This as many people here have been saying is a good idea. I'd like to see an option to store all user related data in an external database with the option to link to it. That'd make it a lot easier to have cross-site user databases. I have a feeling that this'll be added in vB3.1 as you wouldn't believe the time we've been asked this here.

# Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.

Well vBulletin is a soley PHP program. Alongside with keeping up with features, bugs and other support-related issues - I can never see this happening

[Brad]

IMO the one thing jelsoft needs to do is build a database api for vBulletin. This would allow you to run vbulletin on a wide rang of database servers. I don't see the point in trying to have multiple sets of the source code, I think that would hurt development more then it would help.

[13th_Disciple]

then make an open source distribution of vBulletin.. comes with zero support, zero aid from jelsoft proper.. and contributions are dependant on community input alone.. therefore it can be ported to whatever a person sees fit..

so for a full service, fully supported, ready to run forum setup, you buy your license.. else you opt for an open source code for vBulletin that has none of the above.. it has the basic functionality to run, but past that, it's all up to the end user and community to figure it out..

and to point to a very good case.. FreeBSD.. it is fully free, ready to run, but zero support except from the community.. however, you can get the handbook, all CD's and such for about 60 dollars..

it can also open up a lot of folks to input ideas and info into vBulletin that otherwise have no way to buy a leased or owned license..

[ap0c]

Originally Posted by 13th_Disciple

then make an open source distribution of vBulletin.. comes with zero support, zero aid from jelsoft proper.. and contributions are dependant on community input alone.. therefore it can be ported to whatever a person sees fit..

..

jelsoft would lose alot of money on that deal. Most people like free things better than paying for them,bugs or not

[13th_Disciple]

or potentially open their business to an entire level that has not yet been tapped..

besides.. if this draws little interest.. it would die rather quickly.. i seriously doubt it would put as much of a dent in their pocket book as some folks think..