Fixing vBulletin and bringing the hacker community back together.

[buro9]

I want to gauge an opinion here, and preferably with some input from users of everythingVb, vBulletin.nl, and all of the other hack sites. However I shall only post here solely to allow the discussion to exist in one place.

There are essentially two things that I would like to ponder:

Turning vBulleting into an Enterprise product and adding a developer API.
Giving the hack community a single, searchable breakdown of all hacks, template changes and modifications.


Turning vBulleting into an Enterprise product and adding a developer API.

As I see it vBulletin is quite simply the best forum software there is, I've been using it for ages and hacking the hell out of mine for almost as long.

Now, I work as a Senior Developer and in the past few years have worked for British Telecom and a few other FTSE100 companies... all of which had a need for forum software, none of which purchased vBulletin no matter how much I put the case forward.

To my mind, vBulletin has never cracked the Enterprise market... and in turn we lesser users haven't benefited from the improvements that an Enterprise version would bring us (improved stability, normalised database, external user database, database abstraction layer, choice of scripting language, increase scalability).

When I worked at Premium TV (we did football sites for the UK football clubs), we could expect 40,000 people online on a Saturday afternoon at one time. Nowhere could we find evidence that vBulletin could cope with this, and table locking in MySql suggested to us that it simply could not. Further... with 80 clubs to run sites for we would've purchased 80 licenses, however we would only have desired 1 codebase to maintain.



So here is my proposal: That the community of hackers take it upon themselves to offer a major hack to produce an Enterprise version of vBulletin.

My experience would say that these were some of the requirements of this:

Database Normalisation: Do away with MySql specific features or MySql based tricks (such as overloaded ID columns where a join would have been better).
Database Abstraction Layer: To have a plugin file hold the queries and take advantage of specific database features (the Oracle plugin would use Stored Procs and bind variables for example, and maybe even the Intermedia cartridge for searching), this is dependant on normalisation.
External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.
Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.
A developer API for performing common tasks, such as sending PM's, E-mails, etc. This would need to be for each scripting language and would assist the companies in developing rapidly their own extensions without having to delve deeply into the vBulletin codebase.
vBulletin Engine: Add a site flag and a site_hosts table to map domains to configurations. In effect to allow one codebase to power many sites (Each URL would have to be a seperate paid license of course).


Now that is an enormous amount of work... and something would be daunting for anyone. It's tantamount to a re-write of the ways in which vBulletin does things, almost a core re-write to accomodate some things (like database normalisation and abstraction).

The question would be asked whether this is an extension (major) to vBulletin or a seperate product (vBulletin Enterprise?), and in the case of the latter the relationship that would have to Jelsoft (would they sell it and take upon the additional staff, etc?) or whether it would be legit to create a totally seperate product, open source or paid... etc, etc. But for now the question is whether this could be achieved in the first place... the best way to go about it can be answered later.

But the benefits are huge, not least the choice of many databases very easily and the advantages each offers (row locking, permissioning, stored procs, tree-walking, transactions, searching, etc). But also for those with Windows boxes to be able to pick ASP .Net and C# and those with UNIX boxes to select a Java Tomcat version.

Organisations of this size already have Oracle DBA's, or Sql Server DBA's, they aren't especially willing to take on MySql.
They already have server farms and load balancing and redundancy for their Windows or UNIX web servers... they aren't willing to install PHP and manage that.
They already have their user database and a single-signon policy, they aren't willing to make their users register again.

So the above proposals are to help solve all of those and in turn give the vBulletin community an Enterprise branch of code that will be able to scale to tens of thousands online simultaneously (of course dependant on your hardware, but you can scale through multiple web servers with load balancing in front, caching for non-dynamic things and using Oracle clustering or a similar technology at the back).


Giving the hack community a single, searchable directory of all hacks, template changes and modifications.

Years of watching the hacker community splinter in ego driven tugs of war and petulance. Well I would love to say don't go there, but we have and the damage has already been done.

So what I propose here is not a forum! It's a Wiki... a simple web-site changeable by all, to be non-promoting of each community site and to act as a factual and searchable repository of all of the vBulletin hacks and extensions that exist, those that are paid for and those that are given back to the community for free.

It would NOT be a place to publish hacks, it would simply be a place to find out if a hack exists and where it can be found.

So this proposal is simply to use a Wiki to create a searchable directory of vBulletin hacks, modifications, additions, extensions, template changes, etc.

These would not be categorised by the type of change (template, code, new file), but by what they achieve... so the blink on new PM hack AND template change would both exist in the same place: Forum Home Enhancements.

No non-factual information would be stored... There would be none of the "This is the bestest and greatest hack for... " type thing... it would be just a factual description only.

As such it would act as a gateway to all of the communities, and the glue between them.

Now a fair few of the heavy users would argue that there is no need for this, that we know where to find things... yes, we do... but newbies and less frequent visitors do not, and remain lost in all of these sites and communities not understanding the personal politics that exist between some and the strange policies that each site has about how hacks, etc are published.

So that is the vBulletin Directory.


Summary
So there were the two proposals: One is an enormous piece of work to open vBulletin up to corporate customers and the other a way to help vBulletin administrators locate all changes across all of the developer communities and glue it all back together.

These are only proposals.

I think it's good to discuss such stuff, especially since the latter one shouldn't be driven by any particular owner or ego.

So please feedback what you think of them.

Cheers

David K

[Floris]

I want to gauge an opinion here, and preferably with some input from users of everythingVb, vBulletin.nl, and all of the other hack sites. However I shall only post here solely to allow the discussion to exist in one place.

There are essentially two things that I would like to ponder:

Turning vBulleting into an Enterprise product and adding a developer API.
Giving the hack community a single, searchable breakdown of all hacks, template changes and modifications.


Turning vBulleting into an Enterprise product and adding a developer API.

As I see it vBulletin is quite simply the best forum software there is, I've been using it for ages and hacking the hell out of mine for almost as long.

Now, I work as a Senior Developer and in the past few years have worked for British Telecom and a few other FTSE100 companies... all of which had a need for forum software, none of which purchased vBulletin no matter how much I put the case forward.

To my mind, vBulletin has never cracked the Enterprise market... and in turn we lesser users haven't benefited from the improvements that an Enterprise version would bring us (improved stability, normalised database, external user database, database abstraction layer, choice of scripting language, increase scalability).

When I worked at Premium TV (we did football sites for the UK football clubs), we could expect 40,000 people online on a Saturday afternoon at one time. Nowhere could we find evidence that vBulletin could cope with this, and table locking in MySql suggested to us that it simply could not. Further... with 80 clubs to run sites for we would've purchased 80 licenses, however we would only have desired 1 codebase to maintain.



So here is my proposal: That the community of hackers take it upon themselves to offer a major hack to produce an Enterprise version of vBulletin.

My experience would say that these were some of the requirements of this:

Database Normalisation: Do away with MySql specific features or MySql based tricks (such as overloaded ID columns where a join would have been better).
Database Abstraction Layer: To have a plugin file hold the queries and take advantage of specific database features (the Oracle plugin would use Stored Procs and bind variables for example, and maybe even the Intermedia cartridge for searching), this is dependant on normalisation.
External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.
Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.
A developer API for performing common tasks, such as sending PM's, E-mails, etc. This would need to be for each scripting language and would assist the companies in developing rapidly their own extensions without having to delve deeply into the vBulletin codebase.
vBulletin Engine: Add a site flag and a site_hosts table to map domains to configurations. In effect to allow one codebase to power many sites (Each URL would have to be a seperate paid license of course).


Now that is an enormous amount of work... and something would be daunting for anyone. It's tantamount to a re-write of the ways in which vBulletin does things, almost a core re-write to accomodate some things (like database normalisation and abstraction).

The question would be asked whether this is an extension (major) to vBulletin or a seperate product (vBulletin Enterprise?), and in the case of the latter the relationship that would have to Jelsoft (would they sell it and take upon the additional staff, etc?) or whether it would be legit to create a totally seperate product, open source or paid... etc, etc. But for now the question is whether this could be achieved in the first place... the best way to go about it can be answered later.

But the benefits are huge, not least the choice of many databases very easily and the advantages each offers (row locking, permissioning, stored procs, tree-walking, transactions, searching, etc). But also for those with Windows boxes to be able to pick ASP .Net and C# and those with UNIX boxes to select a Java Tomcat version.

Organisations of this size already have Oracle DBA's, or Sql Server DBA's, they aren't especially willing to take on MySql.
They already have server farms and load balancing and redundancy for their Windows or UNIX web servers... they aren't willing to install PHP and manage that.
They already have their user database and a single-signon policy, they aren't willing to make their users register again.

So the above proposals are to help solve all of those and in turn give the vBulletin community an Enterprise branch of code that will be able to scale to tens of thousands online simultaneously (of course dependant on your hardware, but you can scale through multiple web servers with load balancing in front, caching for non-dynamic things and using Oracle clustering or a similar technology at the back).


Giving the hack community a single, searchable directory of all hacks, template changes and modifications.

Years of watching the hacker community splinter in ego driven tugs of war and petulance. Well I would love to say don't go there, but we have and the damage has already been done.

So what I propose here is not a forum! It's a Wiki... a simple web-site changeable by all, to be non-promoting of each community site and to act as a factual and searchable repository of all of the vBulletin hacks and extensions that exist, those that are paid for and those that are given back to the community for free.

It would NOT be a place to publish hacks, it would simply be a place to find out if a hack exists and where it can be found.

So this proposal is simply to use a Wiki to create a searchable directory of vBulletin hacks, modifications, additions, extensions, template changes, etc.

These would not be categorised by the type of change (template, code, new file), but by what they achieve... so the blink on new PM hack AND template change would both exist in the same place: Forum Home Enhancements.

No non-factual information would be stored... There would be none of the "This is the bestest and greatest hack for... " type thing... it would be just a factual description only.

As such it would act as a gateway to all of the communities, and the glue between them.

Now a fair few of the heavy users would argue that there is no need for this, that we know where to find things... yes, we do... but newbies and less frequent visitors do not, and remain lost in all of these sites and communities not understanding the personal politics that exist between some and the strange policies that each site has about how hacks, etc are published.

So that is the vBulletin Directory.


Summary
So there were the two proposals: One is an enormous piece of work to open vBulletin up to corporate customers and the other a way to help vBulletin administrators locate all changes across all of the developer communities and glue it all back together.

These are only proposals.

I think it's good to discuss such stuff, especially since the latter one shouldn't be driven by any particular owner or ego.

So please feedback what you think of them.

Cheers

David K
vBulletin.nl is not a hack site, it is a vBulletin fan site. There is a source code section yes, but that doesn't mean it is a hack site. Don't pull my site into your quests. Thank you.

[MindTrix]

On your first suggestion wouldnt that be nearly re-making vB which is up to the vB team not us? And we wouldnt be able to release the files would we, mearly write down all the changes and from what you said, would be alot.

On the second point, i dont see whats difficult about going to vb.org for hacks and vb.org for templates and using their search engines :P:P

Just my 2 cents

[13th_Disciple]

Don't pull my site into your quests. Thank you.

I don't see where your site was specifically mentioned in his post.. It is an opinion about how to bring together all for a singular common good. I see nothing bad about that and don't see it as a "quest" of any kind..

Nice post, buro9.. i for one think it is a good idea. But I doubt there is much, if any formal support for either of your suggestions.. Also, the open source idea, it generally doesn't fly around here as everyone "has to get paid" for their services or it isn't worth their time..

i also don't believe some of the things you are looking for out of MySQL are available yet.. Isn't it ver. 5 that will finally support stored procedures? t may do it now, but I thought it was 5 that will support it.. Also, the external user information would make it easy to develop a ton of other things.. as well as be able to pass that information across different authentication protocols..

i really like the vBulletin software, but I really can't stand some of the attitudes from some of the people involved in the heirarchy of it all.. It makes it seem as if you must be loyal to them or worthy of their time before anyone anywhere invlves themselves in anything someone may ask.. there have been a few helpful folks, but most of them are few and far between anymore.. and with sites spreading out, people not caring and no one in the upper eschelon paying attention, it will only get worse..

that is also my 2 cents.. but I am not the only one here who thinks things are falling apart. and aparently, offering ideas to pull things together such as buro9 suggested, already get the drop kick right from the get go.. gg folks..

[sabret00the]

not a bad idea but have you actually spoken to anyone over at .com about this?

[Dean C]

Let me just pick up on two points:


# External User Database: Store local user preferences and signature, etc, but to have done core authentication against an admin defined custom database schema or LDAP.


This as many people here have been saying is a good idea. I'd like to see an option to store all user related data in an external database with the option to link to it. That'd make it a lot easier to have cross-site user databases. I have a feeling that this'll be added in vB3.1 as you wouldn't believe the time we've been asked this here.


# Scripting Language Choice: Allow both Java and C# to be used as the front-end instead of PHP.


Well vBulletin is a soley PHP program. Alongside with keeping up with features, bugs and other support-related issues - I can never see this happening :)

[Brad]

IMO the one thing jelsoft needs to do is build a database api for vBulletin. This would allow you to run vbulletin on a wide rang of database servers. I don't see the point in trying to have multiple sets of the source code, I think that would hurt development more then it would help.

[13th_Disciple]

then make an open source distribution of vBulletin.. comes with zero support, zero aid from jelsoft proper.. and contributions are dependant on community input alone.. therefore it can be ported to whatever a person sees fit..

so for a full service, fully supported, ready to run forum setup, you buy your license.. else you opt for an open source code for vBulletin that has none of the above.. it has the basic functionality to run, but past that, it's all up to the end user and community to figure it out..

and to point to a very good case.. FreeBSD.. it is fully free, ready to run, but zero support except from the community.. however, you can get the handbook, all CD's and such for about 60 dollars..

it can also open up a lot of folks to input ideas and info into vBulletin that otherwise have no way to buy a leased or owned license..

[ap0c]

then make an open source distribution of vBulletin.. comes with zero support, zero aid from jelsoft proper.. and contributions are dependant on community input alone.. therefore it can be ported to whatever a person sees fit..

..
jelsoft would lose alot of money on that deal. Most people like free things better than paying for them,bugs or not

[13th_Disciple]

or potentially open their business to an entire level that has not yet been tapped..

besides.. if this draws little interest.. it would die rather quickly.. i seriously doubt it would put as much of a dent in their pocket book as some folks think..

[buro9]

jelsoft would lose alot of money on that deal. Most people like free things better than paying for them,bugs or not
I wasn't specifically trying to suggest open source as a route, I actually like Jelsoft and the point in an Enterprise edition was partly so that when I have to implement forums as part of my job that not only I can get the pleasure of working with software that I know and am comfortable with, but that Jelsoft would be making considerable money from the licenses and possibly a corporate support package (which both BT and PTV (as I had mentioned them) would pay a princely sum for).

My view is simply that vBulletin has part of the SME and nearly all of the home user 'hobbyist'/gamer markets sewn up. They lack penetration into the higher level markets, and thus the source of revenue in that space. Revenue (and innovation) which helps guarantee vBulletins longevity rather than leave the niche unfilled waiting for another product to appear.

As regards the directory... How many times have you seen something on a site and wondered how they did it? Where the hack was? So has to search at least: vBulletin.com, vBulletin.org, vBulletin.nl, The Admin Zone, vBulletin Templates, Everything VB to name but a few... to find out whether the modification you had seen had been published.

Similarly, when you want to develop a hack and wonder if it exists, don't you have to look all around again?

Do all vBulletin users have to do this? Are they even aware of all these sites? Most importantly though, is this the best thing that we (the community) can come up with to help the community get the most out of vBulletin? Should we not be seeking to find better ways of doing things?

As hack writer when I write hacks it's because I'm putting functionality onto my site and choose to publish the changes to help give back. Increasingly hack writers are having to ask where they should publish, what I've proposed would go some way to make it irrelevant... that a single repository listing all of the hacks and sending the admins to the right thread on the right site ensures that no hacks are lost and the effort in producing them wasted.

That's not a quest, it's a process. The same process that gave birth to vBulletin and nearly all other innovation by humans... the simple question of "Could this be better?", "Is there another way"? I just think that a neutral (run and modifiable by all - not owned by Jelsoft) Wiki would be better.

[Dean C]

vBulletin will never become open source - the idea is you pay for the product and support. I've never had to use support like a lot of people so I think Jelsoft would lose far too much money via this route.

[buro9]

* Sigh *

I didn't say that vBulletin should, would or could be open source, merely that if a major hack were developed to produce an Enterprise version, that the ownership of the hack might later need the benefits of Open Source to be developed... i.e. collaborative development and vision leadership. In no way at all does that imply that vBulletin code would ever be distributed or reproduced: It merely says that a hack of this size and nature and given the complex integration of it could possibly benefit from not being owned by an individual. Thus if such an undertaking were decided to be a positive thing and commenced, were any key developers to then quit the project, that the project life could be continued as that developer wouldn't "take their ball home with them".

Consider for example the life of the HTL if KuraFire to abandon it (sick of disclaiming, but... what IF he got sick of it, what IF he were hit by a bus, what IF his life priorities changed, what IF? etc etc... project chosen arbitrrily, etc, blah blah)... wouldn't those that now rely upon the hack find that their hacks needed re-working, etc... wouldn't several others in the community wish to continue that hack... well, they couldn't as KuraFire has placed a license on it preventing that and owning it.

So my only mention in relation to open source was based on several enormous IF's.
IF the community likes the idea of an Enterprise version
IF Jelsoft doesn't want to pursue this
IF the community wanted to pursue it
IF the scale and size of this project is anything near what you could image
and IF those who wanted to contribute felt that collaborative and open development would be the way forward
THEN they could CONSIDER open FOR THE HACK ALONE and not vBulletin, which of course you would have to own.

I would also like to disclaim whilst I'm here, anything said ever by me that you could possibly interpret as being negative or against whatever beliefs you have personally attached to vBulletin. I am a very clam and patient fellow, and my intentions are solely to try and improve vBulletin... and this thread is merely a DISCUSSION about what we could possibly achieve as a community to help make things better.

* Sigh *

[Floris]

I don't see where your site was specifically mentioned in his post.. It is an opinion about how to bring together all for a singular common good. I see nothing bad about that and don't see it as a "quest" of any kind..

Nice post, buro9.. i for one think it is a good idea. But I doubt there is much, if any formal support for either of your suggestions.. Also, the open source idea, it generally doesn't fly around here as everyone "has to get paid" for their services or it isn't worth their time..

i also don't believe some of the things you are looking for out of MySQL are available yet.. Isn't it ver. 5 that will finally support stored procedures? t may do it now, but I thought it was 5 that will support it.. Also, the external user information would make it easy to develop a ton of other things.. as well as be able to pass that information across different authentication protocols..

i really like the vBulletin software, but I really can't stand some of the attitudes from some of the people involved in the heirarchy of it all.. It makes it seem as if you must be loyal to them or worthy of their time before anyone anywhere invlves themselves in anything someone may ask.. there have been a few helpful folks, but most of them are few and far between anymore.. and with sites spreading out, people not caring and no one in the upper eschelon paying attention, it will only get worse..

that is also my 2 cents.. but I am not the only one here who thinks things are falling apart. and aparently, offering ideas to pull things together such as buro9 suggested, already get the drop kick right from the get go.. gg folks..Try the first sentence.

buro9
As regards the directory... How many times have you seen something on a site and wondered how they did it? Where the hack was? So has to search at least: vBulletin.com, vBulletin.org, vBulletin.nl, The Admin Zone, vBulletin Templates, Everything VB to name but a few... to find out whether the modification you had seen had been published.
Users shouldn't join my site to search for a hack, they should search for resources after registration and finding out there is also a resource section. Again, vbnl is a fan site, not a hack site - I'd appriciate it if I wasn't mentioned on the same list in this type of discussion - since it gives an incorrect assumption to the rest.

[filburt1]

IMO the one thing jelsoft needs to do is build a database api for vBulletin. This would allow you to run vbulletin on a wide rang of database servers. I don't see the point in trying to have multiple sets of the source code, I think that would hurt development more then it would help.
The one thing they need to do is actually document their functions, preferably with PHPDoc style so I know what the hell each argument means...

[buro9]

The one thing they need to do is actually document their functions, preferably with PHPDoc style so I know what the hell each argument means...

That I agree with too.

I'd love more comments in their code too... really plain stuff, the obvious.
I was coding in C, C++ and PERL before picking up PHP, and it's a nightmare that first time you read a massive block of code and wonder what it's doing.

It may be repetitive, but I think that there is a lot that can be gained from highly commenting it, not least that it would make the code less scary to newbies who want to try their hand at hacking and are stuck wondering just what escape_slashes() does.

Then there are things like common tasks... I think the key pieces of functionality (send a PM for example) should be wrapped in a nice little developer API... so fewer hacks required code duplication or file changes... they could exist as seperate files and just call the API's... which of course should be highly documented.

We could take it upon ourselves to do some of that... but this is core product stuff, and I doubt anyone will install a hack that simply adds comments! ;)

[Reeve of shinra]

Lately, I've been looking at other open source projects coded in php and I have a deeper appreciation for VB in how the code is commented. It also seems that hacks are fairly well organized around the vb world as well compared to some open source projects as well.

An enterprise edition of vbulletin may expand thier market presence, but that would be a whole new product for a different market segment. I think changing what we have today may actual lose the maket share they already have.

[Rein Masamuri]

Jelsoft should not go to an enterprise version. Java and the like restrict freedom in the product and limits the flexibility that is inherited by opensource like PHP which would cut down extremely if not completely the possibility to release hacks. It would also alienate their current base, whom enjoy said hackability. The price and lack of expandibility would also prevent people from honestly suggesting vBulletin as a viable software for their company as it wouldn't be able to grow with their company.

[sabret00the]

<a href="http://www.vbulletin.com/forum/showthread.php?t=97747" target="_blank">http://www.vbulletin.com/forum/showthread.php?t=97747</a>

this might be of interest buro

[Giveit2u43]

jelsoft would lose alot of money on that deal. Most people like free things better than paying for them,bugs or not
Jelsoft tried the free route a while back with vBulletin-Lite and decided they didn`t like it..

[sabret00the]

nah it was just a poor product and little development went into it. but theirs alot of sites out there that use vBlite.

[Dean C]

I personally think it was more a matter of they were effectively giving a reasonably good product out for free and people were using it for their main boards which was not part of the argeement to be agreed before downloading vBlite. At that time vBlite was better than some other message board software which tells you a lot ;)