vBulletin 3 Hacks & Quality Assurance at vB.org

[Erwin]

Keep the suggestions coming. We want the members to have input in to this system as it will affect the whole community.

[Vile]

Excellent idea indeed

[KuraFire]

Originally Posted by futureal

A rating system would need to be adjustable as future incarnations of a hack arrive. If a bug-ridden 1.0 is replaced by a stable 1.1 (or whatever) then it may not deserve the potentially shoddy rating it received at the onset.

The rating system doesn't have to be adjustable really, because we will give out this 'award' for only one (main) version of a Hack at a time. If the Hack is buggy at 1.0, it will simply not receive the Quality award. If the Hack at 1.1 is rewritten to have no more bugs (or no apparent ones, at the very least), the hack author can re-submit it for approval and we will then review it again. If at this point the Hack does indeed no longer have bugs, and is secure (no security leaks or dangers), and is running stable (doesn't randomly hog the server resources, or 'crashes'), it will be taken through a rating system for several criteria, and if it then scores well enough on those as well, it will be given the Quality label. If later on, then, a version 1.2 of the hack is released, it will have to be reviewed all over again, before we'll give the 1.2 version the Quality label.

Originally Posted by futureal

Feedback should be appreciated by any hack author (I know I appreciate any that I receive) but I am not sure that public critique is such a good idea (beyond the rating). If it were up to me, I would leave that private for the hack author. Having somebody put a lot of work into something only to see negative comments from experienced coders in public is not going to encourage them to keep working on it. Especially because something that takes me 20 minutes might take someone else 4 hours, or might take Chen 53 seconds, and so on. The chance of people thinking that other, more experienced hackers are "talking down" to them is high.

After reviewing of a Hack, the hack author will be told (regardless of whether or not the Hack was approved) what aspects of coding s?he could still improve upon.
We will aim as much as possible to not be condescending in that, so that people don't get the feeling we're talking down to them or anything. We want to make this system to encourage people to improve their coding skills, and we know that acting all stuck-up and what not is not going to help with that, so we'll make sure to be supportive, friendly, and down-to-earth.
It's good that you pointed it out though. :up:

Originally Posted by futureal

I think a publicly viewable stability rating (or whatever you want to call it) would suffice, along with private comments to the author about why such a rating is deserved and what he/she can do to improve upon it.

The Public will only see one thing: QA Approved: yes/no; (no can also mean it simply has not been submitted for approval).
The Hack Author will see two things: QA Approved (duh), and what areas s?he could still improve upon. This will be referenced using an extensive document will lots of useful tips and guidelines.
The Quality Assurance team will see whether it's been approved or not, will see what areas the hack author could improve upon, and see in a little more detail how the rating process has gone. The latter is important to have for when we encounter hacks where one Team member thinks is -just- good enough to Qualify, and another Team member thinks it's comes a little short.


At least, that's planned so far... if any member sees a problem with that, please tell! It's why we made this thread - to get everyone's opinion on it

[futureal]

Kura: I guess what you are saying is a bit different than what others had said earlier in the thread. Having a simple Yes/No Approved seems like a better idea to me than a percent-based rating. Either it meets the standards or it doesn't.

As far as public comments, I still agree that the Quality Assurance team should communicate with the author in private.

This whole system sounds like a nice idea for the end user, but risks making it more complicated and less friendly to the hacker. Part of what I like about vb.org is that there are tons of different ideas implemented in tons of different ways here, and anybody is free to do it. I wouldn't want to see this system "crowding out" some of the guys who come in just to write one hack here or there.

The other risk is that you are making the process more complicated. Just like my opinion of the hack subforums: the more unnecessarily complex things become, the less useful they become.

This is a good idea, it just needs careful implementation.

[ap0c]

Originally Posted by futureal

Kura: I guess what you are saying is a bit different than what others had said earlier in the thread. Having a simple Yes/No Approved seems like a better idea to me than a percent-based rating. Either it meets the standards or it doesn't.

As far as public comments, I still agree that the Quality Assurance team should communicate with the author in private.

This whole system sounds like a nice idea for the end user, but risks making it more complicated and less friendly to the hacker. Part of what I like about vb.org is that there are tons of different ideas implemented in tons of different ways here, and anybody is free to do it. I wouldn't want to see this system "crowding out" some of the guys who come in just to write one hack here or there.



This is a good idea, it just needs careful implementation.

I agree with the QA team discussing any issue with the author in private. People take comments the wrong the way at times. Maybe a standardized form should be filled out by the QA team and sent to author with basic comments and if the author has questions or simply disagrees with what the QA team "thinks" is the correct way to script something, the issue would be better handled in private. The less public arguements between hack authors and a QA team the better, imo for this site and people( end users) in general.

[KuraFire]

Discussions between Hack Author and QA Team concerning the submitted / approved / unapproved Hack will always be in private.

The rating criteria Erwin mentioned in the first post are for QA team in-house, the results of a review of a hack will not be made public via those. This to prevent endless bickering about details that aren't as important as the big picture: approved, yes or no?

[colicab-d]

grrrrr - eat as tony would say good idea chaps

[Jayk]

I think this is a great ideal.

Also if it can use laymen terms as far as descriptions wrt install. I muttled through a chat hack, got it to work but it took a while.

Great ideal and vision, kudos

[MindTrix]

I love the idea Kura suggested about a step by step guide to creating a hack beeing written up! Im sure it will be VERY usefull to newbie/wannabe hackers like me.

Request--

When vB3 hacks are released can we have them shown on the front page like the vB2 ones are? I was thinking of a column next to the vB2 one, but i guess it might look out of place.

[Xenon]

the vb3 hacks will be visible on the homepage yes.
most likely within the same window, we'll see