Automating testing

[AndrewD]

Has anyone any suggestions on ways of automating the testing of vb hacks? I've reached a point where it is impossible to run anything like complete regression tests, and even the straightforward manual tests have become extremely tedious.

[Dismounted]

I don't see how you can really automate testing, as each modification will require different scenarios.

[AndrewD]

Originally Posted by Dismounted

I don't see how you can really automate testing, as each modification will require different scenarios.

You develop standard test suites for the modification, which then get run as part of a regression process for new releases. What I'm looking for is some sense of the best way of doing these for a web-based application built around vbulletin.

try/error is the best regression test possible... the more you code, the more you know how codes have to be made anyway.. you can't be perfect on first try... (ok, i am, but hey, i have 20 years of coding in the hands.)

[AndrewD]

Originally Posted by nexialys

try/error is the best regression test possible... the more you code, the more you know how codes have to be made anyway.. you can't be perfect on first try... (ok, i am, but hey, i have 20 years of coding in the hands.)

Ditto, my friend, I started serious coding in 1973, including professional responsibility for leading the maintenance team responsible for a million line commercial package, but careers and approaches move on. I do understand the value of hand-on testing but also the need for more automated approaches when the task become too big.

To give a sense of scale, the modification I'm talking about is currently about 25K lines of code, and has gone through over a hundred main releases over four full versions of vbulletin during the last five years, each time offering significant changes in feature set and extensions. I've not seen any regression tools that are well-suited to handling the testing task. Has anyone else?

if YOU did not find anything, who could, seriously ?!...

[Adrian Schneider]

nexia... please

I'm assuming you are talking about your download manager? Usually unit testing is done with object oriented code, because then you re-use much more code than you would be otherwise making it easy to test. Test small chunks independently, and then it will all click together if it is planned out properly.

Fire up simpletest or your favorite testing application and begin writing unit tests for your classes and functions. If your code is broken into logical chunks as it should be, then testing will be easy to do. Most vB mods aren't written in OOP or if they are they are just large library files and not really utilising it as much as they could.

If you cannot test the code, then rewrite the code so it is testable.

If you want more help with this I'd probably send you to the PHP application design forum at sitepoint as there are a few helpful people there who are very active. The fact that it is a vBulletin modification should have little bearing on the ability to test it.

[AndrewD]

Originally Posted by SirAdrian

nexia... please

I'm assuming you are talking about your download manager? Usually unit testing is done with object oriented code, because then you re-use much more code than you would be otherwise making it easy to test. Test small chunks independently, and then it will all click together if it is planned out properly.

Fire up simpletest or your favorite testing application and begin writing unit tests for your classes and functions. If your code is broken into logical chunks as it should be, then testing will be easy to do. Most vB mods aren't written in OOP or if they are they are just large library files and not really utilising it as much as they could.

If you cannot test the code, then rewrite the code so it is testable.

If you want more help with this I'd probably send you to the PHP application design forum at sitepoint as there are a few helpful people there who are very active. The fact that it is a vBulletin modification should have little bearing on the ability to test it.

Thanks for pointing me to sitepoint. I don't really need a lecture on coding standards - I've been responsible for giving too many people such lectures in my time and it wears thin.

I'm sure you understand that while OO-design is extremely powerful, it is not a panacea for the systematic testing of many different options and localising external code instabilities, particularly when the base product is in a state of flux. The problem is not only the new features that I write myself - it's their overall integration alongside many different products and add-ons into users' sites, and people's expectation of absolute reliability from day 1.

vb itself was not written in an OO style until well into the development of LDM, and is still in large part not OO. php functionality is not yet stable, and one has to work with other people's hacks, on sites that are running in multiple languages, multiple operating systems and multiple versions of php/sql. So developing add-on code that is supposed to live smoothly through these releases is not just down to the quality of the local design, and a degree of automation would be useful.

Like you, I'm also disappointed that so many on this site seem to prefer to be smart-alecs rather than try to give helpful answers to the question asked.

[Adrian Schneider]

vBulletin's code is a pain to test just as yours will be because it's not broken up enough. It may be a lecture, but there is truth to it. The more automation, the more you can test automatically. FYI vBulletin is moving to a full OOP design because it's easier to make changes to and test. Anyway ...

What are you wanting to test, exactly? Do you want to test your code on several environments often, or quickly test the code function itself?

and yeah... please ignore nexials.