Following today’s webinar on test strategies, listener Cees Jan had some follow-up questions. I’ll address them below, with answer interspersed in the text.
Rex,
I really enjoyed your webinar.
Thanks, I’m glad it was useful for you.
Here is a question you did not get to: Hi I am what would qualify as a “falsifier”: Find the worst-case scenario and see how product behaves in terms of robustness and graceful termination. Quite the opposite of regression testing which I consider “trivial pursuit” (good for build stability and sanity, but never uncovering new defects). Would that be a wise strategy? I get comments that it is far-fetched, but I am stretching the limits of a system and see where it fails. It provides a lot of eye-openers. CeesJan
It sounds like, based on your earlier question the webinar (which I did answer), that you are basically following a reactive test strategy. You are apparently (based on this comment) focused on finding a lot of bugs, especially by checking out unusual circumstances and extreme conditions, which isa typical characteristic of reactive testing.
I’m a bit concerned about omitting regression testing, though. Far from being “trivial pursuit” (though I realize it’s not fun when done manually), in Agile lifecycles the high rate of change makes regression risk very significant. If you are no addressing regression risk, who is? Are the developers doing good automation unit (e.g., with J-unit) and feature verification tests (e.g., with Fitnesse)? If not, then the confidence-building and risk-mitigating objectives of testing are probably not being met.
The above sounds like reverse engineering: lack of requirements and undocumented legacy code force me to this “out of the box” methodology/averse strategy. Reading books like black box testing (Beizer) and black swan (Taleb) inspired me too much, I guess.
Certainly, in testing we have to play the hand we’re dealt. If you have no requirements and no useable documentation, then this will often force you into a purely reactive test strategy. However, you should be aware of how limited such a strategy is. I would think that adding analytical risk based testing, along with addressing the regression risks, would make sense.
Regards, Cees Jan den Haan (IQSTB Practitioner, CAT)
I have to correct you here, Cees Jan. The ISEB Practitioner certification is not part of the ISTQB program, and neither is CAT. The ISTQB program has the Foundation, Advanced, and Expert certifications. I know that some European training providers have created confusion on this point (perhaps for marketing purposes), but it’s wrong to associate those two certifications with the ISTQB.
Tags: agile testing, test strategies
Rex Black is President of RBCS (
