I was recently in Moscow to give a presentation on testing and quality at the Microsoft QA Day event there.  After my talk, I had a conversation with a software tester, who followed up with an e-mail.  I’ll respond here to Nataly’s questions.

Dear, Rex.

First of all I want to thank you for your speech on the Microsoft QA Days in Moscow. Do you like the snow in late March?

Well, it was a bit cold while I was there.  However, having my expectations shaped by books like War and Peace, Crime and Punishment, Ten Days that Shook the World, and Doctor Zhivago, the wintry landscape was exactly what I imagined.

You may remember, I came up to you at the end of the day with the question about participation of the test team in reviews and risk analysis. But because I have not had sufficient practice in conversational English, I could not properly ask the question

Certainly, Nataly, your English–written or conversational–is vastly superior to my Russian!

I will try to formulate the question more correctly. I’ll be very grateful if you could respond to my letter. Imagine that there is a test team with the appropriate skills, which should participate in review or risk analysis. Also imagine that the team was trained before the partisipation: they read the corresponding books about the review process or risk analysis process.

Well, Nataly, I’m not sure I would consider just reading a book on risk analysis or reviews sufficent training.  It will be for some people, but many of our clients find that they need a little more help than that before they can become truly effective.

Despite the preliminary training, we can expect that the result of the first participation of the team will be low or zero, because of the lack of experience. However, management expects that the money invested in training will pay off immediately. Therefore, it would be wise to prepare them in advance to that the benefits of the participation of the test team will not be visible immediately.

It seems that, if all management did was pay for two books, one describing risk analysis and another describing software reviews, they can hardly have high expectations about how much behavioral and capability change is going to result from that extremely small investment.  What they are going to pay for, in that situation, is the low efficiency associated with the series of practical review or risk analysis sessions that is necessary to learn-by-doing. This is exactly what you are describing.  It’s often more effecient to pay for a training session, since, if the training is carefully chosen and the participants apply themselves to it, the participants will leave the training ready to be effective in their first review.

Could you please tell based on your experience, how much time (in average) usually pass before the moment when the participation of testers will bring tangible benefit? 1 review? 2 reviews? 3 reviews…?

So, we have a one-day training for risk analysis, and a two-day training for reviews.  When we train people in how to do reviews or risk based testing, we find that they are immediately effective upon leaving that training. That’s not to say that they don’t continue to get better over time, but they are immediately capable of participating in an active and contributing fashion.  However, if all the people do is read a book, it’s hard to say what level of effectiveness they would have. 

The reason training is so much more effective than just reading a book is that training–at least good training–includes hands-on, practical, realistic exercises.  This is true for any subject.  Our training on risk based testing and reviews involves actually doing a risk analysis or reviewing a requirements specification.  The instructor is involved during the exercise, to guide the participants to success.  That way, the participants leave the training having actually, effectively and efficiently, carried out the process. 

With a book, even a book that includes exercises, there is no instructor there to help guide the reader through the process.  So, if the reader gets confused, or gets stuck, or thinks they know what they are doing but is actually wrong, the capability gained may be low.

I fear that managers might say something like: “testers spent so much time studying the theory, they spend time and money to the participation in review, so when we can see the benefit of their participation?”

Yes, this is a significant risk.  If testers get involved in activities like reviews and risk analysis, and they are not capable of carrying those activities out, that can cause significant damage to the test team’s credibility and lead to managers deciding not to include them in the future. 

Thanks in advance!

Sincerely, Nataly.

Thanks for getting in touch with me, Nataly.  I hope my response has been helpful.  If you do decide to pursue the avenue of reading books, please be sure that you arrange some exercises, just with the testers, before you try to participate in a real review or to initiate risk based testing.  However, I would recommend that you seek out good training, training that includes hands-on exercises and instructor support, to improve the odds of success. Alternatively, if you have a person in the organization who can mentor the testers, someone with experience in reviews or risk analysis, cross-training can also work.

Tags: quality risk analysis, reviews

This entry was posted on Sunday, April 1st, 2012 at 11:26 PM by Rex Black and is filed under reviews, risk based testing, software testing. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.