Posts Tagged ‘software reviews’
Monday, July 16th, 2012 by Rex Black
My friend and colleague from Colombia, Patricia Osorio, asks the following question:
Hi Rex
Could you please help me with the following question? Is it right to say the order of reviews according its formality – from most formal to less formal is: inspection, walkthrough, technical review and informal review? When I had read foundation level syllabus to present the exam (2005) it was very clear. Now, when I reread the foundation level syllabus in its version 2010, it seems to me that It has changed to inspection, technical review, walkthrough and informal review.
Thanks
Regards,
Patricia Osorio Aristizabal
Patricia, the latest version of the Foundation syllabus is 2011, but I think the text is much the same as the 2010 version. I have checked some of the previous versions of the syllabus, and can’t find any explicit mention of the spectrum of formality.
I believe this idea of the spectrum of formality (informal, technical review, walkthrough, inspection) comes from IEEE 1028. In that standard, metrics and review-based process improvement are not specified for the technical review, and they are for walkthroughs and inspections. So, this makes the technical review less formal. The inspection is more formal than a walkthrough due to the separation of moderator and author.
This leads to an interesting question: Does this spectrum of formality actually matter in the real world? In my experience, it really doesn’t. Here’s why I say that. Most companies don’t do reviews, or at least don’t do them anywhere near as often and as thoroughly as they should. So, when I’m talking to a client about doing reviews, I don’t get into the issue of what level of formality. Instead, my focus is on motivating them to start doing more reviews and doing them better. I can’t remember working with any clients where the main problem they had with reviews was that they weren’t using the right level of formality.
The other reason this doesn’t matter much is because of the naming issue. People use the terms “review,” “technical review,” “inspection,” “JAD session,” “walkthrough,” and more, and whenever I hear those terms I ask people to tell me, specifically, what such an event is, who is involved, what the process is, and–if people mention more than one type of review–what the differences are. I very rarely get clear answers to those questions, which tells me that any particular session where people sit down to discuss a work product could have any one of a dozen or so different names attached to it.
Personally I don’t see this as a problem to worry about. I’m more worried about whether my clients are doing reviews, regularly and with good benefit, than with what they call it.
Tags: IEEE standard, reviews, software reviews
Posted in reviews, software testing | 1 Comment »
Friday, October 7th, 2011 by Rex Black
Long time reader, Patricia Osorio Aristizabal, asked me to clarify the following question:
During a walkthrough meeting for a high-level design, a tester who is participating as a reviewer asks the system architect who created the document why only physical network connections are supported in the design, rather than also supporting wireless connections. The designer replies that they did not have time to address the security concerns raised by wireless access to the system. The tester accepts that answer and the review proceeds.
Which of the following is an objective of a review that was achieved in that exchange?
A. Increased reviewer understanding
B.Early detection of a defect
C. Correction of a defect
D. Analysis of system design
The correct answer is A. The tester now has a better understanding of why the system is built the way it is. The design of the system is not defective, as there is a deliberate reason for not having WiFi support, so C is not the right answer.
Tags: software reviews
Posted in software quality, software testing | No Comments »
Tuesday, August 2nd, 2011 by Rex Black
Reader Patricia Osorio had another good question about the answer to the following question from our Advanced Test Manager e-learning course. Here’s the question:
A given organization is using reviews for development work products like code, requirements and design specifications; test work products like test plans, quality risk analyses, and test design specifications; and, documentation and help screens. The review processes have been in place for two years and are delivering excellent financial, quality, and schedule benefits.
You are attending a management team meeting. A senior executive raises the need to update the objectives by which the individual contributors are measured on their yearly performance evaluations. He suggests using defect counts from review meetings. He circulates a draft plan. Under the plan, people will be rewarded based on the number of defects they find in reviews. Further, people will be penalized if items they have produced incur too many defects during reviews.
Which of the following is a psychological factor affecting review success and failure that is likely to cause such an initiative to undermine the current success of the reviews?
- Scrutinize the document and not the author.
- Focus all participants on delivering high-quality items.
- Try to find as many defects as possible.
- Assemble the right team of reviewers.
The correct answer is the second option. Instead of focusing on delivering high-quality items, the reviewers will focus on trying to claim as many defects found in reviews as their own, and will raise as defects trivialities, in order to try to maximize their rewards, while the authors will try to refute every defect report, even the most serious, in order to try to minimize their punishments. We have seen this play out in real life with clients, so you must be careful.
Tags: software reviews
Posted in ISTQB certification, software quality, software testing | No Comments »
Tuesday, August 2nd, 2011 by Rex Black
Long-time reader Patricia Osorio asks about some material from our Advanced Test Manager e-learning course (and also from Advanced Software Testing: Volume 2):
1. In the section 6.4.1 Defect Removal Effectiveness of Reviews, I founded the following paragraph, Could you please explain me how to get the numbers [of remaining defects]?
First, imagine that you started with 1, 000 defects. You follow worst practices in reviews, but at least you review all types of items. In this case, you would enter testing with about 166 defects. Now, imagine that you started with 1, 000 defects again. However, this time you follow best practices (and again you review all types of items). This time, you go into testing with 3 defects.
The question refers to the following table:
| |
Least
|
Average
|
Most
|
| Requirements review |
20%
|
30%
|
50%
|
| High-level design review |
30%
|
40%
|
60%
|
| Functional design review |
30%
|
45%
|
65%
|
| Detailed design review |
35%
|
55%
|
75%
|
| Code review |
35%
|
60%
|
85%
|
So, if you follow the “least” column, after the requirements review, there are 800 defects remaining (20% removed), after the high level design review there are 560 defects remaining (30% removed), and so forth, with 166 the final figure for defects remaining. If you follow the “most” column, after the requirements review, there are only 500 defects remaining (50% removed), after the high level design review there are 200 defects remaining (60% removed), and so forth, with 3 the final figure for defects remaining.
Now, admittedly the real situation is a bit more complex than this. Defects are not introduced only in the requirements, but indeed in every work product. However, the basic point remains: the more defects are removed earlier in the lifecycle, the easier the job of testing.
Tags: software reviews, software testing
Posted in ISTQB certification, best practices, software testing | 1 Comment »
|
Rex Black is President of RBCS (
