Posts Tagged ‘software test management’
Wednesday, July 13th, 2011 by Rex Black
Last week, as some of you will know, I gave a webinar on the psychological and political aspects of software test management. You can find the recorded webinar here (http://www.rbcs-us.com/software-testing-resources/163) and the PDF version of the slides here (http://www.rbcs-us.com/images/documents/July-6-2011-Psychopolitics-of-Testing.pdf).
Patricia Ensworth wrote me to comment on the webinar:
Just wanted to let you know I thought your webinar was superb. As someone who has experienced many of the pressures and dilemmas you described, I found your analysis insightful and your advice on target. Well done!
With all due respect, there is one point you made about which I’d like to offer an alternative perspective. I completely agree with you that when test managers are labeled Quality Assurance Managers and put in charge of enforcing standard practices in a vague quest for product quality it’s a one-way ticket to nowhere (except maybe martyrdom). However, I have occasionally seen situations where to strengthen the organizational position of the testing group and to leverage the testers’ holistic perspective senior IT management has given the test manager other kinds of Quality Assurance responsiblities. For example, the so-called QA manager might be aligned with Compliance/Security initiatives mandated by regulators, or with Business Analysis projects to re-engineer processes or services. With strong enough support from matrixed senior managers, it can sometimes be a workable arrangement.
In any event, thanks for a thought-provoking, useful session.
I agree with Patricia’s point. I have indeed seen that work successfully. Thanks for mentioning it, Patricia.
I’d be interested in hearing from other readers and listeners to the webinar. What is your experience with psychology and politics in software test management?
Tags: software test management, software test politics, software test psychology
Posted in software test management, software testing | 2 Comments »
Thursday, December 23rd, 2010 by Rex Black
As readers of this blog will know, I’ve spent a lot of time in this blog (and in articles, books, and consulting work) on the topic of risk based testing. I recently spent some time in Japan, working with various teams in Sony to implement better risk based testing. Part of that time was spent working with Atsushi Nagata, who is helping teams in Sony put risk based testing into action. Together, he and I have written an article describing some ground-breaking work that they’ve done on risk based test results reporting. That article was just published in ST&QA magazine. You can read it (and comment on it) by clicking here.
Tags: risk based testing, software test management, software testing
Posted in risk based testing, software test management, software test metrics, software testing | No Comments »
Friday, August 20th, 2010 by Rex Black
I took a few moments today to record another video blog entry, which you can find at Regional Software Testing Immaturity: Fact or Fallacy?
Here’s the synopsis: I’m in Los Angeles, on my way to the STANZ conference in Australia and New Zealand. That and other recent international trips have gotten me thinking about something that I often hear at such international conferences, which are comments along the lines of, “You know, software testing as a profession and a practice is really immature in region X,” where region X might or might not be where I’m at. Based on my experience with clients around the world, though, the gap isn’t as big as people often think it is. Is software testing actually significantly less mature in some regions than others? What has your experience been? I’d be interested in opinions, case studies, and stories from you, especially the many international readers of this blog but also people in North America.
Tags: software test management, software testing, testing immaturity
Posted in best practices, software test management, software test metrics, software test outsourcing, software testing | 5 Comments »
Friday, May 7th, 2010 by Rex Black
As I’ve mentioned before in this blog (and elsewhere), we work with many clients to help them implement risk based software and system testing. Two key steps in this process are the identification and assessment of risks to the quality of the system. In my 15 years of experience doing risk based software testing, I’ve found that the only reliable way to do this is by including the appropriate business and technical stakeholders.
When I explain this to people getting started, I sometimes hear the objection, “Oh, our project team members are all very busy, so I can’t imagine they’ll want to spend all the time required to do this.” Fortunately, this is an easily resolved concern.
First of all, to answer the “why would they want to participate?” implication of that objection, I’d refer you to my previous blog post on this topic here. Next, let’s look at the “how much time are we talking about” implication.
For most stakeholders, risk based testing involves only a little bit of their time to collect their thoughts. Risk identification via one-on-one or small team interviews requires about 90-120 minutes each, with risk assessment interviews either being separate follow up discussions of about the same length or even sometimes included in the risk identification interview. There’s typically a subsequent meeting to review, finalize, and approve the risk assessment.
It’s true that, by using project team brainstorming sessions, the workload on the stakeholders is higher than just 2-3 hours total. Risk identification and assessment via brainstorming sessions requires a single, typically one-day meeting. Most of our clients choose the “sequence of interviews” approach, because of the difficulty of scheduling all-day meetings.
Either way, the interview or session participants need to think about three questions during these steps in the process:
- What are the potential quality problems with the system (i.e., what are the quality risks)?
- How likely is each potential problem (i.e., how often do we find such problems during testing or in production)?
- How bad is each potential problem (i.e., what is the business and customer pain associated with such problems)?
By including the right selection of technical and business stakeholders, and thinking realistically (i.e., with neither excessive pessimism or optimism) about these three questions, the stakeholder team can produce a realistic and practical quality risk assessment.
If you’re interested in more information on risk based testing, you might want to take a look at the videos and other resources available here.
Tags: risk based testing, software test management, software testing
Posted in risk based testing, software test management, software testing | No Comments »
Wednesday, April 28th, 2010 by Rex Black
It’s important for project teams to select the right mix of test strategies for their projects. In our training, outsourcing and consulting work, we typically see one or more of the following strategies applied:
- Analytical strategies, such as risk-based testing and requirements-based testing;
- Model-based strategies, such as performance testing based on statistical usage profiles;
- Methodical strategies, such as checklists of important functional areas or typical bugs;
- Process- or standard-compliant strategies, such as IEEE 829 documentation and agile testing approaches;
- Dynamic or heuristic strategies, such as the use of software attacks or exploratory testing;
- Consultative strategies, such as asking key project stakeholders about the critical quality characteristics;
- Regression testing strategies, such as automated testing at the unit or graphical user interface levels.
It’s important to remember that strategies may be combined; test managers should employ all of the strategies that they can effectively and efficiently employ on a given project. Test managers should carefully select and tailor the strategies they use for a project.
Tags: software test management, software test strategies, software testing
Posted in software quality, software test management, software testing | No Comments »
Thursday, April 22nd, 2010 by Rex Black
Here are four simple rules for good software testing metrics:
- Define a useful, pertinent, and concise set of quality and test metrics for a project. Ask yourself, about each metric, “So what? Why should I care about this metric?” If you can’t answer that question, it’s not a useful metric.
- Avoid too large a set of metrics. For one thing, large collections of charts and tables create a lot of ongoing work for the test team or manager, even with automated support. For another thing, such situations lead to a “more data, less information” situation, as the volume of (sometimes apparently inconsistent) metrics becomes confusing to participants.
- Ensure uniform, agreed interpretations of these metrics. Before you start using the metrics, educate everyone who will see them about how to evaluate them, in order to minimize disputes and divergent opinions about measures of outcomes, analyses, and trends.
- Define metrics in terms of objectives and goals for a process or task, for components or systems, and for individuals or teams. Instead of starting with the metric and looking for a use for it, start with clearly defined objectives, define effectiveness, efficiency, and elegance metrics for those objectives, and set goals for those metrics based on reasonable expectations.
While simple to state, these rules are important. My associates and I find that, almost every time there is a test results reporting problem in an organization, one or more of these rules is being violated. Follow these rules for better software test metrics, and thus more effective software test management.
Tags: software test management, software testing
Posted in software test management, software test metrics, software testing | No Comments »
Wednesday, April 21st, 2010 by Rex Black
One of the great frustrations for testers, test managers, and other project team managers is the elusive test execution completion date. It always seems to take longer than you’d think to get done with test execution, especially on larger projects.
So, how do you predict when will you be done executing the tests? Part of the answer is when you’ll have run all the planned tests once. This involves knowing three things:
- Total estimated test time (the sum of the estimated effort for all planned tests).
- Total person-hours of tester time available per week.
- The percentage of time per day spent executing tests by each tester (as opposed to being involved in other activities like meetings, updating test cases, etc.).
This figure sets a minimum time required to finish test execution.
However, test execution can last longer than this, because the other part of the answer is when you’ll have found the important bugs and confirmed those bugs to be fixed. This involves using historical data (or extremely good guesses) to determine four things:
- The total number of bugs you’ll find.
- The bug find rate at various stages of test execution.
- The bug fix rate at various stages of test execution.
- The average bug closure period (i.e., the time from initial discovery to final resolution).
Obviously, solid historical data from similar past projects really helps with this kind of estimation. For our clients who have such data, and formal defect removal models, they can get quite accurate, often predicting the end date of test execution to within plus or minus 10% even on test execution efforts that last over six months.
For a simple spreadsheet that can serve as a starting point for predicting bug find-fix duration, you can take a look at this one, from the RBCS Advanced Library.
Tags: software test estimation, software test management, software testing
Posted in software test management, software testing | No Comments »
Friday, April 16th, 2010 by Rex Black
As some of you might know, I’m a big proponent of risk based testing. (For example, see the podcasts and videos on the RBCS Digital Library here.) In fact, a major RBCS client–I can’t mention their name but the odds are good that you own one or more of their products–just told us that an entire division of their enormous company is adopting risk based testing, based on their understanding of the technique from our Advanced Test Manager course.
When test professionals first learn about risk based testing, one important question that often comes up is, “How do I convince skeptical testing stakeholders (outside of the test team) that risk based testing of our software is smart?” You can give them a whole lecture in response to this question, but long answers tend to produce a severe case of MEGO (”my eyes glazed over”) in non-test people.
In business, people talk about the “elevator pitch.” If you haven’t heard this phrase, here’s what it means: You have a powerful executive in an elevator with you. She’s getting off in about 10 floors. You have just a few seconds to convey to this powerful person some important piece of information. Start talking.
So, if you find yourself in an elevator, a conference room, or an office with an influential testing stakeholder, and you want to convince them to support your efforts to implement risk based testing, here’s the elevator pitch:
- Risk based testing runs tests in risk order, which gives the highest likelihood of discovering the most important defects early (“find the scary stuff first”).
- Risk based testing allocates test effort based on risk, which is the most efficient way to minimize the residual quality risk upon release (“pick the right tests out of the infinite cloud of possible tests”).
- Risk based testing measures test results based on risk, which allows the organization to know the residual level of quality risk during test execution, and to make smart release decisions (“release when risk of delay balances risk of dissatisfaction”)
- Risk based testing allows, if the schedule requires, the dropping tests in reverse risk order, which reduces the test execution period with the least possible increase in quality risk (“give up tests you worry about the least”).
All of these benefits allow the test team to operate more efficiently and in a targeted fashion, especially in time-constrained and/or resource-constrained situations.
Tags: effective software testing, efficient software testing, risk based testing, software test management, software testing
Posted in risk based testing, software test management | No Comments »
Monday, April 12th, 2010 by Rex Black
Hi all–
After years of hesitation, RBCS is finally initiating a blog on software testing. We’ve hesitated in large part because I’ve always joked–and only half in jest–that the word “blog” seems to rhyme with “blab” given most blog content.
However, as the cliche goes, better to light a candle than to curse the darkness. In this blog, I’m going to focus on sharing immediately useful ideas about software testing, rather than merely opinions, bloviations, or jeremiads. I’m also going to talk about what you tell me you want to hear about, so feel free to send me requests.
In the next couple days, I’ll start with a series of blog postings on some key management concepts related to software testing. I’ll be happy to see comments on those, and we can use the blog postings as the start of a discussion of each of these concepts.
Regards,
Rex Black
President, RBCS, Inc.
Tags: effective software testing, efficient software testing, software test management, software testing
Posted in software testing | No Comments »
|
Rex Black is President of RBCS (
