software testing services
 Software testing training, consulting, and outsourcing from the experts: Rex Black Consulting Services (RBCS)
CALL US TODAY
(866) 438-4830
ISTQB certification testingISTQB certification testing ISTQB certification testing
PMI

Archive for the ‘software test templates’ Category

Testing Terminological Confusion

Reader Art Salwin asks a good questions about an interesting distinction:

There seems to be quite a bit of contradiction in the literature between test conditions and test cases. Some places say the conditions have expected results (IEEE 829, for example). Other places say the cases have the expected results (some of the ITSQB study guides). There’s also disagreement about how high level a condition is. Some put it at pretty high level (test electronic payment, for example), others get down to if good credt card, accept; if bad credit card, reject.

Well, Art, there certainly is a lot of variation in the usage of certain terms in testing.  A number of terms, such as “test case,” “test condition,” “test plan,” “test strategy,” etc., are in very common use, but almost everyone seems to mean something a little different by them.  In addition, we have people using different terms to describe things that are the same; e.g., “test procedure” and “test script” and “test case.”  So, we get a lot of miscommunication in our profession.

To clarify communication, one thing I suggest, when talking to someone about testing, is to make sure that you’re speaking the same language.  If someone uses one of these common test terms, asking them what they mean by that term, and what specific contents he or she is referring to. Variation exists even within organizations, so a quick check to make sure you’re talk to each other, not past each other, is always a good idea.

Within an organization, it’s a good idea to adopt a single glossary for testing terms.  The ISTQB glossary isn’t perfect, but it is being constantly perfected and it’s certainly the most comprehensive testing glossary out there.  So, that’s probably a good place to start. 

It’s also important to adopt templates for common work products, because a definition by itself will not suffice.  For example, you and I could both agree on the ISTQB definition of the phrase “test strategy” and yet write different documents.  It’s important to remember that templates should be useful outlines that help people remember important topics to include in work products; templates are not straightjackets or substitutes for thinking.

So, going back to your original question.  If we take the ISTQB terminology and templates, then the test conditions are the higher level statements of what is to be tested.  In risk based testing, the risk items identified during a quality risk analysis are the test conditions (for more on how that works, see the video series on risk based testing on the Digital Library).  Examples of risk based test conditions would be “system responds too slowly to user input” and “system calculates incorrect report totals.”  In requirements based testing, test conditions are identified by an analysis of the requirements.  Examples of requirements based test conditions would be “check input field validation” and “check tax calculations.”  As you can see, these are at a high level.

The test cases are then developed to cover the test conditions.  One or more test cases should be associated with each test condition.  In risk based testing, the number of test cases for each test condition is determined by the level of risk associated with the risk item.  In requirements based testing, well, determining how many tests to have for a test condition is one of the problems with a purely requirements based testing strategy.  If you use a blended risk based and requirements based testing strategy, you can associate levels of risk with requirements to determine how much to test.

The test cases themselves can be a various levels of detail.  If you check Managing the Testing Process, 3e, you’ll find a discussion in there about the level of detail in test cases.  If enough people are interested, I can post some of that information on the blog.

Document Software Test Cases or Procedures

I received the following question from a reader of this blog, Kathleen Marrs.  She wrote:

Rex,

I have a colleague who swears up and down about a standard test case and process document style guide that is supposedly endorsed by you and the ISTQB and ASTQB. The style that he has introduced makes extensive use of quotation marks to denote controls, fields, and form names.

E.g. Click on the “Orders” form, then click on the “Current Orders” tab, enter “Bob Smith” in the “Customer Name” field and click on the “Find” button.

To me this is not grammatically correct as quotation marks should be used for actual quotes, user entered text, single character entries or familiar words that are used in an unfamiliar context. Have you seen this writing style and do you endorse its use or has my colleague been misled?

I am a proponent of the standard that is used by MIT and Microsoft and others which looks like:

Go to Orders -> Current Orders tab -> Customer Name and enter “Bob Smith” and click OK

Your input on this matter would be greatly appreciated.

In my book, Managing the Testing Process, I discuss various styles and templates for documenting test cases or procedures.  The ISTQB, in the Foundation and current Advanced syllabus, discusses the use of the IEEE 829 templates.  However, I don’t recall addressing the stylistic point raised above in any of these documents.  To me, it’s really a matter of choice.  The test organization should standardize on a single approach and everyone should follow it.  Otherwise, you get into the “Tower of Babel” problem.

What I would comment on is that the example given above is what is called a concrete test case, in the sense that all of the inputs and outputs are explicitly specified.  This is certainly necessary for automated tests.  For manual tests, you can consider what degree of detail must be captured in the test cases.  In some situations, using logical test cases–where the type of input and output are described generally and the tester is allowed to use discretion, white-box and black-box test design techniques, and exploratory testing to select the specific inputs and to evaluate the results–is often more lightweight, effective, and maintainable.  This issue of the degree of specificity in test cases is something covered in Managing the Testing Process.

Free Tool for Calculating Software Testing ROI

I recently gave a workshop at the STANZ conference, first in Wellington and then in Sydney.  In this workshop, I mentioned that connecting software testing to business value is a key test management challenge of the 2010 decade.  (Of course, it’s really been a challenge for the entire time there has been software testing, but it’s a challenge we’ve yet to resolve.)  Everyone in b0th audiences agreed, and a number of people offered examples of how this challenge was affecting them.

Earlier this week, I gave a webinar on how to calculate the return on the software testing investment.   You can listen to a recording of that webinar if you missed it.

In that webinar, I walked through a case study of calculating software testing ROI.  This case study was described in an article originally published in Software Test and Performance magazine, and you can now find the article here on the RBCS website.

After the webinar, a bunch of people sent e-mails saying, “Hey, could you please post the spreadsheet that you walked through during the webinar?”  Here at RBCS, we like to say yes to our friends, clients, and supporters, so we did.  You can find the free software testing ROI spreadsheet on our Advanced Library now, under the name Case Study Info Appliance Test ROI.xls.

Before you use the spreadsheet, I suggest you read the article I mentioned above.  The article explains how the spreadsheet works and explains the case study numbers included in the spreadsheet by way of example.

Useful Software Testing Templates

Many of our clients and course attendees ask about templates.  We have many templates posted on our Basic Library and our Advanced Library.  A few worth particular mention are:

The last one isn’t actually a template, but it’s something many people find interesting. 

If there’s some other template you need, search the Basic Library and Advanced Library.  If you still don’t find it, post a comment here on the blog letting the readers and me know what you’re looking for.  Maybe someone can help.

Remember, though, a template is not an excuse to turn your brain off.  Be sure to use templates thoughtfully.



 
`