Archive for the ‘ISTQB certification’ Category
Friday, August 10th, 2012 by Rex Black
I received some questions from a reader and webinar attendee, which I’ll answer below:
Hi Rex,
I am Jason, and few days ago was in the webinar for Reviews. As promised, I have collated the lists of questions to ask based on the Study Preparation Guide for CTAL-TM that I purchased from RBCS recently. Please do not hesitate to contact me should you require further information from me. Below are the questions and my comments:-
71. An organization follows a requirements-based test strategy for most of its projects. Which of the following is the best example of modifying the test approach for a project based on an understanding of risks?
A. Past performance issues lead to an increased effort on performance testing
B. Test estimation is based on the number of pages in the requirement specification.
C. Test execution is outsourced to a testing company based on a low-cost bid.
D. Unit test effort is limited to ensure early commencement of system test execution.
I did not really understand this question very well. How does performance issues in the past are related to risk?
Jason, the reason that A is the right answer is because we are using past defect information (in this case, performance defects) to assess the likelihood of particular types of problems.
84. You are managing a test effort that uses entirely reactive techniques, including a list of past bugs found in the field, a checklist of typical bugs for products using this technology, and exploratory testing based on tester experience. No written tests are developed prior to test execution, other than the list of bugs. Consider following statements
i Immune to the pesticide paradox
ii Repeatable for regression and confirmation testing
iii Useful in preventing bugs during system design
iv Cheap to maintain
v Makes no assumptions about skill tester skills
Which of the following is true about this particular testing strategy?
A. I and III are benefits of this strategy.
B. II and V are benefits of this strategy.
C. I and V are benefits of this strategy.
D. I and IV are benefits of this strategy.
Argument: I should not be part of the answer. “Exploratory testing based on tester experience” allows different test cases combination to be tested. If that is the case, why is that immune to the pesticide paradox? To my understanding, immune to the pesticide paradox refers to doesn’t have any effect to the pesticide paradox.
Since it is based on list of past bugs found in the field and checklist, I would consider II as tester can go through the regression based on these checklists.
The Foundation and Advanced syllabi are clear on the fact that detailed test cases–which do not exist here–are useful if repeating tests precisely for regression and confirmation testing purposes is needed. So, II cannot be a benefit.
The Foundation syllabus, in the section about general testing principles, says that the only way to overcome the pesticide paradox is to run different tests rather than repeating the exact same tests. Because all tests would be subtly (or even dramatically) different if repeated, especially if testers with different experience are used for subsequent executions.
125. You manage a test team for a bank. Your test team uses two primary test strategies, checklist-based and dynamic. As its checklist, your team has a list of main areas in which the test team, in-house users, or bank customers have reported defects on past releases. For the dynamic testing, it employs members of the test team with experience in the bank branches and back-office to do exploratory testing.
Based on this information alone, which of the following is an improvement that you would expect from a STEP assessment?
A. Get involved earlier in the lifecycle
B. Analyse requirements specifications
C. Run only scripted tests.
D. Improve the office environment.
The answer given in the guide is B. Analyse requirements specification. Could you please elaborate the reason to this answer?
Yes. The reason is that, as mentioned in the Advanced syllabus, STEP is based on an assumption that requirements analysis and requirements-based testing will occur.
Looking forward hearing from you soon. Thanks.
I hope this is helpful.
Tags: advanced software testing, ISTQB, ISTQB certification
Posted in ISTQB certification, software test skills, software testing | 3 Comments »
Thursday, December 29th, 2011 by Rex Black
Reader Art Salwin asks a good questions about an interesting distinction:
There seems to be quite a bit of contradiction in the literature between test conditions and test cases. Some places say the conditions have expected results (IEEE 829, for example). Other places say the cases have the expected results (some of the ITSQB study guides). There’s also disagreement about how high level a condition is. Some put it at pretty high level (test electronic payment, for example), others get down to if good credt card, accept; if bad credit card, reject.
Well, Art, there certainly is a lot of variation in the usage of certain terms in testing. A number of terms, such as “test case,” “test condition,” “test plan,” “test strategy,” etc., are in very common use, but almost everyone seems to mean something a little different by them. In addition, we have people using different terms to describe things that are the same; e.g., “test procedure” and “test script” and “test case.” So, we get a lot of miscommunication in our profession.
To clarify communication, one thing I suggest, when talking to someone about testing, is to make sure that you’re speaking the same language. If someone uses one of these common test terms, asking them what they mean by that term, and what specific contents he or she is referring to. Variation exists even within organizations, so a quick check to make sure you’re talk to each other, not past each other, is always a good idea.
Within an organization, it’s a good idea to adopt a single glossary for testing terms. The ISTQB glossary isn’t perfect, but it is being constantly perfected and it’s certainly the most comprehensive testing glossary out there. So, that’s probably a good place to start.
It’s also important to adopt templates for common work products, because a definition by itself will not suffice. For example, you and I could both agree on the ISTQB definition of the phrase “test strategy” and yet write different documents. It’s important to remember that templates should be useful outlines that help people remember important topics to include in work products; templates are not straightjackets or substitutes for thinking.
So, going back to your original question. If we take the ISTQB terminology and templates, then the test conditions are the higher level statements of what is to be tested. In risk based testing, the risk items identified during a quality risk analysis are the test conditions (for more on how that works, see the video series on risk based testing on the Digital Library). Examples of risk based test conditions would be “system responds too slowly to user input” and “system calculates incorrect report totals.” In requirements based testing, test conditions are identified by an analysis of the requirements. Examples of requirements based test conditions would be “check input field validation” and “check tax calculations.” As you can see, these are at a high level.
The test cases are then developed to cover the test conditions. One or more test cases should be associated with each test condition. In risk based testing, the number of test cases for each test condition is determined by the level of risk associated with the risk item. In requirements based testing, well, determining how many tests to have for a test condition is one of the problems with a purely requirements based testing strategy. If you use a blended risk based and requirements based testing strategy, you can associate levels of risk with requirements to determine how much to test.
The test cases themselves can be a various levels of detail. If you check Managing the Testing Process, 3e, you’ll find a discussion in there about the level of detail in test cases. If enough people are interested, I can post some of that information on the blog.
Posted in ISTQB certification, software test templates, software testing | 2 Comments »
Wednesday, December 21st, 2011 by Rex Black
E-learning Advanced Test Analyst student Donna Hungarter asks:
[An exam question] shows a state transition diagram, sets some criteria, and then asks how many tests are needed for this level of coverage. If we were looking for the fewest number of tests needed to cover all states and transitions, I believe the answer is 6 (not one of the answer options). If all 4 different credit cards need to be tested for each state and transition, I can get to 24, but I do not understand how the correct answer is 26. Can someone please explain? Thank you, Donna
Here’s the question:
You are testing a computerized gas pump that allows users to pay using credit cards. The pump is initially in the waiting-for-customer state. A transaction is initiated when a customer first inserts a credit card. Four types of cards are accepted: Visa, MasterCard, Discover, and American Express. The pump will reject any other type of card. If given an accepted credit card, the pump validates the card. If the card is valid, the pump is turned on and the customer is told to begin pumping gas. The pump remains on until the transaction ends. The transaction ends when one of the following events occurs:
• Pump handle is returned to pump
• Amount reaches transaction limit for card
• No gas is pumped within two minutes of validation of card
• The station attendant throws the emergency shut-off switch
Once the transaction ends, the pump processes the transaction, charging the credit card and producing a receipt. It then returns to the waiting-for-customer state.
Assume a test always starts and may only end in the waiting-for-customer state, a test must end after a transaction-ending event, and a test input consists of (initial state, event[condition], next state, event[condition], …, event[condition], initial state). Design tests covering every unique sequence of up to four states/three events. How many tests are needed for this level of coverage?
The state transition diagram is shown here:
 Gas Pump State Transition Diagram The correct answer is 26. The reason is because the question is asking for two-switch coverage. You have to create the two-switch table. You can short-cut the work by noticing that many of the two-switch elements can’t occur in a test, because they involve returning to the initial state as an intermediary step. You can see the answer in the image below:
 Switch Table and Tests You might need to expand the image to read it. This picture was taken in Kuala Lumpur, Malaysia, after a long 15 minute session to solve the problem. It’s a true brain-twister, and almost certainly harder than anything you’d see on a real exam.
Posted in ISTQB certification, software tester certification, software testing | No Comments »
Wednesday, November 16th, 2011 by Rex Black
Reader and RBCS licensee student Sandor sent me the following question:
Hello Rex,
In the training material of the Advanced Technical Test Analyst Course (V1.1) on the slide 268 I found a misleading example. There is a code fragment
…
if (n<0) {
…
n = -1;
} else {
…
for (i = 1; i <= n; i++)
{
…
}
…
}
...
There is a comment under this code:
“Statement coverage: We tested(n < 0) and (n > 0). To get decision coverage,we still need (n == 0).”
I think the n = 0 test case is not needed because with n > 0 the false branch of the if is fully covered decision point of view (the for cycle is evaluated to true and false). What do you think? Am I correct?
Thanks,
Sandor
So, I think that depends on how you interpret it. If you interpret decision coverage as narrowly meaning that each decision evaluates at least once true and at least once false, then you would be right.
However, let’s think about what we’re trying to test with decision coverage. We are not only testing for bugs in the decision code itself. We are also testing what happens when alternative paths are taken in the code. Certainly, doing the loop body code one or more times, then exiting to the next block of code, is a different path than not doing the loop body code at all and proceeding directly to the next block of code.
Therefore, my interpretation is that decision coverage (or branch coverage, if you prefer), applied to loops, must include not only testing that the decision can be made correctly both true and false, but also that the loop body is executed and is not executed. Not everyone agrees with that, so you’re likely to encounter other opinions in books and articles.
Tags: decision coverage, structural testing, white box testing
Posted in ISTQB certification, software tester certification, software testing | No Comments »
Wednesday, October 12th, 2011 by Rex Black
Long time reader Patricia Osorio wrote to ask about the sample exam questions at the end of Chapter 7 in Advanced Software Testing: Volume 2. She’d like me to explain the answers.
Here are the questions:
Assume you are a test manager working on a project to create a programmable thermostat for home use to control central heating, ventilation, and air conditioning (HVAC) systems. In addition to the normal HVAC control functions, the thermostat also has the ability to download data to a browser-based application that runs on PCs for further analysis. During quality risk analysis, you identify compatibility problems between the browser-based application and the different PC configurations that can host that application as a quality risk item with a high level of likelihood. Your test team is currently executing compatibility tests.
Consider the following excerpt from the failure description of a compatibility bug report:
1. Connect the thermostat to a Windows Vista PC.
2. Start the thermostat analysis application on the PC. Application starts normally and recognizes connected thermostat.
3. Attempt to download the data from the thermostat.
4. Data does not download.
5. Attempt to download the data three times. Data will not download.
Based on this information alone, which of the following is a problem that exists with this bug report?
A. Lack of structured testing
B. Inadequate classification information
C. Insufficient isolation
D. Poorly documented steps to reproduce
The answer is C. We would expect, given the bug that’s being reported, that the tester would try other operating sytems.
The questions continue:
Continue with the previous scenario. Your test team is still executing compatibility tests. Consider the following excerpt from the failure description of a compatibility bug report:
1. Install the thermostat analysis application on a Windows XP PC.
2. Attempt to start the thermostat analysis application.
3. Thermostat analysis application does not start.
4. Re-install the thermostat analysis application three times. Thermostat analysis application does not start after any re-installation.
5. This test passed on the previous test release.
Based on this information alone, which of the following is the most reasonable hypothesis about this bug?
A. The bug might be a regression.
B. The bug might be intermittent.
C. The application didn’t install on Windows XP PCs before.
D. The bug might be a duplicate.
The answer is A. The bug is very likely a regression, because the test worked on the previous version of the software installed in the test environment.
Tags: software bugs, software defects, software testing
Posted in ISTQB certification, software testing | No Comments »
Wednesday, October 5th, 2011 by Rex Black
I had a recent e-mail from Eleni Chee, a listener to our webinars. She wrote:
Hi,
Firstly, I just wanted to thank you for conducting awesome webinars, I’ve definitely learned a lot from it.
I noticed that you also provide training for people interested in pursuing the ISTQB certifications. I’m just wondering why ISTQB – why not QAI or ASQ? Is it because ISTQB is a better certification? Please advise. Thanks!
Regards,
Eleni Chee
First, Eleni, thanks for the kind words about out webinars. We definitely aim to provide the best testing webinars in the business, with a focus on good content, not advertising pitches. I’m glad to hear that we’re hitting that target with you. I hope you will pass on our webinar invitations to other friends and collegues who could also benefit.
Now, to answer the question about why we only follow the ISTQB software tester certification, yes, we do think it is better. I chose to get involved with the ISTQB program after careful study of the different options, including ASQ and QAI. We not only offer accredited training, but I also chose to get involved in the program itself, to help improve it. I’ve had the privilege of serving as ISTQB President, ASTQB President, and ISTQB Governance Officer, as well as being involved as an author for the Foundation syllabus, Advanced syllabus, and Expert syllabus.
You can learn more about the ISTQB certification and what makes it the best by listening to this webinar (http://www.rbcs-us.com/software-testing-resources/155) and/or reading this article (http://www.rbcs-us.com/images/documents/ISTQB-Certification-Why-You-Need-It-and-How-to-Get-It.pdf). If you want to learn more about the Advanced certifications specifically, check out this webinar (http://www.rbcs-us.com/software-testing-resources/96) or this article (http://www.rbcs-us.com/images/documents/The-ISTQB-Advanced-Syllabus.pdf).
Tags: ISTQB certification
Posted in ISTQB certification, best practices, software tester certification, software testing | 2 Comments »
Monday, August 15th, 2011 by Rex Black
Blog reader Angee Tong asks the following interesting question about equivalence partitioning:
Hi Rex,
I have a question on the sample test analyst exam question below. I can’t figure out how the answer is derived. Can you walk me through the solution for this exam question?
Thanks.
Exam Question:
You are testing a computerized gas pump that allows users to pay using credit cards.
Four types of cards are accepted: Visa, MasterCard, Discover, and American Express. The pump will reject any other type of card. If given an accepted credit card, the pump validates the card. If the card is valid, the pump is turned on and the customer is told to begin pumping gas. The pump remains on until the transaction ends. The transaction ends when one of the following events occurs:
1. Pump handle is returned to the pump.
2. Amount reaches transaction limit for card
3. No gas is pumped within 2 minutes of validation of card
4. Emergency shut-off switch is thrown by the station attendant.
Assume that a test is consists of a triple set of values (card type, card validity, transaction ending). Design the minimum number of tests needed to cover all the equivalence partitions for each card brand, including both valid and invalid cards of each accepted type along with covering each equivalence partition for transaction endings. How many tests do you need?
I came up with 17 tests but it is the wrong answer. My reasoning is I counted the amount reaches card limit or within card limit count as two other equivalence partitions as well. Also I counted the emergency shut-off option as an equivalence partition. Should these three items not be counted as equivalence partitions? Can you explain how the answer 9 is derived?
Can you provide other sample questions similar to this for additional practice so I can make sure I learned to apply the test technique correctly? Thank you.
| Test Case |
Card |
No Gas Is Pumped Within 2+ Minutes |
Amount Within Card Limit |
Gas Is Pumped Within 2 Minutes |
Pump Handled Returned to Pump |
Amount Reaches Card Limit |
Emergency Shut-Off |
| 1 |
Invalid Card |
|
|
|
|
|
|
| 2 |
Visa |
Y |
Y |
|
|
|
|
| 3 |
Master Card |
Y |
Y |
|
|
|
|
| 4 |
Discover |
Y |
Y |
|
|
|
|
| 5 |
Amex |
Y |
Y |
|
|
|
|
| 6 |
Visa |
|
|
Y |
Y |
|
|
| 7 |
MasterCard |
|
|
Y |
Y |
|
|
| 8 |
Discover |
|
|
Y |
Y |
|
|
| 9 |
Amex |
|
|
Y |
Y |
|
|
| 10 |
Visa |
|
|
|
|
Y |
|
| 11 |
MasterCard |
|
|
|
|
Y |
|
| 12 |
Discover |
|
|
|
|
Y |
|
| 13 |
Amex |
|
|
|
|
Y |
|
| 14 |
Visa |
|
|
|
|
|
Y |
| 15 |
MasterCard |
|
|
|
|
|
Y |
| 16 |
Discover |
|
|
|
|
|
Y |
| 17 |
Amex |
|
|
|
|
|
Y |
Angee, what you’ve done in your solution is a form of combinational testing. In other words, you are testing each combination of credit card with transaction ending. That is not required for equivalence partitioning. In addition, you’re not testing the invalid card numbers for each accepted card type.
The answer is nine tests. Here’s why. There are four equivalence partitions for transaction endings. Those are all valid partitions. There are nine equivalence partitions for cards: four valid card numbers, one for each type; four invalid card numbers, one for each type; and, one invalid card type (e.g., a JCB).
Angee, there are a number of other practice questions for equivalence partitioning provided in the book and e-learning course. Make sure to go back and do the Foundation sample questions on test design techniques (Chapter 4), too.
Tags: equivalence partitioning
Posted in ISTQB certification, software testing | No Comments »
Tuesday, August 2nd, 2011 by Rex Black
Reader Patricia Osorio asks two good questions about a little-known test technique:
This is a question/suggestion. In the chapter 4 [of the Advanced syllabus], Test Techniques, I found Classification tree method technique. I know this technique…but it does not appear in the Foundation Syllabus. It appears in Advanced Syllabus. Why is this technique not included in Foundation Syllabus? Why it is classified as the same level of pair wise and orthogonal arrays? I have understood this technique as join application of BVA + Equivalence Partitioning + Error guessing + Defect based.
The answer to the first question, Patricia, is that this technique is really too sophisticated and complex to expect entry-level testers to master. Remember, the Foundation syllabus is meant to serve as an entry-level certification, such that people with no testing knowledge can take the training and exams as preparation to enter a testing career (or simply to be competent if posted temporarily in a testing role).
The answer to the second question is that pairwise testing techniques, include orthogonal arrays, are about testing combinations. That’s what classification trees allow you to do: define sophisticated types of combinations that you want to test. However, if you see affinities with other test techniques, that’s good; it probably means you’re thinking of ways to combine those techniques together and use them for specific problems, which is exactly what you want.
Tags: software test techniques, software testing
Posted in ISTQB certification, software testing | No Comments »
Tuesday, August 2nd, 2011 by Rex Black
Reader Patricia Osorio had another good question about the answer to the following question from our Advanced Test Manager e-learning course. Here’s the question:
A given organization is using reviews for development work products like code, requirements and design specifications; test work products like test plans, quality risk analyses, and test design specifications; and, documentation and help screens. The review processes have been in place for two years and are delivering excellent financial, quality, and schedule benefits.
You are attending a management team meeting. A senior executive raises the need to update the objectives by which the individual contributors are measured on their yearly performance evaluations. He suggests using defect counts from review meetings. He circulates a draft plan. Under the plan, people will be rewarded based on the number of defects they find in reviews. Further, people will be penalized if items they have produced incur too many defects during reviews.
Which of the following is a psychological factor affecting review success and failure that is likely to cause such an initiative to undermine the current success of the reviews?
- Scrutinize the document and not the author.
- Focus all participants on delivering high-quality items.
- Try to find as many defects as possible.
- Assemble the right team of reviewers.
The correct answer is the second option. Instead of focusing on delivering high-quality items, the reviewers will focus on trying to claim as many defects found in reviews as their own, and will raise as defects trivialities, in order to try to maximize their rewards, while the authors will try to refute every defect report, even the most serious, in order to try to minimize their punishments. We have seen this play out in real life with clients, so you must be careful.
Tags: software reviews
Posted in ISTQB certification, software quality, software testing | No Comments »
Tuesday, August 2nd, 2011 by Rex Black
Long-time reader Patricia Osorio asks about some material from our Advanced Test Manager e-learning course (and also from Advanced Software Testing: Volume 2):
1. In the section 6.4.1 Defect Removal Effectiveness of Reviews, I founded the following paragraph, Could you please explain me how to get the numbers [of remaining defects]?
First, imagine that you started with 1, 000 defects. You follow worst practices in reviews, but at least you review all types of items. In this case, you would enter testing with about 166 defects. Now, imagine that you started with 1, 000 defects again. However, this time you follow best practices (and again you review all types of items). This time, you go into testing with 3 defects.
The question refers to the following table:
| |
Least
|
Average
|
Most
|
| Requirements review |
20%
|
30%
|
50%
|
| High-level design review |
30%
|
40%
|
60%
|
| Functional design review |
30%
|
45%
|
65%
|
| Detailed design review |
35%
|
55%
|
75%
|
| Code review |
35%
|
60%
|
85%
|
So, if you follow the “least” column, after the requirements review, there are 800 defects remaining (20% removed), after the high level design review there are 560 defects remaining (30% removed), and so forth, with 166 the final figure for defects remaining. If you follow the “most” column, after the requirements review, there are only 500 defects remaining (50% removed), after the high level design review there are 200 defects remaining (60% removed), and so forth, with 3 the final figure for defects remaining.
Now, admittedly the real situation is a bit more complex than this. Defects are not introduced only in the requirements, but indeed in every work product. However, the basic point remains: the more defects are removed earlier in the lifecycle, the easier the job of testing.
Tags: software reviews, software testing
Posted in ISTQB certification, best practices, software testing | 1 Comment »
|
Rex Black is President of RBCS (
