I recently received an e-mail from a software tester, which raises some interesting issues about testing in an Agile environment. I’ve interspersed my comments with the original e-mail, using “RB:” to indicate my text.
Hi Rex, I am a ISTQB Foundation Level certified software tester and your book on Foundation level helped me clear the exam. Thanks for that. I never knew you had a website and visited it recently, infact today only.
RB: You’re welcome. I’m glad you found the book and the website useful. I hope you like the blog, too.
There is somethign that I want to know and I thought you might help. I am into manual testing and currently into a project which follows Agile methodology.
RB: Many people are in this situation these days, though you seem to be suffering from a bad implementation of Agile.
The problem that I face is, we are given new features to write test cases and test them once or twice every week but some of them do not have any specification at all. And when we raise our concern that we need some requirement or specification to identify our test cases etc , the functionality is explained in one liners over chat windows and the excuse that we are following Scrum methodology and hence we do not have any written specification or requirement for this feature.
RB: That’s not following Agile best practices, but we have run into situations with a signficant number of clients where not all of the Agile best practices are in place. Ideally, there are user stories (lightweight requirements) which are documented and managed. Techniques and tools for doing so differ, but what you describe is not sufficient for testing, and probably not for development either.
This is a classic mistake of interpreting a situation as a Manichean choice (i.e., one hand good, other hand evil), rather than understanding that the question is one of degree. In this case, the Manichean choice is being made with respect to the Agile principle that says to “value working software over complete specifications.” Notice that this does not say, “Don’t bother to write any specifications whatsoever,” but plenty of companies following Agile have interpreted it that way.
Another concern is, we do not have any test reporting template. We are just given features to test over email and we test and report them over email as well. But I would like to maintain at least some bare minimum documentation so that I have at least some record of what I have done which should also help me in keeping everybody on the same page.
RB: Yes, this is another challenge that happens in testing, and not only in Agile. The failure to record what was tested (both in terms of tests themselves as well as traceability back to the test basis) is common. As you point out, this leads to test results that are completely subjective and undocumented. Not only is this inadequate for release decisions, it’s completely useless in terms of process assessment.
What do you think should be my approach in such a scenario? Please advice.
RB: You are facing a lot of the challenges I first discussed over three years ago in a presentation on Agile testing challenges, which later became a webinar (http://www.rbcs-us.com/software-testing-resources/92) and also an article (http://www.rbcs-us.com/images/documents/agile-testing-challenges-english.pdf). I’d recommend you take a listen/look at those.
Thanks and Regards Neha Srivastava
RB: You’re welcome. Good luck. You might need to find a place where you can practice more effective and enlightened testing, because you’re unlikely to learn much other than bad habits from working in an environment like this one.
Tags: bad Agile implementation, effective software testing, software testing
Rex Black is President of RBCS (
I can appreciate the challenge. Depending on seniority and leverage in the organization, there may be some room to change the dynamics.
But if such leverage does not exist, such an assignment can be an educational (if somewhat frustrating) exercise in learning how to derive requirements and define a testing plan and results reporting plans for a highly chaotic environment. IMO, such plans should probably be well-understood (at least by the practitioner), though perhaps not formalized (because they will need to change as quickly as the product and requirements).
In my experience, I’m finding that learning to make sense from limited knowledge and bring some semblance of order out of chaos is a critical factor in testing success. For good or bad, high chaos is common. And learning to manage that well and be effective despite it can help build some of the credibility that earns the right to be heard when you want to propose changes.
As a followup, it seems to me that these are some of the challenges that are common in testing careers, and for which most certifications don’t prepare someone. (I’ll freely admit that I’m only familiar in a limited capacity with ISTQB certs.) I think the fact that these are *not* addressed in certification exams and that they *are* addressed in webinars, written materials, etc, tells us something about the nature of professional maturity in testing. Academics (such as certification prep) can prepare for some of the technical challenges and the mechanics of management. But there is a whole other category of learning that is essential, dealing with how one influences change as a tester, how one brings order out of chaos, etc. Much of this is better “coached” or “modeled” than “taught”.
Thoughts/comments? Please correct me if my cursory assessment of ISTQB certifications is inaccurate.
I think that the ISTQB program does address the different lifecycles, and I know in our training courses we try to talk about testing strengths and challenges associated with these lifecycles. That said, the ISTQB programs do strive to be broad (covering as many common practices as possible), which necessarily means sacrificing some amount of depth in the discussion of these practices.
That said, I do agree that, in general, special cases (such as the specific problem the original commentator brought up) are best addressed in webinars, consulting, and other more-focused approaches to helping people solve problems.
Hi Rex,
I have gone through your webinar on Agile Testing Challenges and found it really useful.
All the tips shared in there could prove to be really helpful in improving the Agile Processes in my company.
I have worked in better places as well and since I know the differences, I try to keep my head above water and learn and mark areas of improvement.
Toough I am not in a position to make drastic changes, but I try my bit. And ur webinar and articles helped me lot.
Thanks again.
Neha
I find the statement, “the functionality is explained in one liners over chat windows” very telling. It sounds like this is a case where the team is divided with programmers on one side and testers on the other. My guess is that they are divided by great distances and multiple time zones.
This division is the real problem, not the lack of documentation.
If the programmers and testers acted as an integrated team, the testers would be part of the requirements and design discussions. They wouldn’t have to be satisfied with little crumbs of information dropped over chat. They would have access to the full banquet.
Hi Elisabeth–
I don’t know if that’s the case here in this situation, but geographical distribution in Agile teams is certainly something that I’ve seen happen with other clients, with similar problems sometimes. Given that geographical distribution of teams–including “developers here, testers there”–will happen in some cases, do you have any tips for how to bridge those gaps, Elisabeth? What have you seen work?
Regards,
Rex
The situation explained by Elisabeth is indeed the case with me. And I believe this is a very common scenario.
What are your suggestions in such a scenario to reduce the problems?
I work on a heavily distributed team, and problems in information flow (which can already be inadequate between dev and test teams) are certainly exacerbated by geography. But even within my team focusing on test (sort of), geography causes information flow challenges; it’s just a fundamental part of geography and time zones.
I’ve not found a silver bullet, but there are things that help:
o Build relationships as far as possible between dev and test teams.
Relationships make a big difference in whether info is shared
grudgingly or eagerly.
o Make some specific times to meet in real-time. It gets harder when
you’re split across continents, and sometimes one has to accept meetings
at inconvenient times. But the increase in efficiency and satisfaction
can more than make up for a little inconvenience.
o Be patient with remote teams. Lack of communication can lead to
making false assumptions of incompetence. Recognize that they have to
be patient, as well, and actively fight against making assumptions.
Instead, make time to meet.
o Try to capture discussions in email or docs, when possible, rather than chat
sessions. When information *is* shared in chat sessions, follow up with
an email to yourself (and anyone else who might need the info) to capture
the info.
In the end, geographical diversity can help a lot of things. I find a lot of
benefit from having team members in another time zone. But it takes a lot
of patience, effort and graciousness on everyone’s part to make it work. And
that brings it back to an issue of people, not just process.
John
Geographically distributed teams are common, and Agile certainly can work with distributed teams. However, it requires extra effort. It’s particularly difficult if the geographical division reinforces the traditional dev/test silos.
In order for Agile to work, testing has to be involved all the way through as part of the development effort. Early involvement to prevent defects and tight feedback loops to discover them while a story is still in development are essential to succeeding at frequent delivery with high quality.
So, given that, I’ll echo John’s comments, and add:
- The whole team has to participate in the standup meeting (the purpose of standup is coordinating activities, not reporting status; sending a proxy doesn’t work). That means it has to be scheduled for a period when the timezones overlap and everyone is in the office.
- I watched Marlena Compton deftly manage to participate in a real time conversation while simultaneously including people on IRC chat. She later told me that at her company when the people who are face to face have an important discussion that results in a conclusion, they often recap that discussion on IRC since that’s how most people in the company communicate. This repeating of key information on multiple channels is very effective.
- I was on an XP project where we had a remote developer. He paired using Skype for communication and a remote desktop for sharing the keyboard. Because we had Skype running on multiple computers through out the office, he could tune in to discussions in multiple places at will. We didn’t always have to dial him in; he could dial himself in. Ultimately, it felt like he was just another co-worker in the office, only with the curious disability of not having a corporeal body. In short, telecommunications technology has come a long way and I advocate making the most of it.
Also, a final note on Agile: just because an organization says they’re doing Agile doesn’t mean they are. Agile is an umbrella term that encompasses a number of methods including Scrum, Kanban, XP, and Lean. The common thread that unites these methods is the results: Agile teams deliver production-quality business value frequently (at least monthly), at a sustainable pace, while adapting to the changing needs of the business. If the team’s practices don’t support that, they’re not Agile no matter what they claim and no matter what their process.
Good thoughts, Elisabeth and John. Elisabeth, I particularly like you comment about “The common thread that unites these methods is the results.” It’s easy enough to lose track of the objectives in any project, regardless of lifecycle, and the methodology should serve the objectives, rather than vice versa.