I recently received an e-mail from a software tester, which raises some interesting issues about testing in an Agile environment.  I’ve interspersed my comments with the original e-mail, using “RB:” to indicate my text.

Hi Rex,   I am a ISTQB Foundation Level certified software tester and your book on Foundation level helped me clear the exam. Thanks for that. I never knew you had a website and visited it recently, infact today only.

RB:  You’re welcome. I’m glad you found the book and the website useful.  I hope you like the blog, too.

There is somethign that I want to know and I thought you might help.   I am into manual testing and currently into a project which follows Agile methodology.  

RB:  Many people are in this situation these days, though you seem to be suffering from a bad implementation of Agile.

The problem that I face is, we are given new features to write test cases and test them once or twice every week but some of them do not have any specification at all. And when we raise our concern that we need some requirement or specification to identify our test cases etc , the functionality is explained  in one liners over chat windows and the excuse that we are following Scrum methodology and hence we do not have any written specification or requirement for this feature.

RB:  That’s not following Agile best practices, but we have run into situations with a signficant number of clients where not all of the Agile best practices are in place.  Ideally, there are user stories (lightweight requirements) which are documented and managed.  Techniques and tools for doing so differ, but what you describe is not sufficient for testing, and probably not for development either. 

This is a classic mistake of interpreting a situation as a Manichean choice (i.e., one hand good, other hand evil), rather than understanding that the question is one of degree.  In this case, the Manichean choice is being made with respect to the Agile principle that says to “value working software over complete specifications.”  Notice that this does not say, “Don’t bother to write any specifications whatsoever,” but plenty of companies following Agile have interpreted it that way.

Another concern is, we do not have any  test reporting template. We are just given features to test over email and we test and report them over email as well. But I would like to maintain at least some bare minimum documentation so that I have at least some record of what I have done which should also help me in keeping everybody on the same page.  

RB:  Yes, this is another challenge that happens in testing, and not only in Agile.  The failure to record what was tested (both in terms of tests themselves as well as traceability back to the test basis) is common.  As you point out, this leads to test results that are completely subjective and undocumented.  Not only is this inadequate for release decisions, it’s completely useless in terms of process assessment.

What do you think should be my approach in such a scenario? Please advice.

RB:  You are facing a lot of the challenges I first discussed over three years ago in a presentation on Agile testing challenges, which later became a webinar (http://www.rbcs-us.com/software-testing-resources/92) and also an article (http://www.rbcs-us.com/images/documents/agile-testing-challenges-english.pdf).  I’d recommend you take a listen/look at those.

Thanks and Regards Neha Srivastava

RB:  You’re welcome.  Good luck.  You might need to find a place where you can practice more effective and enlightened testing, because you’re unlikely to learn much other than bad habits from working in an environment like this one.