Lisa Crispin is an Agile testing practitioner and coach as well as the author of the book Agile Testing. In this article she explains the nuts and bolts of Agile testing to Kristoffer Nordström of Softhouse.
Lisa Crispin has a favorite definition of Agile which she often paraphrases when people wonder what the Agile testing buzz is all about. It’s from Elisabeth Hendrickson, Quality Tree Software, who wrote something to the effect that ”[Agile testing means] delivering production-ready business value frequently (at least monthly) at a sustainable pace.”
The key is that we can’t work at a ’sustainable pace’ if we don’t master core practices such as continuous integration, specification by example, test-driven development, refactoring, and all the other techniques that allow us to keep technical debt at a manageable level,” Lisa says. ”The big difference with Agile is that we recognize that testing is not a ‘phase’ (again, I am paraphrasing Elisabeth). It is an integral part of software development, along with coding. Therefore, the whole development team, including programmers, testers, BAs, UX designers, DBAs, system administrators, everyone who helps deliver the software, takes responsibility for quality and for making sure all testing activities are complete.”
As Lisa Crispin sees it, the fundamental difference between traditional testing and agile testing is that in traditional phased and gated projects, testing is – as she puts it – ”almost an afterthought.” Various specialists gather requirements, produce a design, write a test plan, write the code, and then – with what small amount of time is left – a ”QA department” tests.
Shortening the feedback loop
The result is that the feedback loop is very long – weeks or months. When bugs are found in the “test phase”, they’re entered into a defect tracking system, perhaps a review board prioritizes them, and they may or may not get fixed. ”Agile teams focus on shortening that feedback loop,”, Lisa says. ”We start by getting examples of desired and undesired system behavior from stakeholders. We turn these into automated tests, ’living documentation’, which in turn lets us know what code to write and when we are done. We do all the same types of testing as in traditional projects, all four ‘quadrants’ in the Agile Testing Matrix, including exploratory testing and all the ‘ilities’.” ”One difference is that these testing activities are a team effort, with constant collaboration between testers, programmers, customers and everyone else on the development team,” Lisa continues. ”Another difference is that we start out with the goal of zero defects, and as soon as we find a problem, we fix it and make sure it can’t happen again.”
Changing role
The way the role of the tester changes when they become an Agile tester is often a huge mindset shift for traditional testers.
”I myself was guilty of a ’quality police’ mentality back in my waterfall days, even though I collaborated with both programmers and product folks throughout the project life cycle. I was the one who decided when the software was ready to release. In hindsight, that was crazy!”
When Lisa started her first XP team, it was a shock for her to realize that it was her job to help the customer define external quality – not judge it herself. She has also observed how hard it is for traditional testers to get away from the traditional view, which is ’Don’t test that code until it has met all the exit criteria, or you’ll have to test it more than once, and that’s wasteful’. ”Working in small increments means we may test the same code over and over,” she says, ”but the value of short feedback loops offsets that cost, and we can automate a lot of that testing as well.” Lisa points out that testers aren’t used to working directly either with the development team or with the customer team. They need a lot of time and support to learn how to do this.
Good testers a help for customers
Lisa has observed that many people have started to think that testers aren’t needed anymore, due to the fact that we now have Agile developers doing automated unit testing and even functional testing getting continuous feedback.
”When I was a tester on a traditional team that did not automate unit tests, I spent 80 % of my time finding unit-level bugs. I never had time to do the important exploratory testing that would identify serious issues or gaps in features. My current team has automated 100 % of our regression tests. We know within minutes if a check-in breaks a unit test, and within 45 minutes whether any of the higher-level functional and GUI tests have found a regression failure.”
Lisa Crispin praises the luxury of time to be able to make her own unique contributions to the product. In her view, good testers are good at helping customers articulate examples of the behavior they want, turning those into acceptance tests, and keeping an eye on the ’big picture’ as the team delivers in small increments and short iterations. ”Exploratory testing is a skill that few programmers I know have mastered,” Lisa says. ”Collaborating with programmers to get regression tests automated frees our time to make these essential contributions.” So, in summary: what are the benefits from adopting agile testing practices? ”When the whole team takes responsibility for quality, and adheres to the mantra ’no story is done until it’s tested’, they will keep technical debt to a manageable level, deliver the features the customer actually wants, and keep defects out of production. The whole team reaps the intrinsic rewards of continually improving and doing their best work.”
