Getting Testing Involved

This content is syndicated from LeadingAgile by Dennis Stevens. To view the original post in full, click here.

Testing anti-pattern

There is a common anti-pattern I consistently run into involving testing. In this anti-pattern testing is about finding technical defects near the end of a project. Testers view their job as preventing a buggy product from being shipped to the customer. To accomplish this, testers define test cases from the requirements documents. Since requirements are going to change during the project, the testers need to wait until near the end of the project to produce their tests.

What’s wrong with this?

This model doesn’t work. The requirements documents don’t tell a perfect story in the first place. And the documents never stay in perfect alignment. Sometimes the tester interprets the requirement differently than the developer. Inevitably, development delivers something that doesn’t match what testers expect and they file a defect. To avoid defects, the developers have to anticipate everything that might be meant by the requirements. A result is that developers significantly overbuild or over-engineer the product.  The bigger and more “dynamic” the product is, the bigger the testing effort.

A result of the anti-pattern is that the cost of producing working, tested software is significantly increased. We are less predictable and we don’t really know where we are until very late in the project.  Because we test late we find out late in projects that we aren’t meeting the schedule, or scope or cost. We end up in the blame game trying to decide if defects are a requirements problem versus a development problem versus a test problem. There is vicious cycle that produces a culture of low trust and a lack of collaboration and coordination. This is very common – I see this low collaboration environment every day.

Define the acceptance criteria before you build

Test (n): A procedure for critical evaluation; a means of determining the presence, quality, or truth of something.

Testing isn’t about finding bugs. And it isn’t just about evaluating.  A critical activity in testing is defining the criteria for the evaluation – we will call these the acceptance criteria. As we pointed out above it isn’t effective when someone in a silo sets the criteria for evaluation after the product has been built. We want to define the acceptance criteria before we build the thing. This doesn’t incur more cost to the testers. It is no more initial work writing the test cases whether you do them before or after.

Define acceptance criteria as a whole team

Defining them before isn’t enough. Everyone needs to have a shared understanding of the outcomes. There is a lot of benefit in the discussion around defining acceptance criteria. When we agree on the tests upfront developers can build the smallest and simplest thing that will solve the problem – not overbuild trying to anticipate everything that requirements could possibly mean.  When we agree on how we will evaluate the product before we build it we will have fewer defects so there is less rework.

Progressively elaborate acceptance criteria

Defining the acceptance criteria before you build something isn’t big up front design.  We want to do whole-team just-in-time detailed design. In Scrum, this happens while grooming the backlog and in Sprint Planning. In our Enterprise Agile approach, we break the product into a series of releases in product road-mapping. We break releases into features during release planning. We break features into stories and we elaborate the stories further before Sprint Planning.

As we progressively elaborate the requirements – we progressively elaborate acceptance criteria. In fact, acceptance criteria are an effective way to express business rules at the story level. We also find that non-functional requirements, compliance requirements, and operational requirements can be expressed as acceptance criteria at the feature level so we can ensure we are finishing all aspects of the product as the project progresses.

A better testing strategy

We’re on the way to solving the testing problem. When the acceptance tests are elaborated jointly the testers, analysts and developers don’t have different understandings. When we elaborate them before we build developers can build the smallest and simplest thing that will solve the problem. When we elaborate acceptance criteria at the same time we elaborate requirements we don’t have as much change control required and we actually get better requirements to communicate. This strategy can have a significant impact on reducing the cost of testing – and the time to release of the product.

Leave a Reply

Your email address will not be published. Required fields are marked *

one × 1 =

There are 101 ways to do anything.
To find the best way, sometimes you need expert help

What People Say

“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”


“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”


“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”


“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”


“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”


“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”


“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”


“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”


“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”


“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”


“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”



To explore how we can help you, please get in touch