Estimating Non-Functional Requirements

This content is syndicated from Mike Cohn's Blog - Succeeding With Agile® by Mike Cohn. To view the original post in full, click here.

A few weeks back I promised someone I would blog about the unique challenges of estimating non-functional requirements.

First, let’s remember that a non-functional requirement is a requirement that is more about the state of being of the system than about one specific thing the system does. Non-functional requirements often have to do with performance, correctness, maintainability, interoperability, portability, and so on. They are often called the “-ilities” of a system because so many end in “ility.”

(By the way, in case you’re wondering, non-functional requirements can be written as user stories.)

The challenge with estimating non-functional requirements is that there are really two costs. First is the cost of initial compliance. Second is the cost of ongoing compliance.

To see these two costs at work, let’s consider an example. Suppose we have a performance requirement and that the team is working on a new product. In the first sprint the team may be thinking about performance but they aren’t going to do any performance testing. There’s no code yet. There’s definitely code being developed over the next few sprints and say by sprint five the team decides there’s enough code that they want to start doing some performance testing during that sprint.

Remember our first cost is the cost of initial compliance. In this case, that is the amount of work the team will spend on performance testing in sprint five. That’s not really much harder to estimate than most other product backlog items. The team here thinks about it and they put some number of story points or ideal days on it as their estimate.

To continue our example suppose they do the performance testing and any necessary tuning it reveals during sprint five. Well, in sprint six the team is adding some new features and here is where the second cost comes in–the cost of ongoing compliance. Once the team accepts a non-functional requirement into the project (as our team did here in sprint five) they need to remain in compliance with that non-functional requirement for the remainder of the project. I think of this cost as a tax. Doing performance testing (or staying in compliance with any non-functional requirement) creates some amount of overhead on the team (the tax). This overhead or tax must be paid regularly.

In some cases, the team and product owner will decide the tax must be paid every sprint. For this team, that would mean that every time they add a feature, they do performance testing of that feature and, likely, the whole system. In other cases the team and product owner may agree to pay the tax every few sprints. After all, a team might reason, it’s unlikely we affected the performance characteristics with the user stories we added this sprint and, besides, we aren’t really shipping after this sprint. The first of these cases, in which the team pays the tax every sprint can be thought of like as a sales tax or VAT. The second, in which the team pays every few sprints, is more like an estimated quarterly tax like independent contractors in the US pay.

So, back to the issue of how do we estimate this type of work? Well, estimate both parts separately. Estimate the cost of initial compliance just like any other user story or product backlog item. The team and product owner will need to incorporate an estimate of when they’ll do this. Adding performance testing after five sprints is different than adding it after 20 sprints. For the tax portion, the team and product owner need to agree on when they will do the work: every sprint or after every n sprints. The team can then estimate how much work will be involved over the planned number of sprints and allocate that amount of work to each. For example, suppose the team and product owner agree that they will do performance testing in every fourth two-week sprint. The team then estimates that it will take six points of work every fourth sprint. That’s about 1.5 points per sprint. If a team has a velocity of 30, 1.5 can be thought of as about a 5% tax. It’s likely that the team could be quite wrong on this estimate the first time. Fortunately, the team can easily track how much effort goes into performance testing over a number of sprints and use that to revise their estimate of the ongoing cost.

Leave a Reply

Your email address will not be published. Required fields are marked *

three × five =


There are 101 ways to do anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”

JULIE PEEL
GLASS'S INFORMATION SERVICES

“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”

LUKE SHARKEY /STRATEGY & IMPLEMENTATION LEADER
SUNCORP

“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”

PETER THATCHER, SENIOR ACCOUNT DIRECTOR
ThoughtWorks

“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”

BRUCE WEIR/EGM
SUNCORP

“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”

PETER SILVA-JANKOWSKI
IPC MEDIA

“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”

HANNAH JOYCE
GLASS'S INFORMATION SERVICES

“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”

ANDY JEFFRIES/TECHNICAL LEAD
IPC MEDIA

“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”

GINA MILLARD
GLASS'S INFORMATION SERVICES

“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”

GILES BENTLEY, DEVELOPMENT & OPERATIONS DIRECTOR
TIME INC

“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”

BEATRIZ MONTOYA/CONSUMER MARKETING DIRECTOR
IPC MEDIA

“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”

DAN PULHAM, DIGITAL DIRECTOR
TELSTRA

CONTACT US

To explore how we can help you, please get in touch