Estimation as Hypothesis

This content is syndicated from George Dinwiddie's blog by George Dinwiddie. To view the original post in full, click here.

Experimentation is a powerful learning tool. When I was young, I performed scientific experiments by mixing chemicals together to see what they would do. I learned that most random concoctions from my chemistry set would make a brown liquid that was often hard to clean out of a test tube. I learned that sometimes they would create very smelly brown liquids. These were not really experiments, however, and I didn’t really learn from them. Instead, these were activities and I collected anecdotes and experiences from them.

The scientific method rests on the performance of experiments to confirm or deny a proposed hypothesis. Unless you can propose a hypothesis in advance, you cannot design an experiment to test it. Until you test the hypothesis, you haven’t really learned anything.

“In general, we look for a new law by the following process: First we guess it; then we compute the consequences of the guess to see what would be implied if this law that we guessed is right; then we compare the result of the computation to nature, with experiment or experience, compare it directly with observation, to see if it works. If it disagrees with experiment, it is wrong. In that simple statement is the key to science. It does not make any difference how beautiful your guess is, it does not make any difference how smart you are, who made the guess, or what his name is — if it disagrees with experiment, it is wrong.” — Richard Feynman, The Character of Physical Law

When we estimate how long it will take, or how much it will cost, to implement a desired amount of software functionality, we create a hypothesis that we can test. Our hypothesis may not be of enduring and universal value as a hypothesis that predicting physical laws, but it may still be extremely valuable to us.

For example, suppose we have a number of features we’d like to get into our next software release. And suppose we have a date in mind for that release, and a team ready to work on it. We could then ask that team to estimate the features relative to each other, bucketing them into groups of similar sizes. We could also ask them to estimate how much bigger (or smaller) are the features in one size group than the features of another. If this team has previous experience working together, they might be able to guess how long one feature might take to implement. Otherwise, they might just take a guess at it.

I would expect these numbers to be simple, with only one or two significant digits. After all, we don’t have much data to base them on. Their precision should not pretend that we do.

If we were practicing a plan-driven serial software development cycle, we might treat these estimates as promises and try to manage the work to meet them. In such case, I would expect them to have padding for the unknown, and higher precision to hide the fact that they’re padded guesses.

Using an empirical software development approach, we’ll instead treat this projection as a first hypothesis. When we finish the first feature, we’ll have some better data on the rate at which we’re progressing, and can project into the future with a bit more confidence. Does this data confirm our hypothesis of when we’ll be done?

This experiment helps us make decisions. If completing the features by the target date looks unlikely, we’ll want to take drastic action. Perhaps we’ll eliminate some features, or make them all simpler, in order to trim scope and achieve some success. Perhaps we’ll decide to cancel the project altogether, cutting our losses with only a fraction of the budget spent.

If the target still looks feasible, we can continue the experiment. We’ll still have uncertainty about both the rate of progress and the size of the work, but we can reason about those uncertainties. Are our errors in sizing likely to be additive, or random? Is the current rate of progress sustainable? Is it depressed because of one-time startup work? Or is it optimistic because we’ve been cutting corners?

Poorly handled estimation is a means to fool ourselves, but handled with care, it gives us tools to experiment and learn.

Leave a Reply

Your email address will not be published. Required fields are marked *

thirteen − three =


There are 101 ways to do anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”

HANNAH JOYCE
GLASS'S INFORMATION SERVICES

“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”

PETER THATCHER, SENIOR ACCOUNT DIRECTOR
ThoughtWorks

“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”

ANDY JEFFRIES/TECHNICAL LEAD
IPC MEDIA

“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”

BRUCE WEIR/EGM
SUNCORP

“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”

LUKE SHARKEY /STRATEGY & IMPLEMENTATION LEADER
SUNCORP

“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”

BEATRIZ MONTOYA/CONSUMER MARKETING DIRECTOR
IPC MEDIA

“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”

PETER SILVA-JANKOWSKI
IPC MEDIA

“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”

GILES BENTLEY, DEVELOPMENT & OPERATIONS DIRECTOR
TIME INC

“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”

DAN PULHAM, DIGITAL DIRECTOR
TELSTRA

“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”

GINA MILLARD
GLASS'S INFORMATION SERVICES

“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”

JULIE PEEL
GLASS'S INFORMATION SERVICES

CONTACT US

To explore how we can help you, please get in touch