Why Do Products Fail? – Forgetting that Users Learn

This content is syndicated from Tyner Blain by Scott Sehlhorst. To view the original post in full, click here.

Next up in the series on the root causes of product failure – products that fail because you have ignored the user’s level of experience.  The first time someone uses your product, they don’t know anything about it.  Did you design your interfaces for new users?  After they’ve used it for a while, they get pretty good at using it.  How much do you think they like being forced to take baby steps through a guided wizard now?

Why Do Products Fail?

Your product launched a year ago.  People raved about how easy it is to learn to use your product.  Someone posted a video of a toddler using it, it kicked off a meme, you got a nice bump in downloads, and you were thrilled.  Now, people are complaining about how impossible it is to actually get anything done with your product.  And your main competitor is making hay with their message of cost-effectiveness and efficiency.  Sales have dried up to a trickle.  What the heck?!  You loosen your collar and walk in to explain to your investors why they will be lucky to get half their investment back – much less the ten-bagger your confidently forecast for them eleven months ago.

It sure is good you’re performing this thought experiment before you even build the product, instead of running the gauntlet a year after it launched.

There are many reasons a product can fail.  One way to think about your responsibilities as a product manager is to prevent all of them.

[larger version]

In this series, we are exploring as many of them as we can.  In your exploration of the reasons products fail, you started by making sure you’re solving the right problems.

[larger image]

As a firm believer in being user-centric, and making sure you’re delivering value to your users, you’ve doggedly pursued exactly what that means.

[larger image]

You’re confident that

What you failed to do was understand that your users change over time.

Kathy Sierra’s Canyon of Pain

I wish Kathy Sierra was still writing online.  I first came across her canyon of pain model in 2007.


On the other extreme is Apple’s iMovie. It gives you almost no control, but the payoff is high right out of the shrinkwrap. It exceeds my expectations of pain-to-payoff. But pretty quickly, anyone who gets into iMovie–and is bitten by the movie-making bug–starts wanting things that iMovie doesn’t let you control. So… Apple says, “not to worry — we have Final Cut Express HD for just $299″. The problem is, the learning curve jump from iMovie to Final Cut Express is DRASTIC. There needs to be something in the middle, to smooth that transition.

Kathy Sierra, How much control should our users have?

While she uses the above example to describe a market segmentation opportunity, it describes not just the difference between users, but the changes a user goes through over time.  Taking an aerial view of Kathy’s Canyon it would look like this:

Most users are competent users.  It makes sense – you start out as a new user, until you develop competence, and possibly develop expertise (although few become experts).   How many users will use your product for long enough- and invest themselves enough in learning to use your product to truly develop expertise?

Conceptually, you need to build a bridge across that canyon, for your product to meet the needs of your users as they develop confidence (and for the few who become experts).

Learning Curves

This is an artful dance – designing for the competent users – and your personas are likely to represent competent users, since competent users will be focusing on using your product to solve their problems, not trying to solve their problems with using your product.  With your focus on making sure you help your users solve their problems, you may overlook the problem that you introduce – learning how to use your product.  So, you better make sure you understand the nature of how people develop competence and expertise.

Malcolm Gladwell, in Outliers, builds on Dr. Ericsson’s analysis that it takes 10,000 hours of doing something to become an expert at it (not everyone agrees).

Defining Competence

The first step to measuring competency is to define the model.  I am proposing a definition in this article that I suspect will yield insights (to help us manage our products). I was unable to find any quantified definitions of competence, when researching it as part of a client engagement.  If you have, or know of a model, please share it in the discussion below this article.

  • A competent user is someone who learns to perform a task in half the time it initially took them.
  • An expert user is someone who can complete a task in 10% of the initial time.

This definition is guided by an expectation that Alan Cooper’s premise about perpetually intermediate users is true.  Being a novice user is a very transient state, and becoming an expert is very infrequent.  The goal of the definition is to be able to segment your users and make well-informed design decisions.

Modeling User Competency

Building on these definitions (and the hypothesis that Cooper is correct), you can model the level of difficulty of task-learning that best represents your product (see the article) to get insight into how rapidly your users will evolve their competency over time.  It can impact your modeling of ROI-realization, but more (?) importantly, it gives you insight into understanding how quickly your users will tire of the hold-my-hand-wizards you coded to guide them through your product.  As an example, their (measurable) improvement may look like the following:

[larger image]

Of course people are rarely chained to their desks and using your product continuously.  Even call-center employees get 5 to 8 minute breaks every hour.  So you should develop some insight into how quickly ( in hours / days / weeks) your users develop competence.

The following graph shows how an 80% learning curve overlays a calendar for tasks that happen daily, weekly, and hourly.

[larger image]

The graph shows that for a weekly frequency, after 16 weeks, the task time has reduced from 300 seconds to 100 seconds. With a daily frequency, the task time is even lower – about 60 seconds. This graph shows nothing other than converting the academic learning curve graph into one that incorporates calendar time and frequency of occurrence.

Software Usability and Learning Curves

Part of the catch-22 of developing products is that by trying to be all things to people in all-levels of competence, you may try and just add features.  Over time, this can really add up.  So, you have to figure out the right amount of capability to build into your product.  You can play a couple tricks, whipping out your handy Kano analysis skills again, and prioritizing capabilities and features with their characteristics in mind.

Consider the more is better requirements. Think of them in two categories – user interaction improvements, and application performance improvements.

User interaction improvements remove complexity, and make software easier to use. This results in more user happiness from a given feature, and also allows us to implement more features at a given level of happiness (appeasing salespeople).


Application performance improvements don’t create as dramatic of a shift (they don’t make the application easier to use). They do, make it more enjoyable for a given feature set – shifting the curve up.



Failing to take into account the learning journey that your target users go through is just a more nuanced version of the elastic user problem.  The easiest way I’ve found – so far – is to treat “learn how to use your product to achieve their (other) goals” as a user goal, and create a distinct persona that represents your target user while they are a novice.

By analogy, one persona for the caterpillar, one for the butterfly.

If you know of a better way to deal with this as a product manager, please chime in below – this solution is a bit more clunky than I would prefer to use.

Post to Twitter Post to Facebook

Leave a Reply

Your email address will not be published. Required fields are marked *

6 + 19 =

There are 101 ways to do anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”


“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”


“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”


“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”


“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”


“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”


“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”


“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”


“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”


“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”


“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”



To explore how we can help you, please get in touch