Going beyond ‘what success looks like’

#BASummit14: I was lucky enough to be asked to take part in this years BA Summit by Penny Pullan of Making Projects Work Ltd.

To hear my talk on this topic you check out my SoundCloud.

Over the past couple of years I have seen a growing trend in BAs and product teams striving to define what ‘success looks like’. It’s comes from a desire within the business to understand how we’ll know when we’ve achieved what we set out to do and what the world will look like at the finish line.

I think it’s worth recognising that there are probably a collection of terms out there that describe similar things here. You may call the, Critical Success Factors in your organisation and have associated KPIs.

If you’re working in a Scrum team you may refer to theses as the Definition of Done, a series of criteria or success factors that must be met in order for something to be considered as completed, as done.

The closest I could find in the BABOK about measuring success was in chapter 6.3 Specify and Model Requirements [1]:

The activity of analysing expressed stakeholder desires and/or the current state of the organization using a combination of textual statements, matrices, diagrams and formal models

I’m a real advocate of this approach and have found it resonates well with the product team I sit in, as together we defining a testable state in which to measure if we have achieved our intended goal. This is similar to the team’s approach in writing software, where they will write tests that fail, prior to writing a line of code. They then write software that passes the tests that were defined. Once all the tests pass the project is finished.

However, too often I think BA’s try and define success far too late in a projects life cycle and as a result, miss out on being able to show real business value early in the process.

Lean Statup

Many of you may have heard of Eric Ries and Lean Startup. You may have discredited it as applying to your organisation, as you’re not a startup, but I believe that everyone can use Lean Startup principles in their work, especially business analysts.

Eric discusses in his book, The Lean Startup, a new approach to testing hypothesis to help innovation. I believe we can use this same approach when identifying the impact of problems we uncover in our projects, whether it is a bug, an inefficient process or a missed deadline. And using the same principles we can continual measure for the improvements made as we get to work on tackling those issues.

Measuring failure, the “as is” state

So before we can measure what success looks like I encourage people to re-frame the problem. I ask them what failure looks like, often to their surprise. What I mean is, if we did nothing to resolve this issue what would the cost or impact top the business be. This may cause them to go and collect some information in order to answer this, but chances are they have a good idea and can articulate it.

This is a great place to start and will have many other benefits that can feed your product vision, and even help with ROI calculations further down the line.

Once you understand the problem or opportunity, you can start to consider solutions for achieving your goals, but how will we know that we’ve actually been successful?
If you work for a company that sells products online it may be easy for you to measure success. If you sell more items or increase your profit on what you sell, you’ve clearly been successful and perhaps these are some of the most obvious ways of measuring success and may not require models or equations to prove it.

However, if your project doesn’t have an obvious revenue stream to measure, you may need to be a little more creative with what you measure – and this I think, is where Business Analyst can really take a lead.

Have you ever considered the impact your project may have on customer satisfaction? Do you know what your current customer satisfaction score is and would you know how to measure it? Do you know where you rank amongst competitors, especially the ones that you are chasing for market share? I’m willing to bet someone in your organisation already knows the answer to all of those questions!

Do you know how long a manual process takes to run in your organisation? If you could identify improvements then you could calculate an efficiency saving to the business in terms of employee cost or an increase in productivity per employee.

Now measurement isn’t just something that we do at the end of projects. I often hear of the Definition of Done or the definition of what success looks like as being the finish line of a race that once crossed, means the project is closed. While this is true, I sort of dislike the analogy as it suggests it’s the last thing that happens in a project. I believe measuring success is a continuous process. if the definition of what success looks like is our finish line we measure at every step along the race course asking the question, have we crossed the line? Have we improved revenue, have we improved customer satisfaction? For teams that work on incremental delivery strategies, this data is crucial in checking that the team are going in the right direction and if they’re not this data will help support the need for changing direction or pivoting. This in itself is ensuring project success by getting early visibility of failure and risks.

Change, measure, change, measure…

So I’ve explained how I think measuring should take place at every step of the journey and how “what success looks like” is the finish line, the goal we’re aiming for. But in reality the improvements and gains that we see from the success of our projects don’t end once we cross that finish line. If you’ve managed to see an increase of 2% in sales as a result of your project, that’s for life. The business will continue to see that additional value month on month and as a BA I think it’s important that we shout about success and recognise the value that we as BA’s and the teams we support add to the organisation we live in. Everyone likes to have evidence they can show their boss that the project they’ve delivered was successful… And if it over achieved, how pleased will you be that you were tracking the data from day 1!

Conclusion

So to conclude I want to just recap on what I’ve said and recommend some further reading:

  1. Measuring success is linked right back to your initial requirements and stakeholder desires. You’re helping your team prove that they’ve been met with real data
  2. Measuring success is a continuous process, measuring it at the end is too late… you could be miles off and have not known
  3. Measuring success continuously reduces risk and allows for innovation and agility
  4. Evidence of success is the best reward of all, and will really help you as a BA

How to Measure Anything: Finding the Value of Intangibles in Business by Douglas Hubbard

For more on creative measuring check out Douglas Hubbard’s book.

[1] 2009. “IIBA BABOK V2.” [online]. iiba.org/babok