Improving how we evaluate, step by step

I’ve been part of Sport England’s evaluation team for many years and multiple strategies, but the dawn of Uniting the Movement marked a real step-change for how we evaluate and learn from our work.

Following an extensive consultation with partners and colleagues, our approach and action plan set out a shared vision for how evaluation and learning could support our long-term strategy.

This included ambitions to improve how we embed, conduct and use evaluation and learning and the actions that would help us achieve them.

Stepping back to reflect

A year and a half on from publishing this plan, I wanted to reflect on the steps we’ve taken whether big, small, forwards, backwards or indeed sideways.

In short, I can say we’ve done a lot (though not as much as we’d hoped) and often in different ways to what we expected – a good learning in itself!

Some of the highlights include:

• Evaluation and learning has a stronger internal profile, meaning greater desire from senior leaders to evaluate our major investments, so we’re investing more time and resources into evaluations of greater depth and breadth than ever before.
• Evaluation and learning is better integrated into Sport England’s internal performance reporting with evidence from our evaluations directly feeding into our performance framework. We also talk about evaluation and learning more and more honestly with senior external stakeholders including the government.
• We’ve experimented with new and developmental evaluation methods that fit the context and values of Uniting the Movement. This includes co-designing an impact and learning model with System Partners, pioneering the emerging configurational comparative analysis to evaluate our Place investments, and trialling ripple effects mapping in the evaluation team itself to evaluate our work.
• We’ve updated our webpages, including a new tab with details of our current evaluations and a report library that houses over 50 publications.
• We’ve improved how the team operates, including team accessibility training, changing how we run team meetings, a skills-development audit and updating our day-to-day processes.

Stepping up to the challenges

Despite all the positives, it’s in my nature and the team’s to also notice the things that haven’t gone so well too – we are evaluators after all!

Some of the difficulties and challenges we’ve faced include:

• Finding time for the action plan whilst managing our daily work and all the major evaluations. We care about the plan but too often it’s had to give way to higher priorities.
• We’ve sometimes struggled with the complexity and scale of the work. It’s an ambitious improvement plan containing many interconnected actions and objectives, set against the backdrop of a sector-wide strategy to reduce inequalities. The context is anything but simple and knowing how best to make progress hasn’t always been straight forward, obvious or easy.
• The tailored, context-driven nature of our work made it difficult to communicate our overall priorities for evaluation and learning in a clear and simple way. Our research and evaluation lead, Tim Fitches, reflected on this with a LinkedIn post last year.
• Some discussions with senior decision-makers have felt too narrowly focused on accountability, thus limiting opportunities for learning and improvement.
• We know there’s lots more we can do to share, interpret and act on what we’re learning from our evaluations, in a way that’s more visible, accessible and useful to the people and organisations who can use them.

Taking learning steps

These challenges, as well as contextual changes within Sport England, our partners and the sector, have shaped how and what we’ve delivered through the action plan.

We’ve learned that it’s ok to change course when we need to and to see changes as windows of opportunity.

Other things we’ve learned include:

• moving away from seeing the action plan as a list of standalone tasks to be ticked off. Instead the plan is seen as a connected, ongoing and long-term effort to live our principles by and realise our ambitions. In other words, we’re tying the action plan more closely to our wider work programme, delivering it within our major evaluations rather than separately
• using a more pragmatic and disciplined way of managing the action plan, like Microsoft Planner to organise our work, allocate tasks and log updates, plus regular structured team meetings to make decisions, reflect on progress and move things on together.

Next steps

We still firmly believe that the ambitions and areas of work in our approach and action plan are the right ones but the last 18 months have taught us that we need to change how we go about it.

We’ll continue to learn from the plan and adapt how we deliver it in response to changing circumstances, but always with our long-term ambitions in mind.

We hope this update sparks ideas and starts conversations, firstly because we think our experiences are likely to be echoed elsewhere and secondly because we know we can only achieve our ambitions and realise the true value of evaluation and learning for Uniting the Movement by working together.

We’ll get there. Step by step.

And if you’re interested in finding out more and working with us, then we would love to hear from you so make sure you get in touch.