Evaluation matters

Posted By Laura Venning on 30 January 2017

Laura Venning, Evaluation and Impact Research Manager at The Reading Agency, blogs about the importance of working together to improve evaluation across the sector.

Evaluation matters. It’s not always easy; it can feel intimidating, it is often time consuming, but I promise you it is worth the effort. Evaluation is about reflecting on and improving your practice: mapping where you want to get to, finding out if you are getting there and indicating how you should change what you are doing if you’re not. It’s how we know whether our work is making a difference and helps us find out how to do things better. Evaluation can also help make the case for investment: if we know the outcomes that result from our work it is easier to explain why the activity should be funded.

A young girl in a black and white dress sits in a circular green pod in a library, reading
Photo © Dave Warren

There are lots of brilliant organisations that run different programmes and projects to encourage reading, but evaluation of this activity hasn’t always focused on measuring their impact. Often data is collected about the number of people taking part in an initiative, but not the difference that taking part has made to them. And, when impact evaluation is carried out, different tools are developed for each project, which can be a duplication of effort and also makes it hard to compare the results.

It’s really important to recognise that you can’t measure everything

This is why we at The Reading Agency launched a partnership project to develop a reading outcomes framework, with funding from the Peter Sowerby Foundation. We hoped to develop a shared understanding of the outcomes of reading for pleasure and empowerment, create a framework within which evaluation could take place, and bring together existing tools to measure impact that could be used by the different organisations that run activities with similar aims.  

You might be wondering ‘what exactly is a reading outcomes framework?’ Put simply, it is a diagram that maps out how we get from A (activities to encourage reading) to B (the benefits of reading for pleasure and empowerment). A range of types of outcomes are included in the framework, such as further reading engagement, improved health and wellbeing, and intellectual, personal and social benefits.

A book lies open on someone's lap, who is sitting on a bed making notes
Photo © Jazmin Quaynor

The intention isn’t that every organisation will measure the impact of each initiative on every outcome. When planning evaluation for any project, I think it’s really important to recognise that you can’t measure everything. But you do need to follow a logical process to decide on the key outcomes to focus on, and this framework will help to structure this decision making process.

We identified the outcomes to include in the framework by looking at the existing evidence, via a literature review about the impact of reading for pleasure and empowerment. When thinking about outcomes and evaluation, it’s really helpful to look at what’s already known, to ensure you aren’t making assumptions that are contrary to the evidence or duplicating work that’s already taken place. Taking the review as a starting point, we have refined the outcomes included through discussions with our steering group, and consultation workshops that we’ve been holding across the UK.

Evaluation is always going to require the investment of time and energy, but when we work together on it the results can be stronger

We’ve had many debates throughout the course of the project, for example defining exactly what we mean when we say ‘reading for pleasure’: When you feel sad or scared or angry after reading a book is this still ‘reading for pleasure’? When reading is hard can it still be fun? Is the enjoyment of reading fundamental to the activity? And we’ve come to some conclusions, including a definition of reading for pleasure and empowerment, which is included in the toolkit that we have created.

Taking account of lots of different viewpoints brought its own challenges, of course, but it was definitely worth it. As a result, we’ve created a reading outcomes framework that we can all be really proud of. And the collaboration that was at the heart of this project has had wider benefits for the organisations involved, promoting knowledge sharing and encouraging joined up working.

My experience working on the reading outcomes project has convinced me that this process is something that could be replicated across other parts of the arts and culture sector. Evaluation is always going to require the investment of time and energy, but when we work together on it the results can be stronger and the quality higher.

An older man with a beard sits reading, surrounded by stacks of books
Photo © Jilbert Ebrahimi

Find out more

The toolkit is ready for people working to encourage reading for pleasure and empowerment to start using it. It’s available here, and is free for anyone to download and use and we would love to hear what you think of it: email readingoutcomes@readingagency.org.uk. We are now piloting the toolkit in five sites across England, with funding from the Arts Council.

The key project partners for this work are: Arts Council England, Association of Senior Children's and Education Librarians, BookTrust, Chartered Institute of Library and Information Professionals, National Literacy Trust, Publishers Association, Scottish Library and Information Council and the Society of Chief Librarians.

Find out more about how the Arts Council supports and develops writers and literature organisations across England

Follow The Reading Agency and the Arts Council on Twitter