Tag Archive for: evaluation

Learning about data; Working with qualitative data

/in /by

From time to time, we share a director’s blog post, where we share some learning about how to use your data (information) better. This is a post all about how to use qualitative data, that’s the information that is made up of words rather than numbers.

The chances are that your organisation already has a bank of this data but it may not be well organised and easily accessible. Below is a list of some of the many ways you might have collected qualitative data, possibly without realising it.

How credible is your qualitative data?

There is a distinction between primary and secondary data and it’s important to understand what you have collected as the credibility differs between the two.  Primary data is more credible and robust.  It is the data that people told you or have written themselves.  Secondary data is that which is overheard in discussion or something that someone has heard and told you about.

To check the credibility of your secondary data, you can apply the Rule of 3. If you’ve heard it from 3 different places or if it’s come from 3 different people independently then it can be considered as credible and representative of a thought or opinion.

How to make the qualitative data more usable? 

We use word or phrase frequency analysis to evaluate our qualitative data to look for patterns of frequency of words to identify common themes.  This can be done online using free tools, like this one,  and they just count the words for you.  We often use the top 10 words or phrases.  You can also use the ratio of positive to negative words that are used or how far down the feedback the first negative word appears.  Using these methods helps to quantify data and make it more digestible and can be used in marketing or to track changes over time.

Mind maps can be useful to illustrate and develop on the themes identified. You can just draw these freehand to have a look at what the main themes are.

Word clouds provide a visual representation at a glance of the qualitative data and this is also a resource that can be sourced freely online.  The most common words appear the biggest in the cloud, making this data easier for the reader to understand visually.

Top tip – try to be objective, it can be hard to hear negative comments but it is how we improve and know what to fix, and they’re often in the minority compared to positive comments.

How to use and share qualitative data?

This data can be used in many different ways:

We also use qualitative data to create case studies which illustrate people’s journeys and direct engagement.  Case studies can be shared at board level and to show case your project’s work for marketing purposes.  Case studies appeal to a wide audience and are particularly useful in attracting funders as voices are recorded and reflected in these studies.

In our experience as evaluators, a case study from a project that we evaluated got shared with funders (Green Recovery) who sent it on to The National Lottery Heritage Fund who then sent it on to the Department of Work and Pensions.  Case studies are impactful in that they record real voices and can attract publicity and raise awareness of change.

 

If all of this sounds like too much work and you don’t have time, get in touch. We are happy to have a chat about your individual requirements and to see how we could help.

New Project: Urban Forest Accelerator

/in , /by

The Evaluator is delighted to be starting a new project with National Trust, the Urban Forest Accelerator. We are excited to be working on measuring concepts of urban tree cover, tree equity, environmental justice and developing a new approach to measuring connections. Just as joined up canopies can be more that the sum of their parts, we believe our work relationships can also mirror this and aim to develop a new way to track this over time.

Along the way, we are lucky enough to get to work with inspirational people who want to transform and green our urban areas. We get to have discussions around who should decide what is best for urban trees, to look at tree cover metric developmental work and to use words like arboriculture career pathways and knowledge. Plus, we love trees!

The Urban Forest Accelerator is part of the Future Parks Accelerator and aims to address concepts including; partnership, urban forest capacity, sustainable finance, tree equity, community engagement and learning and scaling.

 

 

Recent Client Feedback

/in , /by

At The Evaluator, we are committed to learning and improving our work over time. To make sure we achieve this we do evaluate ourselves and ask our clients to give us feedback. Here are some recent results.

 

And let’s end on a detailed client testimonial, from when The Evaluator worked in partnership with Simon Lees from Countryside Training Partnership to create an evaluation framework.

“Kirsty from The Evaluator and Simon from Countryside Training Partnership have challenged and encouraged the Climate FORTH project team and Board to think about monitoring and evaluation in a holistic way, right from this project development stage. The Monitoring and Evaluation Framework created will support our application to NLHF for delivery phase funding and show how we wish to go beyond monitoring outputs to considering the wider impact of National Lottery and other investment through our project. Communication was excellent throughout this contract and an extension was kindly accommodated when our timelines were delayed. The final report is clear, concise and well considered – spot on – we hope that we get an opportunity to enact it! The Evaluator & Countryside Training Partnership have proposed workable and proportional monitoring and evaluation methods to follow in a project delivery phase, identified suitable baseline data and provided friendly advice as we move through our project development phase. I would be pleased to recommend both parties to other organisations and partnerships looking for a personalised professional approach to this type of work.” Kate Fuller, Climate FORTH Project Manager, Inner Forth Futures.

New Client: National Trust

/in /by

Blue border and text saying new client national trust

The Evaluator is delighted to be working with a new client: National Trust.

The National Trust is Europe’s largest conservation charity, who look after nature, beauty and history for everyone to enjoy. They do it with the help of millions of members, volunteers, staff and donors. Without that, they couldn’t care for the miles of coastline, woodlands, countryside and the hundreds of historic buildings, gardens and precious collections they protect.

The Evaluator is delighted to be working with them to focus in on a people engagement evaluation, for their Green Recovery Challenge Funded Project ‘Uplands’. We will be getting to know the teams, carrying out analysis including an economic assessment of the supply chain for the project and sharing our learning through interactive workshops, which we are calling ‘Action Learning Sessions’.

 

Evaluation: To Have and To Heal

/in /by

When you hear Ancient Egypt for Modern Times, what do you think of? We thought of pyramids, and mummies and actually the whole of Egyptian Culture. We have loved evaluating this project around connecting modern people to ancient themes. This project aims to take the whole of that culture and make it relevant now. Of particular poignancy is how people have coped with tough times, and bereavement throughout the generations, which is timely given the last few years.

The Evaluator is delighted to be working the Manchester Museum team to understand the impact of their project, which includes an inter-generational project working across schools and care homes and an online project, of which more is below.

The official project description is:

To Have and to Heal is a unique new arts and wellbeing programme, supporting Covid recovery and resilience, which explores Manchester Museum’s World Class Egyptology collection and popular fascination with ancient Egypt. Sessions provide a safe space to share responses to and personal reflections of experiences of the pandemic.

Participating groups are invited to explore a chosen theme, through which discussion and reflections are shared in relation to the objects and the personal experiences they may connect with. Themes range from Magical Healing and Touch, to Loss and The Meaning of Art.

The interactive online programme includes real time engagement and Q & A with the Museum’s Curator of Egypt and Sudan, showcases high quality digital images of rarely seen objects from the collection, together with a behind the scenes virtual visit to the Museum’s storerooms.

​The online programme uses Manchester Museum’s collections as a catalyst for providing inspiration, and meaningful support to groups and organisations that have been heavily affected by the coronavirus pandemic, and is open to:

*​Third sector organisations offering vital support to people at risk of social isolation, with health conditions, or caring responsibilities.

*Schools and home-schoolers seeking inspirational new resources to support ‘catching-up’ on disrupted education

*Care homes and living independent living facilities

You can find out more about the project by clicking here. There are a number of short films which can be freely viewed to explore some of the project themes and objects from Ancient Egypt on the same link.

This project is funded by The Esmée Fairbairn Collections Fund – delivered by the Museums Association.

Manchester Museum reopens soon at 10am on the 18th February and is opening with a free exhibition, Golden Mummies. You can find out more here. 

Thanks to all our 2022 clients!

/in , /by

Images of all our client logos - there are a lot including national trust, wildlife trusts, in-situ, county councils and more

Huge thanks to all our 2022 clients, many of whom we will still be working with in 2023 and beyond.
The office is closing for Christmas on Thursday 22nd December and will reopen refreshed and renewed on Tuesday 3rd January.

Happy holidays to all.

Kirsty and all at The Evaluator

New Evaluation: Green Loop

/in , /by

The Evaluator is delighted to be working with a new client, Fylde Borough Council on their arts and environment project; Green Loop. Based around Lytham St Anne’s, Green Loop as a name was inspired by the idea of creatively re-using the waste products which wash up on the shore.

There are three parts to this project including a new Eco-Market taking place at Fairhaven Lake, running with eco-friendly makers and crafts people and curated by Hopeful & Glorious. It has been really nice to work with Heather from Hopeful & Glorious as we worked together many years ago when both of us were working in Arts Development in Lancashire. Hopeful & Glorious curate many high quality events, and you can see more about them on their website here. 

Environmental artist Nerissa Cargill Thompson, is both creating an artist commission and working with a group of volunteers to take part in a 12 week programme to learn how to become an environmental maker with a creative business. That’s a journey of discovery for sure! We will enjoy writing about that as creative business is one of our particular interests.

Nerissa describes her work as:

Exploring change over time, not just eroding or decaying but new layers of growth, giving juxtapositions of structure and colour. Recent mixed media sculptures combine embellished textiles and cement cast in plastic packaging to highlight the legacy of disposables. Naturally inspired textures emphasise the way our waste becomes subsumed into the natural world around us.

Images of recent works can be seen on her website here. 

This project is run by Fylde Borough Council whose website is here, and the project is funded by Arts Council England.

New Client: VocalEyes

/in , /by

The Evaluator is delighted to be working with a new client, VocalEyes. We are going to be working with VocalEyes to evaluate their Heritage Access 2022 project.

An ambitious project working with 50 volunteers, Heritage Access 2022, will support and train volunteer access researchers in a large-scale digital volunteering project that will benefit over 3,000 heritage sites in the UK and their visitors.

VocalEyes (great name!) helps bring arts and culture to life for blind and visually impaired people. They support arts and heritage venues through audio description services, training, consultancy and advice. You can see more about VocalEyes on their website here. 

New Client; Surrey County Council

/in , /by

The Evaluator is delighted to be working with a new client, Surrey County Council. We’re working on an innovative placemaking project being delivered between a partnership lead jointly by Reigate & Banstead Borough Council and Surrey County Council to put the community at the heart of placemaking.

We are following and measuring the process and creating a capacity building report, a toolkit for others wanting to do placemaking in their locality and an animated case study to put the story of this work back into the mouths of the people taking part.

 

New Client; Attitude is Everything

/in , /by

We are delighted to be working with a new client, Attitude is Everything, evaluating Beyond the Music. Beyond the Music is an ambitious programme, funded by Reaching Communities from the National Lottery Heritage Fund to encourage more Deaf and disabled people to work in the music industry.

Here is an extract from the Attitude is Everything website. You can see the website directly by clicking here. 

“Attitude is Everything is pleased to announce Beyond the Music, a new three-year programme that aims to boost employment opportunities for Deaf and disabled people in the commercial music sector.

Findings from Arts Council England show just 4% of staff at National Portfolio Organisations, and just 1.8% of staff at music industry organisations, consider themselves to be disabled. This is a significant disparity from the UK’s general population, where 19% of working adults are considered disabled under the Equality Act.

Such a disconnect may be the result of barriers and discrimination, but our own research suggests that many Deaf and disabled people in the industry may also lack confidence to make their impairment known. Last year we revealed that that 70% of disabled musicians hid details of their impairment for fear of losing opportunities, and that two-thirds had compromised their health to perform in inaccessible conditions.

Funded by the National Lottery Community Fund, Beyond The Music will look to explore these issues and employment gaps, while taking a two-pronged approach to identify solutions: supporting Deaf and disabled people to gain the necessary skills, experience, support and contacts they require to work or volunteer in the music industry, while providing training, resources and guidance to help music businesses build a truly inclusive work environment.

The project launches with a new survey, open to any Deaf or disabled person who works or is aspiring to work in the industry. Their responses will play a key part in shaping the programme over the next three years.

Over that period, we are aiming to create:

  1. A Beyond The Music Network – a place for Deaf and disabled people working or seeking to work in the industry to meet, network, share ideas and find support around navigating the industry.
  2. Structured opportunities for Deaf and disabled people to develop their skills through training, mentoring, shadowing and skilled employment and volunteering opportunities.
  3. Accessible Creative Environments training – a new training course tailored to support companies within in the industry to create a truly inclusive workplace culture.
  4. An Accessible Employment and Volunteering Toolkit
  5. A Future Leaders programme – a year long skills development programme designed to help talented Deaf and disabled people develop the skills to lead the industry in the future.

Head of Volunteering and Skills Development for Attitude is Everything Paul Hawkins said

“This is a challenging time for everyone in the music business, especially within live events. The industry’s #LetTheMusicPlay campaign has highlighted the gravity of the situation, but, as we plot a pathway back from coronavirus, Attitude is Everything believes it is crucial that Deaf and disabled people have full and equal access to any employment opportunities on offer.”

Beyond The Music will allow us to try and identify why Deaf and disabled workers are so underrepresented in the sector, and to take positive action to implement change. The first step towards that goal is the survey we are launching today. We are enormously grateful to the National Lottery for funding this project, and also for support we’ve received from venues and others in the business. More will be needed on the road ahead as we strive for equality and inclusivity.

A number of music industry organisations are already backing Beyond The Music, with The Barbican, the Brighton Centre, Manchester Arena, the SEC, the South Bank Centre and Norwich Arts Centre all offering expertise to a Venues Advisory Group that will help formulate a strategy around the survey findings. Further support has been confirmed by Sony Music and Youth Music.

Industry umbrella body UK Music have also invited Attitude is Everything to join their Diversity Taskforce to help ensure access for Deaf and disabled people is high on their agenda. UK Music Acting CEO Tom Kiehl said:

“For a number of years UK Music has been a proud supporter of Attitude is Everything’s great work to improve access to music and the music industry for Deaf and disabled people. Beyond The Music is an exciting new initiative that everyone must now get behind. We look forward to working with Attitude is Everything on this and welcoming them to the UK Music Diversity Taskforce.”

Alongside the initiative, we are pleased to announce a new role within the Attitude is Everything team. The Skills Development Manager will lead on our work with Deaf and disabled people aspiring to work in music by helping them to access training, advice and guidance and brokering placements with music industry partners.

To find out more about Beyond the Music and how you can support it, please contact Paul Hawkins via paul@attitudeiseverything.org.uk.

The Evaluator is looking forward to helping this organisation to measure the impact of it’s work, on individuals and on the music industry too. As part of this project we are looking into concepts including ambition, what holds people back, and how industry can change attitudes over time.