What are the Indices of Multiple Deprivation?

/in , /by

What are the Indices of Multiple Deprivation?

If you have ever read one of our reports, you’ll likely see reference to something called the Indices of Multiple Deprivation, or ‘IMD’.

We use the IMD as a powerful tool to analyse and understand the barriers and socio-economic status of audiences, participants, and volunteers. We use postcodes to measure the IMD status, and this means that people find it easy to answer. This does mean that we are looking at the household status too.
The results give our clients an idea of which people they are attracting to their projects and what barriers they may be facing in life.

Statistically, people who live within areas of greater deprivation have more barriers in their lives and are less likely to engage with the arts and nature and are more likely to have greater issues with their wellbeing. That’s one of the reasons funders do like to understand if projects and organisations are reaching people from the most deprived areas.
How are the indices of multiple deprivation worked out?
The IMD is worked out using a range of factors which are assessed across the UK and given a score. The factors are listed below:

  • Income deprivation: how many people in the area are experiencing low-income levels.
  • Employment deprivation: what is the rate of unemployment in the area?
  • Education deprivation: what are the average qualifications within the area, and how easy is it to access quality education?
  • Health deprivation and disability: what is the average life expectancy and disability prevalence in the area?
  • Crime: how much crime occurs within the area
  • Housing deprivation: how affordable is housing within the area, and are there other barriers to people accessing housing
  • Living environment: what is the air quality like in the area, and are there green spaces?

These scores are compiled to give a ranking, which are then categorised into ten deciles. The entirety of England falls equally into one of ten deciles, with decile 1 indicating the most deprivation, and area 10 having the least deprivation.
If you were targeting deprived areas, you might want to reach 50% of people living in IMD1 and 2 for example. Or if you wanted a perfect section of the UK represented in your project, you might want to see 10% of your audience from each decile.

We think the IMD is a useful tool and aim to use it in every evaluation where relevant.

Cheers to a Festive Break: Wrapping Up a Year of Amazing Projects!

/in , /by

As the year draws to a close, we reflect on how grateful we are to work with such outstanding clients, providing evaluation and data analysis services throughout the year. It has been our pleasure to have been involved in some of the world-changing positive projects our clients have been working on.

In the spirit of the festive season, the Evaluator team will be taking a break to refresh and recharge during Christmas. Therefore, we would like to inform you that our office will be closed for the holiday period starting from the 22nd of December and will resume operations on the 8th of January.

We appreciate your understanding, and we assure you that we will return in the new year fully revitalised and ready to continue delivering our exceptional service.

 

Below are some of the clients we’ve had the pleasure of working with this year:

 

Wishing you a Merry Christmas and a Happy New Year!

New client: Community Arts by ZK

/in , /by

The Evaluator is delighted to be working on a new project, Healing Arts for All (HAFA) from Community Arts by ZK.

This project is place focused and based in Pendle, on our doorstep!

HAFA describes itself as a “3-year people powered art and heritage project”, funded jointly by the National Lottery Heritage Fund and the Paul Hamlyn Foundation.  The project will include participants working with professional artists as well as gaining an understanding of working within heritage.  Heritage will focus on understanding the history and impact of Pendle’s Brierfield Mill, an iconic grade II listed former cotton mill built in 1931.  This mill originally attracted the South Asian community to move to the area to work in the textile industry.

You can see more about  Community Arts by ZK on their website here. 

We are particularly looking forward to evaluating the personal impact that this project, blending creativity and heritage, has on the participants from the South Asian community.

 

Learning about data; Working with quantitative data

/in , /by

In a previous blog, we described quantitative data as data that can be counted or measured in numerical values. A spreadsheet is a good basis for organising data ready for analysis. Now let us imagine you have some data to hand, this is how you might want to try and use it.

Ways to analyse quantitative data.

Example 1: Looking at what the data tells you itself.

We generally start with calculating averages to consider the range of data (the highest score and the lowest score collected).  If we consider 2 sets of data (Data set A and Data set B):

From Data Set A, it is apparent that a similar experience was shared by the large majority of the audience because the average score is 40, but the range is from 36-44. That means the lowest answer was 36, which is quite close to 40 and the highest was 44 which is still quite close to 40.

However, in Data Set B, there was a much wider range of experience.  This should be questioned as to why experiences are so varied, illustrated by the wide range in the data.  Why did some individuals have such a higher score than others? Did some people have a different tutor, or venue, or have more time? There are many reasons why the range could be so high.

Example 2: Comparing data to other national data available.

Another possible way to compare data could be as in the worked example below where the Warwick Edinburgh Scale of Wellbeing was used to measure participant wellbeing.

Participants scores were collected over a period of time, firstly at the start of their engagement and then, at the end.  This graph represents a comparison in:

  • Each individual’s wellbeing from the start versus at the end. This is often a reflection of the impact of their engagement.
  • The individuals’ wellbeing who took part versus the national average
  • The individuals’ wellbeing score versus the NHS score of 40 which is believed to be indicative of poor wellbeing.
  • The individuals’ wellbeing versus one another.

Using quantitative data to help with forecasting & future decision making

Using percentages makes making comparisons easier to relate to and understand.

We use an example of a cinema and record audience attendance numbers of time.  Attendance could be affected by the popularity of the content of the film, seasonal trends or weather.  The trend line shows the overall audience is growing and then this line can be used to forecast on at the same trajectory.  This will help you to identify if you are likely to achieve your audience targets.

This information can also help you to make decisions about your capacity too and streamline your resources. For example, if you were thinking about moving a cinema location to a larger space, it would be worth looking at the trend line to think about venue size. If you were looking at two venues and one had a capacity of 100, and another 200, you might want to look at what your predicted audience size would be in six months’ time and negotiate the lease or hire accordingly. This is an example of making a data-driven-decision, something we are passionate about at The Evaluator.

Taking notice of negative space in data

It’s important to take note of the 70% who are agreeing but also to take note of the 30% who are disagreeing and find out why this is the case.  It is worth delving a bit deeper into on the minority and finding out what was the cause of their response.

At The Evaluator, we tend to represent 3 answers ‘yes’, ‘no’ and ‘prefer not to say’ in our reporting. If there’s a high number who indicate that they would ‘prefer not to say’ then it would be suggested they might insecure about completing the survey.  Often their indecision is explained in their qualitative answers and this is worth taking note of when creating future surveys.

Dealing with ‘satisficing’ survey responses.

This is the term we use when people respond with the answers that they think you are looking for and may not read all the questions.  An indication of satisficing is when respondents repeatedly choose 3 when presented with a 1-5 scale. We don’t come across these very often, as we spend a lot of time making sure our surveys are easy to complete, and varied, but if we do spot them, we will try and remove these answers from our data analysis.  It’s important to encourage honesty in answering survey questions.

A final tip

All data can be segmented but you do need to think about the time you have to spend on this as it is time consuming.  In segmentation ideally you are looking for what is more than 10% different to the average. In the example of the graph below, it is worth looking at segmenting to see the results demographics of the areas that are 10% above and 10% below the average line.

 

 

Learning about data; What is quantitative data?

/in , /by

What is Quantitative Data?

Quantitative data is data that can be counted or measured in numerical values.  As with qualitative data, there’s a good chance that you already have some collected for your organisation.

You might have collected some of the following:

  • Sign-in sheets
  • Feedback forms
  • Surveys
  • Polls
  • Social media statistics
  • Reports

We often find clients already have quite a bit of data they didn’t know they had collected!

The differences between primary and secondary quantitative data.

There is a distinction between primary and secondary quantitative data.  Primary data is the data that your organisation has collected directly, such as footfall counts or feedback forms. Secondary data is data someone else has collected, for example a national age profile, or a partner shares their footfall data. It can helpful for you to draw comparisons between your collected data and national averages to see how your organisation compares.

Overcoming the challenges of working with quantitative data

There are some challenges to working with quantitative data.  Often the biggest challenge is that it’s not collected in a format that makes it easy to compare to other collected data, or to secondary sources. The best solution for this is to plan in advance and use standardised questions at every opportunity to collect data. The answer format is also important so choosing a standard answer format will make it easier to compare data.

Tracking codes can be useful in identifying if you know the same person will be answering a survey multiple times so you can monitor their progress.  A tracking code can be created within a survey using data such as: a combination of a person’s date of birth and their initials.

It’s always important to date the data particularly for paper copies of surveys which makes identifying the event possible and the data relatable to that event. If one of your feedback forms reveals a problem with the venue or experience, you need to know the date on which that particular event happened to make sure you can address the problem. Don’t forget, feedback forms may be input or analysed as a batch of forms after a few months of collection so it may prove difficult to find out which venue the problem occurred at if you don’t have a way to check.

Managing personal data from surveys

GDPR (General Data Protection Regulation) are regulations which relate to how we retain and use personal data.  Within these regulations it is important to:

It’s important to maintain confidentiality and anonymity with personal information.  Recording date of birth and full name poses a risk to personal identity, however, recording only a date of birth is not identifiable. There are also additional regulations regarding collection data from under 16-year-olds. It is possible to collect identifiable information, but if you do so you need to ensure that the data is obtained with consent, is properly secured and then destroyed once no longer needed.

Thinking about when to collect data

Recording information in the moment is valuable so it can help to set up processes to ensure you don’t miss out! One tactic that works is to have a standard question you ask at the end of every event.  This, and the size of audience questioned can be collected for contextual purposes to see if the responses were representative of the larger audience.

Top tip – you don’t need to collect data from everyone!

Deciding how much data to collect

It’s important to consider whether sample sizes are large enough to provide you with sufficient data to base a decision on.  10% to 20% of the audience is usually a reliable sample size to base a decision on. If you hold many smaller events, it would be advisable to collect evidence from each event and consider it accumulatively to make decisions.

 

 

 

 

 

 

 

 

New Client: Love Withington Baths

/in , /by

The Evaluator is delighted to be working with a new client, Love Withington Baths.

The Evaluator is working with the community leisure centre to carry out a Social Return On Investment (SROI). You can read our blog post all about SROI here, but it’s basically a formal process of identifying what matters to people taking part, starting with the changes they identify. We are looking forward to chatting away to people using the centre, and figuring out what changes it has led to in people’s lives. It’s a really well used centre, so we should get to talk to lots of people!

You can see more about Withington Baths here. 

New client: The Bureau for the Arts, Blackburn

/in , /by

The Evaluator is delighted to be starting a new evaluation; Chip In. Isn’t that a great name? Northern, does what it says, and everyone loves chips!

Helping people, building community, changing lives!

Chip In is a place- based project which aims to engage and mentor young people aged 16-30 in volunteer opportunities across Blackburn with Darwen’s arts, culture, sport, heritage and environment sectors over the next two years. The Project aims to break down barriers which young people face in volunteering, bring together opportunities and build skills, confidence and community.

The Bureau Centre for the Arts is among over 160 organisations across arts, culture, sport, civil society, youth, and heritage sectors to benefit from the £4.6 million Volunteering Futures Fund.

Chip In (formerly Volunteering Now) is our two year volunteer partnership programme for young people aged 16-30. We have teamed up with a range of local partners in the borough to provide opportunities for volunteering in a range of areas such as events, music, project development, admin, exhibition invigilation, workshop support, stewarding, curation, collection handling, supporting youth sessions, social media and blogging, food growing and much more!

You can see more about the project here or read more about Volunteering Futures here. 

Successful Client: The £1.6 million Climate FORTH project gets the go ahead

/in , /by
Earlier this year, The Evaluator worked with Climate FORTH (Furthering Our Resilience Through Heritage) on their application to the National Lottery Heritage Fund, where we developed an evaluation framework for the whole project. They have just announced they were successful. That’s always a lovely day, sharing in the success of our clients.
The details are below:
A new £1.6million project in the Inner Forth has been granted just over £1million by the National Lottery Heritage Fund to boost the area’s climate resilience through the promotion and enhancement of local heritage.
image.png
Climate FORTH is the latest project from Inner Forth Futures, a landscape-scale partnership which has been operating across the coastal areas of Clackmannanshire, Stirling, Falkirk and Fife since 2012. Since the partnership’s inception, it has delivered both the Inner Forth Landscape Initiative (IFLI) and Wanderings and Windings projects, enhancing and celebrating the local area’s heritage while improving access through a series of waymarked walking and cycling trails.

The new Climate FORTH project will build on these successes by piloting methods of increasing resilience to climate and other changes in the local area’s natural and built heritage, and in its most at-risk communities.

The River Forth at Kinneil Lagoons_Falkirk_Chloe Wells_RSPB.jpg
The project will soon begin work on the delivery of several new initiatives, including green skill training for young people and community groups, enhancement of sustainable travel options around the local area, and capital works at four locations across the project area. The works will improve the climate resilience of greenspaces at Glendevon Drive in Stirling and Rannoch Park in Falkirk, as well as an important community building, Cochrane Hall in Alva, Clackmannanshire and the Gardener’s Cottage ruin at Valleyfield Estate in Fife.
One of the first activities to get underway will be a rebranding of sections of the National Cycle Network which pass through the Inner Forth. Supported by Sustrans, this work will encourage more people to make use of the network and help establish the landscape as a destination for sustainable tourism.
A key aim of Climate FORTH is to pilot new techniques that can be shared with other initiatives and policymakers to inspire further action. The pioneering work of the project will be closely monitored so the knowledge obtained can support Scotland’s aim of achieving net zero emissions by 2045 and help communities move towards a just transition.
Four communities in particular have been identified as being key to the project’s goals, with members playing an active role in decisions involving their local area through the co-design of local resilience plans. They are Hawkhill in Clackmannanshire, Fallin in Stirling, Bainsford & Langlees in Falkirk and Kincardine in Fife.
Climate FORTH Project Manager, Kate Fuller, said, “We’re delighted to have been awarded this Heritage Fund support that will allow us to show how the diverse natural and cultural assets of the Inner Forth can be adapted to tackle the threats of climate change. We’ve seen through our previous work just how special this part of Scotland is. Thanks to National Lottery players, Climate FORTH will enable us to re-invigorate our work with the amazing communities that live here and support them to lead by example as Scotland moves towards achieving net-zero.”
Dave Beaumont, Operations Director for RSPB Scotland said, “We are in a nature and climate emergency and the only way we can meet this challenge is by facing it together. Estuaries such as the Forth are where we are seeing impacts from climate change right now – as sea levels rise, storm intensities increase and rivers flowing into our seas are carrying floodwaters much more frequently. The communities and the incredible wildlife found in and around our estuaries are more fragile than ever and this project will continue our efforts to make them more resilient to the changes ahead.”
Cosmo Blake, Network Engagement Manager for Sustrans Scotland, said, “We are delighted to work with Inner Forth Futures to create a new brand identity and better signage along sections of The National Cycle Network. As well as creating a new brand identity, we are also developing eight new day trips with maps, directional signage and additional bike parking and maintenance stations at local heritage sites and transport hubs. We hope this will encourage more people to walk, wheel and cycle and make sustainable choices when visiting the Inner Forth.”
Heather Macnaughton, Strategic Partnerships Manager at Historic Environment Scotland said, “This funding will build on the exciting work our partnership has already undertaken in the area to help protect this heritage rich estuary from the impact of climate change. Working with the local community, Climate FORTH will shine a light on the area’s natural and built heritage and the important part it can play in improving climate resilience and biodiversity. By working closely with our partners and communities, Historic Environment Scotland looks forward to developing solutions for future resilience and moving together towards a just transition.”
Caroline Clark, Director for Scotland, The National Lottery Heritage Fund said, “This is such an important project for our times – Climate FORTH will enable communities to take the lead in developing their own local responses to the challenges of climate change and Net Zero. What makes this project even more significant is that it will then create templates that can be adapted for use by other communities across Scotland and beyond.”
Inner Forth Futures is a partnership between RSPB Scotland, Clackmannanshire Council, Falkirk Council, Fife Council, Green Action Trust, Historic Environment Scotland, NatureScot, Stirling Council and Sustrans. Climate FORTH is partly funded by The National Lottery Heritage Fund, with additional funding from the UK Government’s Shared Prosperity Fund, the Crown Estate, IFF members and the Scottish Government through Sustrans Scotland’s Network Engagement Programme. The project will be running from August 2023 – July 2026.
For more information, visit innerforthlandscape.co.uk.

Social Return on Investment; What is it? And do I want it?

/1 Comment/in , /by

Social Return on Investment (SROI) has been around for quite a while, but it’s still not that well understood. Our director, Kirsty Rose Parker, completed the training back in 2010 and has used it on many occasions.

Carrying out a SROI is a robust framework for writing and describing a wider context of value. It puts a financial proxy on to a  project. It is presented in monetary terms but describes value, not actual money. Because SROI is robust and has a structured methodology, it does have some key principles.

  • Change is change and might not always be positive; there may be some negative implications
  • Stakeholders are key to the whole process; an SROI should start with stakeholders
  • SROI uses financial proxies to value change. Throughout this SROI, existing measures of other similar experiences that could also make the changes our participants told us about have been used as financial proxies.
  • It can’t include everything – not everything that changes in people’s lives is down to the project. It is just as important to know when to stop.
  • An SROI should be truthful. It goes without saying really, but on an exercise like this honesty is the best policy. For an SROI to be believable it does need to be based on evidence and data, and not over-claimed.
  • An SROI should be transparent; explaining the process to give transparency to all the decisions made.
  • Results should be verified, either by stakeholders or experts or both

Some funders do recommend ‘Social Return on Investment’ and all respond positively to it. It is a good methodology and we often follow points 1 and 2 in other evaluations, as we like the structure of reminding us to keep an open mind and see what people tell us.

It works best on projects which make big changes to peoples lives; whether that is changes to mental health or overall health, if people get jobs, or are supported to get housing, for those living with dementia or addiction although the process can be adapted to almost all projects or even an organisation as a whole.

It works best on projects which have a clear start point which can be measured, and ideally would look at value over time. If everyone started a project on the same day, it would need to allow a few months for changes to take place. If a programme allows people to join at any time it can be quicker, as then a team of evaluators would be able to talk to people who had just joined and those who had been taking part for a while in the same month.

It is a process of reflection, and evaluation. It will help teams to think about the changes they enable and to see what their impact is. It can help to build pride in a project or organisation. It can definitely increase investment as it gives investors a clear sense of the change their investment enables.

To simplify; If your project changes lives, then yes you want a SROI. If you can get hold of people taking part then yes you can do a SROI. It does need to be done during a project, it’s not impossible to do one retrospectively but it is much harder, so do try and plan one in earlier rather than later.

Give us a call if you want to know more, we’d be happy to chat through the process. If you want to do it for yourself…

More information, including a guide to working out a SROI can be found here. 

New Project: Urban Forest Accelerator

/in , /by

The Evaluator is delighted to be starting a new project with National Trust, the Urban Forest Accelerator. We are excited to be working on measuring concepts of urban tree cover, tree equity, environmental justice and developing a new approach to measuring connections. Just as joined up canopies can be more that the sum of their parts, we believe our work relationships can also mirror this and aim to develop a new way to track this over time.

Along the way, we are lucky enough to get to work with inspirational people who want to transform and green our urban areas. We get to have discussions around who should decide what is best for urban trees, to look at tree cover metric developmental work and to use words like arboriculture career pathways and knowledge. Plus, we love trees!

The Urban Forest Accelerator is part of the Future Parks Accelerator and aims to address concepts including; partnership, urban forest capacity, sustainable finance, tree equity, community engagement and learning and scaling.