Impact investment benchmarks: what, why, how and when
The impact investment community is hungry for impact benchmarks, but many investors are trying to run before they can walk with their data.
This post outlines what benchmarks are (what), challenges motives for their use (why), signposts to current sources (how), and considers when sector-based benchmarks might be ready for wider use (when).
Impact investment and the pursuit of benchmarks
An impact benchmark is a measure that helps stakeholders understand the performance of a project over time, or compared to similar interventions. Essentially they are comparison points. The idea of comparing impact is major focal point in the impact investment field right now. Respondents to the GIIN’s global impact measurement and management survey ranked benchmarks the most important resource for improving impact performance.
Impact investors are not alone in feeling this imperative to compare. I am systematically asked by clients to help them benchmark their performance. And it is one of the greatest disappointments when we break the news to clients that it is inappropriate to compare to other impact studies, even using structured methods like SROI ratios (I know, I know, it is frustrating for us practitioners too!).
The push towards standardisation in social impact reporting is a centralising call within our field. I have no doubt that it is the future. However, in the short term, efforts to standardise are fraught with complications.
To contextualise this, it took hundreds of years to achieve standards in the accountancy field (and they are still slightly different all over the world). Impact measurement metrics must mature in the same manner. However, the challenge is much larger, because the standards we agree on must encompass a much broader range of subjects and metrics.
Do you know why you want a benchmark?
I have a gripe with benchmarks. The use of the word is often loose and lazy. People ask for comparators without really knowing why they want them or what research question they will answer . There is a blind innate social pressure that we feel to compare, and this is dangerous in a field where context is everything, and metrics are still very diverse.
The temptation is to reach for the first framework which sounds relevant and credible. The problem with this kind of (lazy) impact reporting structure is that you often measure the wrong things. Most available frameworks, even the IRIS+ metrics are full of outputs (as oppose to outcomes) which are poor impact measures. Additionally, unless the intervention is very standard you are likely to be measuring the wrong things, which will mislead your understanding of real impact. This is why we always advise starting with a Theory of Change.
To give you a simple example, you might have two school attendance programs. Perhaps one is working with a general cohort of children and the other is working with a group of children with special needs. The standardised impact metric around school attendance might be number of days of unexplained absence.
The first, general cohort are much more likely to make improvements on this indicator relative to the amount of effort put into an intervention. For example, a 75% decrease in absences might come from fining parents. However, small changes for the children with special needs, for example, a 20% decrease, might be indicative of life-changing outcomes.
Making it to school might be the end result of very complex self-esteem and independence building activities. But as an investor, your quick sector-level comparison would lead you to place your funds with the first cohort because according to the benchmark, this is where the most impact is taking place.
The advantages of benchmarks
Benchmarks might not be an exact science, but there are good reasons to pursue them if they are handled with care. For impact investors they will facilitate two important breakthroughs.
Setting targets and predicting impact
Most impact management is still done retrospectively, after the intervention is completed. And even then, the findings are not always channeled back into design improvements. For example, the GIIN found only 65 per cent of impact investors are using impact data to set and revise goals. This is a major missed opportunity for predicting and enhancing performance. Where relevant benchmarks exist, discussions can be had on success factors and the appropriate level of impact to pursue in your own context.
Aggregating data
The bulk of the impact investment market is made of large institutional investors and asset managers like banks, development finance institutions, pension funds and insurance companies. To efficiently manage impact at that scale there is a need for consistent metrics which can be aggregated across investments, even funds. To aggregate we need to be sure that we are combining apples with apples, not pears.
How to benchmark: three places you can find impact investment benchmarks now
A social benchmark is composed of an outcome level, a defined stakeholder group, and a set time period. Critically, the benchmark should be calculated in a consistent manner, for example, survey questions on life satisfaction are measured on several different response scales. Make sure yours is consistent with your benchmark and that you are calculating change against that scale (see my post on how to select indicators).
To achieve this level of control you have three options:
1. Past performance
Using the baseline performance of your own intervention is a great way of benchmarking. It ensures that you are picking a benchmark that accurately reflects your context and that the metrics are calculated in the same way. It will allow you to understand and explore impact performance for individual investments within each reporting period.
2. Control or comparison groups
Good data on the broader trends for the population that you are helping may be available in some cases. For example, there is good data on outcomes for young people in out-of-home (residential) care and for ex-offenders. This will often be available in the form of national, government datasets or sector publications. This kind of benchmark will give you the additionality of your program, rather than the performance of your program compared to similar interventions.
3. Established intervention areas
Some interventions are more standardised than others and have impact reporting frameworks which are widely used across sectors. If you have an investment in one of these areas, you can benchmark with greater confidence.
Interventions with frameworks include housing (see the GIIN’s 2020 report on benchmarking housing investments and HACT’s impact valuation workin the UK), healthcare, education and early childhood services (often based on country frameworks) and environmental initiatives (see the Green Bonds Standards). There are more and more of these frameworks springing up, for example I can think of at least four digital inclusion indices including the Australian one and the World Benchmarking Group’s work on Digital Inclusion. So check to see if there is sufficient data available covering the outcomes you are interested in by size and relevance of the datasets that you are benchmarking against.
When will sector benchmarks be widely available?
These days it seems like everyone is developing a standard or a framework that will one day encompass all social (environmental and economic – but especially social) outcomes. Impact geeks like me love following these initiatives, but for the ordinary impact investor they must be exhausting.
The GIIN’s 2019 survey of impact investors found that on average, impact investors are each using three different frameworks to measure their impact (such as the Sustainable Development Goals (SDGs), IRIS+, UNPRI, GIIRs e.t.c.). Curiously, they also found that no framework is actually seeing a decline in use since their last survey in 2017. Additionally, investors who are focussed primarily on social outcomes have the least coherence in their choice of tools, metrics and frameworks.
It is not surprising then that investors would like to see simpler sector-based benchmarks. So, with all these frameworks and tools to navigate, when might that be a reality?
A few hurdles remain before impact investors will have reliable and easy access to sector benchmarks. For benchmarks to exist, there will need to be:
a) A shared understanding of the principles of impact – the Impact Management Project (IMP)’s dimensions of impact has accelerated a common frame of reference;
b) A mapping of existing outcome metrics – the IRIS+ catalogue has made huge progress on this front but the linkage to outcomes and financial proxies needs work;
c) Controls on how metrics are calculated – this is a major missing piece;
d) A conceptually discrete taxonomy of outcomes – many efforts have been made on this front including by Big Society Capital and IRIS but these are still too orientated around interventions rather than capital or outcomes;
e) Fluency between financial reporting and impact reporting formats.
The controls around how metrics are calculated (point c) is the most important next step for investors to be able to compare and aggregate with confidence. Where benchmarks currently exist, for example in the IRIS+ catalogue, there is still fuzziness around how to calculate them and what levels of assumptions are acceptable.
A straightforward metric like number of jobs created is still interpreted very differently by impact reporters. Does it count if the job is a short-term contract, or a zero-hours contract, or what if the person quits after a month? We need a workable rule here such as the person needs to remain in the job for six months. Similarly, there are metrics which sound comparable but may be misleading like “number of children with access to education”, “number of children enrolled in school” and “children attending school”. These might seem comparable but in practice they can mean very different things.
The good news is there is a lot of (tech enabled) energy going into standardisation right now. I think we will need to wait at least five more years before you begin to see impact reporters confidently benchmarking their impact across sectors. In the meantime, you should check the leading frameworks to ensure you are aligned with the best practice in impact investment reporting.
Final thoughts: impact investment and the pursuit of benchmarks
Every system of measurement has defined units, methodological rules and reference standards. The impact measurement field is going through a process of rapid maturation towards this, a process that took financial accounting several centuries to achieve. The difference is that impact measurement metrics are far more diverse than classical accounting.
It is brilliant to see the impact investment field aspiring for the best in impact performance and reporting. As you look to apply benchmarks to your impact portfolio remember to start by answering the fundamental question of why you want a benchmark. This might lead you to discover there are effective ways of comparing your performance which do not depend on whole sectors aligning and comparing their value.