Five essentials for an impact management system
An impact management system is like an accounting system: it tracks activities, monitors value creation and helps set performance targets.
There is craft in creating a structure that fits your organisation, but anyone can put the essentials in place and here are five steps to get you going.
We need to get serious about impact
Social services are going to be hit by a wave of need as we progress into 2021. Many of the temporary COVID fiscal measures provided by governments will drop away removing the safety net for those with reduced incomes in this global recession. We need to start building a social economy that really works for people, and that delivers value for money.
In a tight funding environment, donors, including the more discerning public investors, will compare similar services based on their quality of impact evidence. The maturity of impact management systems will be put to the test this year!
Setting up a system
So where do you start? Most purpose organisations have a scattered collection of case-studies or quantitative measures that have been assembled for certain funding streams. Knowing how to set-up a system that is integrated into your organisation is the next level of maturity.
Naturally, there will not be a one size fits all solution for every organisation. The scale, services type, stakeholders and quality of the existing and needed evidence base will all influence your approach.
The essence, however, remains the same for all organisations. You need to build an overarching impact narrative that clearly identifies the intended outcomes; and then set up a quantitative reporting mechanism to track your causal contribution to any changes.
Five essential steps
The first step for any organisation is to map your impact story through a Theory of Change. This will articulate your most material outcomes and capture how and why your services achieve these changes.
I sometimes encourage organisations to draw up a simple, central change logic to get started. This ensures a basic architecture can be mapped out for your system. If you take this approach, be prepared to revisit your Theory of Change and build more detail onto it as you progress (perhaps on a programmatic basis) because it will ultimately be the blueprint specifying what outcomes are measured.
Your Theory of Change is your impact narrative and your strategy. Remember that best practice is to involve internal and external stakeholders in identifying meaningful outcomes from your work.
I’ve laid out a method for building a Theory of Change in this blog.
2. Review existing reporting
Your Theory of Change will provide you with a list of impact areas to measure that are created by your work. These can then be cross compared to existing evaluation and outcome data to review your existing coverage.
This exercise is about identifying any gaps. Are there outcomes you do not evidence, but which are critical to your value creation story? For example, funders might identify “innovation” or “relationship building” amongst funded services as critical, but they might have overlooked collecting relevant data as they are often focussed on end outcomes for beneficiaries.
The cross comparison should reveal variations in methods of reporting on impact, and this will provide an opportunity to harmonise approaches across services and funding streams.
A bonus from this exercise is that organisations often find they can save work by stopping some data collection. This is because they discover they are collecting data that is either irrelevant (perhaps a relic from old processes) or too poor in quality to be useful.
3. Develop a framework
The review will lead into planning ongoing impact management systems. This is where you will need to make some trickier decisions about the quality of evidence you require (robustness), while balancing this out with the available resources for managing that impact data.
Your material outcomes and implementation theory should guide an examination of the existing evidence base and what kind of ongoing tracking (or primary data collection) is needed to prove your impact. If you are implementing a service that has strong evidence underpinning it, for example, a hotline for domestic violence victims, you may not need to collect as much evidence on the impact as a new or novel service.
Your Theory of Change will become the blueprint or structure for documenting a framework so here you should capture:
- What evidence (indicator types) is appropriate for tracking each of your outcomes
- How you will collect the data
- Your sample size and frequency of data collection
- How you will (if at all) evidence causality.
- Benchmarks if desired (if prior years or by sector)
Setting yourself up with an indicator bank can also make ongoing management more seamless as users will have some flexibility to ensure the data they collect is relevant to the context in which they operate, yet consistent with the needs of your framework.
Designing data collection tools, such as surveys, should also be drafted at this point. Logs should be included for recording secondary data streams or using activity monitoring data like the number of service users.
Strong organisational systems depend upon staff engagement and capacity. If your team believe in the value of impact management, they will be a wellspring of observations and ideas that will make your data more meaningful. Teams who do not understand the purpose of the data they collect are likely to collect less data and cut corners.
Work with your teams to build their skills in managing good impact data and analysis including qualitative and quantitative sources. More about creating impact thinkers in your organisation in this blog.
5. Set-up data systems
Today we are lucky to have a range of digital solutions that can streamline impact management. Your aim in designing a digital system should be to have a central dashboard for collating evidence and reviewing performance across services and time.
Purpose-built impact management platforms normally include tools for collecting the data (automated online surveys), pre-programmed analysis and a dashboard. If a standalone impact management system works for your organisation you should certainly check out the available products. Just remember to use your own discernment on whether the type and nature of the data you are collecting is appropriate as these systems will reduce flexibility and control.
Another option is to integrate your impact data into existing organisation software. This might be a CRM or a finance system. Ultimately this will put impact data within easy access of senior decision makers and reduce duplications.
Integration can be time intensive and is likely to require some engineering. Before you take that leap, you might want a simpler solution which is to design a more manual database and data visualisation dashboard for example with Power BI or Tableau. This will give you more control and tailoring.
Regardless of your chosen approach, make sure you have set up an appropriate database for managing data collection. Accompany this with written operating procedures. Ultimately you want to pick the solution that enables you to make joined-up, smart decisions.
Play and repeat
When it comes to road testing your new framework and system, my advice would be start small. The Theory of Change should outline the impact strategy for your whole organisation, but you can test the impact measurement system by piloting it in one area, adapting, and then rolling it out more broadly across other departments.
An impact management system should be an ongoing learning tool. The collection and review of data should take place routinely to enable you to properly allocate resources, stop work, pivot, innovate and scale. Make sure you system is genuinely flowing and interacting!
Final thoughts: five essentials for an impact management system
We are in the midst of enormous social turmoil. It will not be enough for purpose organisations to continue doing what they have always done and hoping for the best.
As we respond to the social challenges of 2021 we need to see more organisations getting serious about impact by making it a core part of their operations. The five essential steps above will help services, organisations and funds explore how they can deepen their efforts.