Testing the strength of evidence FOR Impact (TestE)
HR&S has developed a management strategy for the assessment of if outcome and sustainable impact have been reached. The strategy is based on evidence and is called TestE (Testing the strength of Evidence). We compile evidence for each expected outcome and each expected sustainable impact and use scientific evaluation methods including control, randomization, quantitative statistics, qualitative probability assessment, and contribution tracing.
The strongest cases use multiple forms of evidence, some addressing the weaknesses of others. Reported effects should be plausible as outcomes of the programme activities and consistent with our management strategy ROPE and the observations compiled as evidence should be easier to reconcile with the programme’s claims than with other possible explanations. It is of no importance whether an outcome was achieved or not, this is plain testing of evidence without bias. Lessons learned on lack of outcome is as important and give rise to reflections and informed decisions.
Monitoring data are addressed using Trello.com.
Measuring Change & Sustainability
The key targets of our surveys are change and sustainability.
Change
Concerning change we measure two aspects:
i) Contribution tracing; is it our programme that causes the change?
ii) Outcome; do we reach Outcome?
The measurement of change needs a reference and the contribution tracing measures against a control. The control can be a randomized population that did not take part in the programme. Measurement of change can also benefit from the macro survey, how is the situation elsewhere at large, in the region, in the country, in the world?
The outcome measures against the base-line, thus the situation within the population prior to the initiation of the programme. Outcome benefits from progress markers that are identified during the planning stage of a new programme.
Sustainability
Concerning sustainability, we also measure two aspects:
i) Institutional capacity.
ii) Financial balance.
Institutional capacity is a wide concept that covers all aspects of the programme management’s ability to maintain the programme.
Financial balance measures whether the income from the programme, without donations, are more than the costs.
Outcome challenges
It is true that we face a number of outcome challenges. Even though our sincere ambition is to ensure TestE in all of our doings, we have not yet fully reached this goal, but we will. Reasons being are for example that i) there is no tradition of local survey management within the aid sector programmes and ii) we lack educated survey managers.