Testing the strength of evidence for impact (TestE)
Please find details about the method here.
Kenya_Amani Women group
By Millicent SIFUNA 2020?, RISE Centre Kenya.
Purpose with the survey
The key aspects of our surveys are change and sustainability.
Concerning change we measure two aspects; i) contribution tracing; is it our programme that causes the change, and ii) outcome; do we reach Outcome?
The measurement of change needs a reference. The contribution tracing measures against control, the control can be a randomized population that did not take part in the programme.
Measurement of change can also benefit from the macro survey, how is the situation elsewhere at large, in the region, in the country, in the world?
The outcome measures against the base-line, thus the situation within the population prior to the initiation of the programme.
Outcome benefits from progress markers that are identified during the planning stage of a new programme.
Concerning sustainability, we also measure two aspects; i) institutional capacity and ii) financial balance.
Institutional capacity is a wide concept that covers all aspects of the programme management’s ability to maintain the programme.
Financial balance measures whether the income from the programme, without donations, are more than the costs. Sustainability is with the ROPE terminology referred to as the impact.
Purpose of the manual
The survey manual is addressed to HR&S survey management team, who must read all these sections. It aims to assist the HR&S team in preparing and implementing surveys. This survey manual clearly outlines the methodologies for managing, conducting, and reporting on HR&S surveys.
Team: Interviewer, photographer, interpreter, coordinator.
Equipment: Questionnaire, paper, pen, voice recorder, camera.
About the programme
Please insert the information about the programme.
Approach to data collection
The manual follows the subsequent steps to pass through when carrying out a survey for this programme. This survey will be used to evaluate the impact of the programme activities on the livelihoods of members of xxx. The target population will be drawn from members of xxx. We will employ random sampling in selecting a sample of respondents from the population. Both primary and secondary sources of data collection will be used. Primary data will include semi-structured questionnaires while secondary data will involve written records. The questions will have both open and closed-ended questions. The validity of the questionnaires will be determined by HR&S Sweden. Consent to collect data will be sought from xxx.
For purposes of statistical measurement, the typography of progress markers from the programme was generated prior to the start of the programme. This generated three types of information:
These will be acquired from three different sources; i) interviews with people in charge (plus a review of book records if any), ii) interview with participants, iii) observation by interviewers. The indicators used here will include: xxx
Qualitative information will be obtained through; i) interviews with people in charge and ii) participants. The sources of information will be social harmony at homes, contribution to family, and financial independence, etc.
These will be obtained through an interview with; i) the person in charge, ii) direct examination of record books and iii) personal interview with members. This include; loans are given, delinquent loans, money in the bank account (Local / Euro), etc
Please tick the boxes that apply to you.
- I AGREE to take part in this survey / I DO NOT AGREE to take part in this survey.
- I have had the study explained to me.
- I have understood all that I have read and heard and I have had it explained to me and had my questions answered satisfactorily. I understand that I can change my mind at any stage.
- I AGREE that photos are taken and posted on HR&S and Action10 websites and social media platforms with the purpose of explaining to supporters how their investments have been used and encourage new investments / I DO NOT AGREE that photos are taken.
- I AGREE that videos are taken and posted on HR&S website and social media with the purpose of explaining to supporters how their investments have been used and encourage new investments / I DO NOT AGREE that videos are taken.
- I AGREE that testimonies are taken and posted on HR&S website and social media with the purpose of explaining to supporters how their investments have been used and encourage new investments / I DO NOT AGREE that testimonies are collected.
- In case I have agreed on photos/videos/testimonies as above, I want my name to be mentioned / I DO NOT want my name to be mentioned.
- In case I have agreed on photos/videos/testimonies as above, I have selected the below to be posted: description of photo/video/testimony No 1, description of photo/video/testimony No 2, etc.
I (name of the person taking consent) have explained the study information to the participant above, and that she/he has understood the nature and purpose of the study and consents to the participation in the study and/or photos and/or videos. She/he has been given the opportunity to ask questions which have been answered satisfactorily.
Participants names, signatures & date:
Investigators names, signatures & date:
Guidelines for completing the survey forms
The person doing the interview should explain the reason for the survey in simple but clear terms. For each respondent, the interviewer must conduct the interview in ethe same way in order to minimise differences in responses. The interviewer should not be patronising. Also, a rushed interview or the interviewer’s lack of interest might affect responses. Participation in the survey is voluntarily, and the respondent can refuse to be interviewed at any time. Interviews with the respondents will be face-to-face, in the local language, using paper and pencil questionnaires. An interpreter may be required.The interviewer will read the questions and possible responses if indicated and mark the respondent’s answers on the questionnaire.
The role of the interviewer is to ask questions, answer the respondent’s queries, record answers and edit the completed questionnaire. The interviewer must always check to ensure that the respondent has understood the question by using certain techniques like neutral probing. It is the role of the interviewer to set the pace of the interview and ensure that the respondent is focused and interested.
On the other hand, the respondent should cooperate with the interviewer and follow all the instructions. She/he should listen to questions attentively without interrupting, and ask for clarification when the question is not clear.
Guidelines for completing each question
We use open-ended questions. An open-ended question is a question that is open to any answer. They are not limited to one- or two-word answer, instead, they have multiple potential responses, and they often give room for further probing by the moderator.
Part A: General information
- Age of Respondents:
Below 20 years / 21-25 years / 26-30 years / 36-40 years / 41-50 years / 51-60 years / 61-70 years / 71-80 years / above 80 years.
- Respondents level of education:
No formal education [ ] / Primary [ ] / Secondary [ ] / College [ ] / University [ ] / Others specify……….
- Current member status:
Still a member of the programme / ex-member ofthe programe.
Note: For ex members kindly ask the question in past tense where necessary.
- How long have you been a member?
Below 12 months [ ] / 1-3 years [ ] /4-6 years [ ] / Above 6 years [ ]
- Which position did you hold in the group?
Official / member / other specify…
- Have you ever obtained a loan from the programme?
Yes [ ] / No [ ]
- What did you use the loan for?
Please indicate the extent to which the programme has helped you realize improvement over the last period of time one year / two years / three years / five years. Tick your response in the appropriate answer box.
- Since I joined the programme, I have managed to improve my knowledge.
- Since I joined the programme, I have managed to improve my skills.
- Through the programme, I have been able to pay my children’s school fees.
- Since joining the programme, I have managed to increase my monthly income.
- Since joining the programme, I have been able to establish and improve income generating activities.
Part B. For the group leader
Group structure and attributes
Please indicate the officials present; name of the person, position and the time that the person has been in the office.
- What motivated the initiation of the programme?
- What motivated the formation of the programme group?
- Indicate challenges that your group has faced.
- What are you proud of that you see your group has achieved.
- How would you desctiber the group cohesiveness.
- How would you describe the group operations.
Membership & sustainability
- For how many years has this group been in existence?
- How many members did the group have when it first started working with HR&S?
- What is the current group membership?
- Are there any other people from the community seeking to become members? If there is, how many?
- Are there any people who left the group? If so, how many?
- What are the reasons for people to have left the group?
- What is the number on average that normally turns up for group meetings?
- Please indicate how your group has grown in numbers over time, for example per year.
- Do members of this group have other saving options? If yes, which ones?
Money lenders / Borrowing from friends and relatives / Borrowing from local money lender or cooperatives / Hire-purchase or lease finance / Venture capital finance / Table banking or revolving funds or investment clubs / Banks / Other saving groups / Others, Specify
- Does the group have a separate bank account? If not, why?
- Did the group experience an increase in total revenue over the past yearsof the programme?
- Please indicate in local currency how your savings have grown over the years.
To what extent have members of programme group experienced advantages with the project?
- All the money belongs to the group / Members savings are not taken away but instead used for loaning / Interest earned remains with the group / Sources of finances used for startup of the project / Others, Specify
- What is the approximate size of total assets owned as a result of the programme?
Trainings & capacity building
- Does the group benefit from any training activities as a result of this initiative?
- If yes, what skills have you learnt as a result of this and which one do you use?
- How frequent were these meetings?
- What skills learnt by other members in the group has been used?
- Have members applied any of these skills in other contexts e.g. family, individual business
Accountability & group governance
- Do you keep a record of your group finances?
- If yes, what forms/records are you using to keep track of your group finances?
Ledger / Cashbook / Logbook /Others, Specify
- What is the maximum amount one can borrow?
- What is the least amount one can borrow?
- What is the largest loan take in the last per year by any member of the group?
- Please indicate how the maximum amount of loan one can borrow has grown over years.
- Which repayment schedules do you use? Please tick all that apply.
Regular instalments / End of loan term /Others, Specify
- How easy is it for members to borrow money?
- Have members had to extend their repayment period? Please reply per year.
i) Number of members per year, ii) Value of their loans, iii) Size of extended repayment period.
- What has the loan been used for?
How difficult is it to obtain external finance?
Part C: Focus group discussions
About the participants in each focal group; as related to Section A.
The objective of this activity will be to learn how the group members use their loans. The interviewer will ask the group members what they use their loan for. She/he will generate a table with the activites and each individual will raise the hand if she/he took a loan for the purpose mentioned.
The main aim of this will be to learn the types of businesses started as a result of the programme. The interviewer will ask about the types of businesses that different individuals started since joining the programme. She/he will ask individuals to raise their hands to respond that they had started the business types. She/he will count the number of hands raised and record in the table.
- Has monthly income increased as a result of loans through this programme?
- Are individuals expanding their businesses?
This will help get a more reliable picture of the ctivities to business development/diversification and how individuals can be assisted to improve their monthly income.
The purpose is to assess the programme performance in ensuring socio-economic development of members. The interviewer will ask members to rate how the project had ensured their socio-economic development in the following order.
- Built trust and empowerment which is aimed at benefitting the society
- My saving ability has been enhanced
- I can provide adequate food for the family
- I can keep my children in school as a result of timely fees payment
- I have improved my household assets
- Since joining, I have made a number of investments
- Wealth creation is enhanced
- Health and wellness are guaranteed
- Improving housing
- I now practice modern farming
- Others specify…………
- What are your groups plan for the future?
- Any new venture?
- What are the benefits with this programme?
- How can this programme be improved?
- In which ways is the programme operating well?
- In which ways can the programme operation improve?
- Anything else that you want to share, propose or discuss?