In 2018-2021, ABM’s Anglicans in Development (AID) funded program evaluations in the Philippines, Kenya, Papua New Guinea, and Vanuatu. Why?
The clearest reason is because evaluations provide AID and our partners with a detailed analysisof what has succeeded and what hasn’t succeeded in the program. Evaluations can also provide a more objective way for AID to be accountable to our donors, whether these are organisations like the Australian government or AID’s generous supporters. Three of the above five evaluations were conducted by external evaluators, and two were conducted by AID itself.
In Vanuatu between 2016 and 2021, AID’s partner ACOM (the Anglican Church of Melanesia) installed water supply facilities and organised adult literacy classes across 24 communities on nine islands in the provinces of Torba, Sanma and Penama in Vanuatu. ACOM adopted sustainability measures such as building local people’s skills and working through pre-existing community groups (church groups) so they could continue to maintain community facilities after the project finished. The evaluator suggested improving coordination with local organisations and more systematically checking on who was using previously installed water supply facilities and how well these were being maintained.
In Papua New Guinea, AID’s partner Anglicare PNG organised Adult Literacy and Life Skills classes for over 3,220 program participants during the period July 2016 to December 2018. Participants said they had begun reading shop signs, exchanging notes with their family members, engaging with their children’s schooling, sending and reading mobile phone text messages, and participating more actively in church services and other community events. Some of them even gained sufficient skills to proceed to formal education, open their own business, or gain waged employment. Whilst praising these achievements, the evaluator suggested that the program could be improved by increasing the level of support to teachers, including increasing training opportunities and increasing monitoring visits to schools.
In the Philippines, AID’s partner ECARE (Episcopal Community Action for Renewal and Empowerment) helped 44 community groups to build up the capital assets of their individual members as well as their whole-of-organization capital from July 2015-Dec 2018. Two community groups had become financially independent and stable to the point where they would no longer need to rely on external funding. ECARE’s program had also empowered women economically and gave them more opportunities to participate in public activities. At the same time, the evaluator noted challenges. One of these was the need for higher and more complex management knowledge, skills and processes as community groups grew in both size and capital.
ECARE also ran a Typhoon Haiyan Response in 2013-2017, which was evaluated in 2018.
In Kenya, between 2014 and 2017, AID’s partner ADSE (Anglican Development Services – Eastern) helped farmers build 15 sand dams and 12 farm ponds, providing them with a reliable year-round supply of water. ADSE’s program also installed water tanks in 16 schools, helped farmers construct 404 energy saving stoves (jikos) and planted more than 45,000 trees in school yards and communities. Farmers who were interviewed during AID’s evaluation spoke of incomes and food security being boosted through improved water supply for irrigation, through new agricultural and post-harvest technologies, and through improved access to credit. One of the key recommendations made by the evaluation was for ADSE to improve its data collection on aspects like productivity increases, unplanned impacts and delayed impacts from a project run in a previous year.
AID discusses evaluators’ recommendations with our partner organisations during planning of upcoming programs and many recommendations, like those related to monitoring and evaluation, are addressed in an ongoing way.
But it is not just the findings from program evaluations that have value. The actual evaluation process itself is an important tool for motivating program implementers, ranging from community members and our partner organisations to AID staff ourselves. People who put time and effort into running a program want to know that others care enough to examine the program and want to know what others felt was successful and unsuccessful. Visits by an evaluator are only a small part of what motivates program implementers to work hard, but communities and staff are usually proud to show off their work and indeed there is often a flurry of activity immediately before and after a visit by an evaluator!
The actual evaluation process itself can also be an important tool for building capacity of program implementers. A visit by an evaluator can build individuals’ public speaking skills and questions asked by the evaluator can give field staff and community members an insight into the priorities and ways of thinking of “outsiders”. AID-commissioned evaluators are also required to consciously play an educating role, explaining the evaluation process to local staff who accompany their travels and sharing their understandings of effective approaches to community development. When AID and evaluators visit partner organisations and communities, we constantly seek information and understandings from them, so it is only fair that they have opportunities to learn from evaluation processes.
Thus, evaluation reports are useful for donors to see how their money has been spent. Moreover, the reports and the evaluation process itself help AID, our partners, and the communities they work with in terms of motivation, ideas for future program planning and even new skills.
Dr Terry Russell
AID Program Effectiveness Officer
