Some of the challenges we have faced

Overall, the main NUSAF office has appreciated the impact evaluation and recognizes its importance for understanding the impact of the program. Difficulties though remain.

The biggest problem for the evaluation has been that some people in the districts do not appreciate the randomized design, which is understandable. While there has not been one standard solution to convincing the districts to cooperate, we have re-iterated with them how, faced with over-subscription, a randomized selection process is the most fair.

I do though agree with their common critique that, while we have more people eligible than we can afford, there are many groups that are more deserving of funding than others. This of course begs the question, what does more deserving mean?

For some groups, it means they are significantly more vulnerable. For instance, some groups are composed of single child mothers, or minority ethnic groups. It was decided that these groups should be kept out of the evaluation and given automatic funding.

For some groups though, it is not clear that the most vulnerable are the most deserving. I think it is an open question, given the way NUSAF is currently designed, that the most vulnerable will benefit as much as they could from the program (see my previous comments on the cross-cutting design).

There is also a programing problem that affects group selection. District officers are not responsible to the NUSAF main office, but instead to the district Chief Administrative Officer (CAO), who is a political appointment by the district leaders. There is thus both a political element to NUSAF, as well as an oversight blackhole for the main NUSAF office.

This setup has undoubtably lead to the numerous corruption problems in NUSAF, which can be found almost every day in the papers. Some district offices, rather than using their funds to help the people, are embezziling funds through "ghost projects" that do not actually contain real members, but instead the money is sent to phony accounts. When it gets real bad, and evidence has piled up against an office, NUSAF files formal charges. For instance, many people in the Acholiland offices have been sent to prison, including the Gulu district officer, who is sitting in a cell right now.

All of this then means that a randomized evaluation is not only more fair for participants, but it is likely the ONLY way to accurately measure the impact of NUSAF. The decision of who to fund, while officially an organized process, is actually quite messy at the district level, as I am finding out during this trip. The decision often targets the needy, but can also be motivated by politics and cronyism, and even a confused sense of the most needy.

While I believe we have successfully addressed all of the challenges we have had for this evaluation, I think these issues point to important program design issues that should be resolved in future NUSAF programs.