Global SW Midterm

This paper describes an instrument for the monitor- ing and evaluation of programs designed to improve the psychosocial well being of children: the Community Participatory Evaluation Tool (CPET). The community plays an important role when the evaluation tool is properly utilised. The rationale for use of the tool is explained, and its application in practice is illustrated with a case study.

Keywords: evaluation, community coping mechanisms, developmental tasks, monitor- ing

Evaluating community-oriented programs The effects of violent events on children are both well documented and well known. In 1991 Macksoud, Aber, Deregor & Raundelen developed the Child Behaviour Inventory (CBI), an instrument widely used to measure children’s reactions follow- ing violent events. These studies showed that children were profoundly and develop- mentally affected by war and violence, and they were falling victim to these environ- ments in increasing numbers. Following these discoveries, programs were designed and established to alleviate the impact of war trauma. Pynoos & Eth (1986) pointed out that, in

developing children, the effects of violence must be seen within a developmental per- spective. Boyden & Gibbs (1997) showed those developmental tasks, as well as the concept of childhood itself, varies markedly within different cultures. It is important to remember that the vio- lence that affects children does not only happen to them as individuals. Whole com- munities, countries and regions are affected. Furthermore, surveys based on the CBI revealed the ‘trauma’ children suffered was often not associated with one incident alone, but rather a result of the cumulative effects of incidents and deprivations that occur over the years in armed conflicts. In fact, Hans Keilson published a longitudinal study in 1979 of Dutch war orphans from World War II indicating the psychological suffering they faced was cumulative as well as sequential. Vanistendael (1996), and Anthony & Cohler (1987) studied the fac- tors that promote resilience in children. These studies confirmed that children cope best when there are consistently available adults in their lives, care is given by people they know, and who are able to help them develop their personalities and cognitive capacity. In 1996, Graça Machel led a study on the needs of children affected by armed conflict

3

Martha Bragin

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation

Martha Bragin

03_24_Bragin 07-03-2005 11:27 Pagina 3

for the Secretary General of the United Nations. Machel recommended best prac- tices for promoting psychological recovery and social integration for all war-affected children. These recommendations include special populations, such as child soldiers, landmine survivors and those who had experienced sexual and gender based vio- lence. The best practices include assuring that ‘all phases of emergency and recon- struction assistance programs take psy- chosocial considerations into account’, and that ‘rather than developing separate men- tal health programs, culturally relevant psy- chosocial programs be established within the context of programs for relief and devel- opment’ (Machel, 1996). Since these recom- mendations were accepted, many programs have attempted to ensure that intervention for children was provided according to the best practices outlined by Machel’s study. However, both practitioners and donors have confronted the difficulty of monitoring these programs and evaluating their effec- tiveness. A tool was needed that encapsu- lated community standards for children’s well being and proper development in a wide variety of circumstances, regardless of whether it is an urban area in Asia or Europe, a rural Internal Displaced Peoples (IDP) camp in Latin America, or in rural Sub-Saharan Africa. Such a measure had to reflect both the local standards for normal childhood development and the coping mechanisms that each community utilises to help children who are not meeting those standards. It was essential that this tool not impose either a set of beliefs about child- hood or a set of ideas about trauma and healing imported from other regions. Rather, it was essential to reflect local com- munity norms and values.

Identifying the effects of violence on a community’s young people This can be done in three steps. 1. The first step is developing an under- standing of daily routines under optimal circum- stances for boys and girls in the community, from morning to night. This is the back- drop for understanding how children usual- ly grow up in a particular community. This must be done separately for each socioeco- nomic grouping, as rural children, urban children, affluent, and poor will each have quite different routines. Changes in these basic routines since the violent event(s) must then be assessed to understand the ways in which basic life conditions have changed for children in the affected group. 2. The next step is to establish a baseline understanding, for each community, of what normal and usual growth and development looks like, age by age, starting at birth and ending with the age that has been deter- mined to represent adulthood. The devel- opmental levels must then be reviewed to ascertain how the current situation has or has not affected children in each age group and whether developmental milestones are being met. 3. Then existing community coping mechanisms for stressful situations should be identified, defined and understood so as to determine the extent to which those mechanisms are now functioning or need strengthening. One should also explore the potential need for new mechanisms to cope with new cir- cumstances.

General rules of collecting information Official information. Information regarding the extent and quality of basic services such as food, shelter, water, sanitation, health, and educational services may be available

4

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

03_24_Bragin 07-03-2005 11:27 Pagina 4

through official sources. It is important to talk with service providers to obtain their view of the effectiveness of the services and the degree to which all households are being assisted. In order to ascertain the major concerns of the community in times of crisis, however, it is also necessary to gather together groups of effected mothers, fathers, elders, and youth, as well as com- munity leaders and professionals. In the delicate area of child development, such community consultation methods are criti- cal. Community leaders in refugee camps and small communities should be contacted and the assessment process explained to them. Then, community meetings can be called with representative groups of people. In cities, representatives of schools, health care centres, and religious or community organ- isations should be contacted so that focus group meetings can be set up. The informa- tion is obtained here, as it is by market researchers, in focus group discussion. Participant’s information. Developmental data and information about community coping mechanisms should be obtained from com- munity members and young people within the focus groups. It is usually best to gather men, women, boys, and girls into separate focus groups to ensure that each feels free to speak. It is also crucial to seek out ‘notable persons’ for their view of commu- nity standards. Equally important is asking parents which community specialists they consult when children are sick or troubled. The views of these specialists must be included in the establishment of any base- line as well. Asking the same question of all of these sources supports the principle of ‘triangulation’ or relying on at least three sources, which in turn supports validity in Participatory Rural Appraisal. In refugee and displaced persons camps, it

is relatively easy to obtain a random sample of households in groups of twenty-five. In rural villages and urban communities, how- ever, a random sample may be more diffi- cult to acquire. Convenience samples often have to suffice, as volunteers may have dif- ficulty leaving daily tasks in order to attend a meeting. Tools. A structured interview should be used to guide the focus group discussions so that all necessary information is obtained, and so the information requested can be rela- tively uniform from location to location. In this way, differences can be more clearly understood. The interview should be set up and con- ducted using the main steps from the pre- ceding paragraph: 1. Step one: Establishing the common view of normative behaviour. Through the struc- tured interview, the researchers should determine what activities (formal and non- formal) youngsters are normally involved in at different ages and during different phases of their development. Participants should be encouraged to reach an agree- ment among them as to what the normal range of responses to these activities might include, and the results should be docu- mented in chart form. 2. Step two: Normal and usual growth. It is important to determine what specific devel- opmental stages a child passes through, if any. Researchers should also establish how these stages are marked, what the child learns, how the child must be cared for and by whom, and how he or she can be expect- ed to behave. 3. Step three: Community methods for dealing with stressful situations. Normal responses that families use to manage the ups and downs of daily life should be iden- tified. Independent groupings along lines of gender or age should be established as well.

5

Martha Bragin

03_24_Bragin 07-03-2005 11:27 Pagina 5

Do groups of mothers or fathers get togeth- er independently (e.g. at the barber shop or for a coffee ceremony)? Do young people have such gatherings (youth union activi- ties, coming of age rituals, etc.)? Are there community-wide cultural rituals? The community’s normal coping strategies for stressful situations, such as death, ill- ness, and natural disaster should also be identified. How have violent situations been dealt with in the past? Are these tradi- tional methods still in place? The need for new methods of coping due to current conditions should be determined, and if needed, the people who might be involved in developing them should be identified. This last area of inquiry is very important, as it may open the door to opportunities for children who might otherwise have been excluded from these community processes, for example those with disabilities. New educational programs for children might be established, or clubs and youth organisa- tions for the promotion of peace and leisure activities might be developed. These organ- isations or programs may not have existed in the past, yet they could allow a more nor- mal life to continue, even during an emer- gency.

Collecting information from children In order to give children a voice in the heal- ing process, they should be asked to describe their daily activities from morning until night prior to the war, if they can remember. They should also be asked to describe festivals, holidays, and the basic structures of community life. Questions about the process necessary to become a respected man or woman in their commu- nity, and steps required to reach that goal are other ways to understand children’s per-

spectives. Researchers should also inquire about young people’s hopes for the future prior to the war or extreme acts of violence. They should be asked about the mecha- nisms used in the community to cope with stressful events and whether or not they used to participate in them. It is also impor- tant to learn in the course of the interview whether the young people in the community were accustomed to participating in youth organisations or sports leagues in previous times and whether those groups could be restored or reformulated with peaceful objectives. Children should then be asked how things are different under current conditions, by once again going through a typical day. Interviewers should question them how the tasks necessary to become a respected man or woman could be accomplished under the current circumstances. Goals and plans for the future should be determined as well. Furthermore, young people should identify key issues in the community that need attention and rank those issues in order of importance. They should be asked what the barriers are to resolving those issues and who the protagonists should be in cre- ating resolution. Finally, they should be asked what concrete steps they could or would like to take in ameliorating one or more of the problems they listed.

Collecting information from children’s war experiences Some researchers have found that it is the internal and cultural meanings of any experience, rather than the type of experi- ence, that cause developmental difficulties for children exposed to violence (Boyden & Gibbs, 1995; Nikapota & Samarasinghe, 1990). But others have noted that the most severe experiences — such as prolonged periods of physical and mental torture, or

6

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

03_24_Bragin 07-03-2005 11:27 Pagina 6

death camps — contributes difficulties to everyone, regardless of other factors (Hovens, Falger, Op den Velde & Schouten, 1992). Therefore, in order to assist the adult community in determining whether special remedies are necessary for the new situation, it is useful to know what a popu- lation of children has witnessed and direct- ly experienced. However, as Machel (1996) points out, it is dangerous to ask children to confide terri- ble experiences to strangers for the purpose of research, even if the long-range goal is to assist them. For that reason, it is important that programs be developed based on the reported behavioural and developmental difficulties of children in their specific com- munity, rather than through interviews intending to determine children’s individual reactions to specific events. When it is deemed absolutely necessary to inquire about war experiences, it should be done carefully and generally. After asking each focus group about life in the commun- ity where the children were raised, simply inquire as to what caused them to leave. When atrocities are listed, ask whether the community had been warned that these things might happen or whether they had been experienced or observed by anyone in the group. It is important not to coax any- one to answer. If even one person tells the story of an atrocity experienced, make sure that there is time for further discussion after the group ends. Ask the group how they feel they have been affected by the experi- ences described. Inquire if they would like any particular assistance in overcoming any of the painful consequences or emotions that are reported, and make sure that there is actual follow-up with these persons on the specific remedies for which they ask. With careful investigation atrocities can be usefully shared in the presence of strangers.

Some experiences however, such as homo- sexual rape and sexual assault, carry a spe- cial social taboo in many societies. It may be dangerous to discuss them with out- siders, people unfamiliar to the victims, or even with those close at hand. For example, in Pakistan, female rape victims are com- monly put into prison, unless four men have observed the rape and are willing to swear that the girl was a victim of violence. In many communities in the United States, survivors of homosexual rape are often assumed by families to be tainted for life. Therefore, to avoid inflicting irreparable damage, it is important that programs put into place mechanisms that can assist chil- dren whose development has been affected before learning the details of their experi- ence. That way, those experiences can be appropriately revealed to trusted persons within the community who have an ongo- ing relationship with them and are socially and emotionally prepared to help. It is essential that the emotional status of chil- dren never be compromised in the name of monitoring, evaluation, or accurate research.

Compiling information into useful data One way to compile the information gath- ered is by creating tables that follow the main lines of the investigation. 1. The first one is a table in which ‘normal’ development is contrasted with the current situation (see Table 1, Developmental mile- stones). ‘Normal standards’ represents the range of responses agreed upon by the respondents. 2. Similarly, community coping mecha- nisms can be placed on a grid to see whether or not they are in place, need strengthening, or are non-existent (Table 2, Community coping mechanisms).

7

Martha Bragin

03_24_Bragin 07-03-2005 11:27 Pagina 7

3. A third kind of table can contain the problem-solution matrix created by the youth and an outline of their proposed par- ticipation (Table 3) These sample tables were created on differ- ent continents within different social con- texts. Youth are asked to rank the major problems of their community, to describe solutions as they see them, who should resolve the problems, and what they can do to partici- pate in solving them.

Using the tables The success of a program can be measured by the degree to which the conditions for

normal developmental milestones are restored, community coping mechanisms are reinstated, and new mechanisms are put in place to meet the contemporary needs identified by the community. Involving communities and young people in re-estab- lishing the conditions for children’s healthy growth and development helps to insure that such interventions are sustainable long after outside interest and attention has faded. Sustainability is established by not- ing whether these conditions remain in place after one, five, and ten years.

Implementing the CPET Implementation of any strategy during

8

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Table 1 Developmental milestones

Age of child

Normal standards War related changes

0 – 2 years

Children learn that family loves them best of all. They learn to know their mother. Learn to eat and grow.

Nervous mothers make babies nervous! Hungry mothers don’t make enough milk.

2 - 5 years

Children go off and follow older children as new babies are born. They learn about the world and how to grow up. They play and make up stories about the world.

No opportunity to practice independence. Children tied to posts instead of allowed to explore with siblings. Play is always war and funerals.

6 –12 years

We want them to go to school and learn. When they are too poor, they follow their parents and learn their parents work

Children don’t play or go to school/ only stay near mom helping with chores/ boys don’t learn fathers’ livelihood. They begin to talk of revenge and boys fight a lot.

12 – 15 years

Those who can afford it should go to secondary schools and become our leaders. Should learn to be men and women in our traditional way. Prepare for marriage,

public, how to respect all people, participate in community activities such as fire prevention, building, helping poor, family business and farm.

Lack of opportunity to learn or earn. Cannot prepare for future marriage. Feel useless and want to go and join militias.

membership in community, learn to speak in

03_24_Bragin 07-03-2005 11:27 Pagina 8

emergencies is always something of an art, with flexibility as the key to success. However, a step-by-step implementation guideline may be helpful. The following is a step-by-step process that includes prepara- tion, taking a baseline, carrying out the pro- gram for relief and development, and post- tests.

Preparation Involving the community. Meet with communi- ty opinion leaders and officials to discuss child protection, development issues, the

purpose of the monitoring and evaluating tools. Form a community committee to oversee child protection and development activities1. Get a random sample of participants for the focus groups. Number. Limit the number of adults or teens to 15. For children 6-11 years of age, 10 children should be the maximum number in any focus group. Time allocation. Children’s groups take about 45 minutes; youth and men about 11/2 hours; and mothers about 2 hours.

9

Martha Bragin

Table 2 Community Coping Mechanisms

In prior community

Festivals for births, weddings and important holidays

Not occurring

Collective mourning ceremonies: feast in honour of joining with ancestors little money

Consultation with traditional healers In place: needs strengthening and integration with modern health practices

Music and dancing at festivals Not occurring

Story telling and beating the drum Not occurring

Children’s games Need strengthening

Assistance to disabled and orphans by community

Coming of age ceremonies: bush training of young people

Not occurring: if replaced may need monitoring for health and human rights standards

Mothers work together and share child rearing advice

Needs strengthening

School Stopped completely

In camp: in place/needs strengthening/not occurring

Needs strengthening: too many deaths, too

Need strengthening/monitoring

03_24_Bragin 07-03-2005 11:27 Pagina 9

Participant selection. In cities, and semi-urban areas, partner non-governmental organisa- tions (NGOs) can select participants from the communities with whom they work. In rural areas, or in IDP camps, after meeting with village officials and leaders, house- holds can be selected for participation. Obviously, there are technical differences that must be worked out on the ground depending on the culture, educational level, and level of development of the region in which this tool is being used.

In regions where the concept of random sampling is not familiar, it is important to include one group for all of those not select- ed for the survey. This ensures participation in group discussions with the monitoring and evaluation team, so that fears are not aroused regarding exclusion. In Afgha- nistan, for example, I often had large group meetings in IDP camps, where we read the questions aloud to large groups of people who then responded. At the end of the day, these ‘unscientific’ meetings did give me a

10

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Table 3 Problem solving solutions

Boys Problem How to solve Who should solve What we can do

Hunger With work People should give us work Work if anyone has work for us

Drought Bore holes, dig wells, irrigation, More rain

People who know how should do the irrigation/educate people/ God could give rain

We will pray

Work Government finds work for us

Government We will request

Poverty Government should help us Government/other aid We will pray

Clothing Someone has to give us work We will work

Girls Problem How to solve Who should solve What can we do

Fighting Unity Everybody should solve – including leaders

Study the holy Koran and pray

Illiteracy Peace and school teachers Government should pay teacher

When we are older, we will help to teach

Poverty Peace - gives work, jobs UN should disarm everybody Study the holy Koran and pray

Destruction Reconstruction Government and UN We can build

Work/buy clothes

03_24_Bragin 07-03-2005 11:27 Pagina 10

broader sense of the views of the popula- tion overall. Separate focus groups should be organised for: mothers; fathers and community eld- ers; female adolescents; male adolescents; girls; and boys. If relevant, groups can be further divided into more/less educated, rural/urban, eth- nicity, and/or by those from areas more/less affected by the conflict. Do not separate orphaned, disabled or other vulnerable chil- dren but include them in the appropriate age and gender group. Random sampling should assure that they would appear in proportion to their percentage in the popu- lation. However if randomisation is not pos- sible, or if these groups are segregated in some special way, simply have an addition- al random selection and then include them in the appropriate groups.

Taking a baseline A baseline of community norms for chil- dren’s development both before and dur- ing the emergency can be obtained by using the following interview guides (for mothers and primary caretakers, for fathers and community elders, and for children and youth). Interview guide with mothers/primary care- takers and others who are intimately involved with children’s growth and devel- opment Instruction: Think about a time when life was good, when there was no war and you were at home, with food to eat, water to drink and your family was around you. 1. What is the first thing that happens when a new baby is born? 2. How are infants cared for and by whom? What is the most important thing for a child to learn in the first months of life? 3. At what age does a child walk? What changes at that time?

4. At what age can a child walk unaccom- panied? 5. At what age does a child first talk? Say ‘no-no?’ 6. At what age can one begin to teach a child right from wrong? How? 7. At what age does a child know that he is a boy (like father), or a girl (like mother)? 8. What is a child like from 3 to 6 years old? 9. At what age can a child begin to help the parent? 10. Which parent and with what chores? 11. At what age can a child begin to learn the important rules of the community? 12. How are children taught these things and by whom? 13. At what age can a child begin school or work? What makes this a good age? 14. What is a child like from 7 to 12 years old? Instruction: Do we all agree that children [below adolescence] [whose bodies are not yet developed] see the world normally as their parents and teach- ers describe it? 15. At what age does a child begin to see the world differently from what is told to him/her? (Think back on your own child- hood) 16. At what age can a person speak of these opinions to the family? In the community? Participate in community councils or meetings? 17. At what age should a youngster carry a weapon? Defend the family? 18. At what age is a person considered to be a proper adult in this community? 19. List the characteristics that make a respected man or woman in this communi- ty. How do boys and girls learn these things? Are there special ceremonies associ- ated with the acquisition of this knowledge? At what age is one expected to behave in this way? 20. Are there any special danger signs that indicate that things are not going well in this

11

Martha Bragin

03_24_Bragin 07-03-2005 11:27 Pagina 11

regard? (For girls? For boys?) What do you do and with whom do you consult if things are not going well? 21. At what age can a person get married? Who makes that decision? 22. Instruction: Has anything changed? What is different under current conditions? What is/is not in place now? (Go through above list of questions 1-21 again)

Interview guide with fathers/community elders Instruction: Think about how it was before the violence, killing, checkpoints, disappearances, etc. How were children raised in the community? 1. What do you believe is most important for a father to teach his children? 2. At what age might a person first have independent thoughts? 3. At what age might a person be permitted to express such thoughts? 4. How and at what age are people taught right from wrong? What of a young person who has difficulty learning these differences? 5. What is/was your view of the most important, necessary things to learn and accomplish in becoming a good man? A good woman? 6. At what age should a young man marry? A young woman? 7. At what age should a young person carry a weapon? 8. How were decisions made in the community? 9. How were disputes resolved in the com- munity? 10. Were there special ceremonies for the return of soldiers following war? 11. Were there any methods by which a per- son who had acted wrongly could make restitution or be restored to the communi- ty? Were they different for children and adults? 12. What were the customs of the commu- nity regarding births and marriages?

13. What were the customs of the commu- nity regarding famine, loss of income, vio- lence, or another major difficulty? Instruction: Has anything changed? What is different under current conditions? What is/is not in place now? (Go through above list of questions again)

Interview guide with children/youth 1. Describe the place that you came from. 2. What is it like there? (Talk about animals, weather, school, religious institutions, traffic if any, and other descriptive factors.) 3. What did a girl do all day from morning until night? 4. What did a boy do all day from morning to night? (Be sure to probe for the following: Do they ever help their parents with chores or work? Which chores, when? When they go to sleep at night, does anyone tell them a story? Do they have dreams?) 5. How does a kid have fun here in this place? 6. What are the celebrations and festivals in the community? Are they the same as before? If not, how are they different? 7. Who participates in them? 8. What are these celebrations like? Are they the same as before? How are they different? 9. Did young people participate in any group activities? (Formal youth organiza- tions? Sports Clubs? Other?) 10. What is the best thing about this place? 11. Is there anything that you don’t like? Is there anything dangerous or frightening for kids here? 12. What is the worst thing about this place? 13. What are the necessary qualities of being a grown man? Woman? 14. Are there necessary things that a person must do to gain that status? Instruction: Close your eyes —- while your eyes are closed, you will grow big and bigger until you are an adult of (?) years (have children give age of adulthood) 15. Tell me about your life. (What do you

12

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

03_24_Bragin 07-03-2005 11:27 Pagina 12

do for a living? Family? Etc?) How will you have prepared for this? Instruction: I will clap my hands, you will return to your own age. 16. Do you have all the conditions neces- sary to grow up to accomplish these things? If not, why not? What is needed?

Because community norms and values vary based on class, ethnicity and life circum- stance, it is critical that the same questions on these matters be put to representative groups from a variety of backgrounds. Doing so can be a somewhat lengthy process. Therefore, a team should do the assessment, and as much as possible, com- munity members should be involved in tab- ulating the results.

Interview guide: Community coping mechanisms The answers can be recorded on a chart: what is in place/what needs strengthening/what is no longer in place. It should then be rechecked following the project to determine improvement. 1. What were the celebrations and festivals in the community? Who participated? What were these celebrations like in good times? 2. What did you do if you had a moment free

from household work? Did you have occa- sion to socialize with other women during the course of the workday? Where and how? 3. Were there women’s organizations or lend- ing groups that women participated in? 4. When your children were sick or had diffi- culties, what did you do, who did you consult? 5. What was the custom when someone died? 6. What was the custom when there was a natural disaster or other community prob- lem? 7. If someone’s child did not behave properly or was not growing up properly, what did you do? Who did you consult on these occasions? 8. If a mother was under stress or having difficulty caring for her children, who would help her? Was there some way she could feel better? Instruction: Has anything changed? What is dif- ferent under current conditions? What is/is not in place now? (Go through above list of questions again).

Get young people to do a community prob- lem solution matrix, by asking them about the five biggest problems affecting their community and filling in the following problem matrix. (Table 4)

13

Martha Bragin

Table 4

Problem matrix

Problem 1 Problem 2 Problem 3 Problem 4 Problem 5

Who should solve it?

How should it be solved?

What can you do as a child in this community?

03_24_Bragin 07-03-2005 11:27 Pagina 13

Post tests After the program (which was based on the principle that it should connect with com- munity coping mechanisms) the following post tests can be carried out. 1. The focus groups with mothers and pri- mary caretakers, fathers and community elders, children and youth can be repeated in order to determine the condition of the children. Are children beginning to meet pre-war developmental milestones? 2. The focus groups on community coping skills can be repeated, in order to determine which community coping mechanisms have been restored and which new mechanisms have been added? 3. The children and young people can be questioned once more: are they taking action according to the matrix?

Case Study: Eritrea during the Border War In Eritrea, during the border war, the Ministry of Labour and Human Welfare and UNICEF commissioned an assessment on the affects of the war on children. Following that assessment, UNICEF, the ministry and local community organisations were to take action in support of children. They needed a program that was sustainable by and for the activist communities, as there were few funds available for such programs. The work was complicated by the diversity of this small country, which is home to nine distinct ethno-linguistic groups who live in climates ranging from cold mountainous goat herding areas to one of the hottest low- lying port regions of the world. The community had been displaced to a camp in the mountains away from their home village following aerial bombardment and the invasion of neighbouring troops. The troops mostly frightened people, and pressed them to leave the village at gun-

point, although there were some allegations of atrocities. The landscape was rocky and a bit barren. The government provided a rudimentary health clinic, food aid with assistance from the International Com- mittee Red Cross (ICRC), and security through mobilisation of local veterans of the recent civil war.

The pre-test Getting started. The first step was putting together a small team of colleagues who knew the local languages and were com- fortable with Participatory Rural Assessment techniques. This team then travelled to the war affected districts and discussed the plan with the necessary offi- cials. The purpose of this discussion was to deflect concerns about the information we were interested in gathering, why we want- ed it, and how we hoped it would be help- ful. Listening is very important at this stage. Officials rarely care about child develop- ment information and this group was no exception. This can be advantageous, as they will therefore not want to influence the information collected. Nor will they block its dissemination or use, as they might material considered of a more sensitive nature. They informed us the main prob- lem was increasing the quantity and quality of food. We were assured the children would be fine if the enemy could be routed and the harvest completed. They them- selves were planning to bring tractors to help the harvest for some of the more vul- nerable of the displaced people. They then assured us that our working in the camp would be no problem and that we could go the next day. It is of course important to get all of the actors in the camp leadership together for an introductory meeting. There, one learns about the reality of day-to-day practical

14

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

03_24_Bragin 07-03-2005 11:27 Pagina 14

camp conditions and problems as leader- ship see them. It also serves to inform everyone of what will and will not happen during the assessment and as a result of its findings. In Eritrea, in 1999, the government had preferred to provide all services through the popular organisations that had operated during its recent war for independence. The usual plethora of non-governmental organisations was absent. Involving the community. We arrived in a chilly rain. We entered the camp and met with camp management, leadership and repre- sentatives of all sectors and organisations present in the camp including the sole relief organisation called E RREC (Eritrean Relief and Rehabilitation Commission). ERREC was mainly staffed by aging vol- unteer activists from ‘the old days,’ and were highly respected by the local popula- tion. They were very helpful in ‘selling’ the program to others in the camp, as yet another form of self-reliance. We were care- ful to tell them that the program was not associated with any material goods or any of the aid that people wanted so badly. Once this understanding was attained we were able to help participants focus on their children’s well being. We found that adults really wanted to talk about their children’s upbringing and their worries that war was harming their chil- dren. The children wanted to talk about their lives to interested adults. Participant selection. We explained randomisa- tion and then made a schedule of which tents would be represented. We ended with a group of all those who wanted to speak who hadn’t had a turn, so that no one could worry that one group was favoured over the others. We took a woman from every 10th tent, a man from every 9th, a teen boy from every 8th, a teen girl from every 7th

a younger boy from every 6th and a young girl from every 5th. Those in tents 1 to 5 could send representatives to the final, ‘talk about it’ meeting. In order to decide whom to see first we learned the schedule of the various camp members. Women and girls had the most work to do, as they were responsible for childcare, food preparation, and cleanliness. The girls were second, helping their moth- ers and being responsible for the collecting of water and firewood. Some boys helped with firewood, while men were largely idle. Many of them returned to the areas from which they had been displaced to attempt to plough the village land. The young men, from 18 to 22 were in the army. The men who were in the camp were either involved in village administration, teaching school, or considered too old to risk the dangerous walk over the mountains to the village land. Focus group discussions. The team should divide into pairs for the focus group discus- sions. Each pair included a team member as well as a community leader who would be involved with the monitoring and evalua- tion, so that they could hear as we collected the actual responses of the groups. They continuously expressed surprise at the time and attention groups gave to discussing the details of children’s development. The moth- ers in particular were eager to give detailed answers and to discuss them. We gave the children paper and markers and asked them to draw while they were waiting their turn for a group. We let the youth leaders keep these materials for use so that some children’s enrichment could begin immediately after the assessment period. Summary of the results of the pre-test. The pre- test gave us some interesting results (tables 5-9). Parents reported that children in each age group showed atypical behaviours

15

Martha Bragin

03_24_Bragin 07-03-2005 11:27 Pagina 15

16

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Table 5 Normal child development

Age Normal milestones

0-2 years Carried by mom. Eat, sleep, learn to talk and walk.

2-5 years Go with other children, learn to play, know animals and plants, must run around to be healthy and strong

6-12 years Those who can go to school. Girls fetch water and wood, care for little children: boys go with the animals in the field. Learn to follow rules, and behave. Learn about animals and plants.

12-15 years Those who go to school go to the city for secondary. Continue helping parents. Girls learn to drum and dance, to make coffee. Boys learn the paths around the caves and mountains, how to travel and come back, learn to build furniture, house, all the things you need for household. Learn proper behaviour in community and at home.

15-18 years In our day we joined the struggle. Our government wants us now to send the children to the youth groups. In the youth groups they learn about government and how to build the country. They must also prepare to be married and have their own children. Learn about proper roles of man and woman in marriage. Of course those who can afford it should continue education.

Table 6 War related changes observed in children

Changes

0-2 years Don’t feed well, irritable

2-5 years No opportunity to practice independence in the camp --- they must stay in the tent or right near with mother/ night terrors, fearful of noises

6-12 years concentrate at school/ some school interrupted/ run out of camp when frightened/ get lost in the mountains/ fearful of noise

12-15 years Nightmares, sleepwalking, night terrors, sense of powerlessness, preoccupation with revenge/ no facilities for continuing education/ disobedient

15-18 years Sense of powerlessness and fear/ lack of opportunity to complete schooling leading to anger and resentment ---girls are depressed by their workload and boys by their lack of appropriate activity/ feelings of hopelessness about future/don’t know how to change situation (different from parents who participated in national liberation struggle and felt an important part of solving problems

Nightmares, sleepwalking, night terrors, headaches, stomach aches, can’t

03_24_Bragin 07-03-2005 11:27 Pagina 16

based on their age and stage of develop- ment. They pointed out that babies did not feed well, toddlers did not feel free to play, children who should have outgrown them suffered from nightmares and night terrors, and that adolescents continued to show these symptoms. In addition, due to suffer- ing from depression and powerlessness, they feel unable to take up the task of build-

ing the new society. Many of these symp- toms are derived from the limitations in activities of children in wartime, loss of opportunities for school and play, and oth- ers from the lack of coping mechanisms available to mothers and others who care for children. These conclusions led us to create pro- grams to address the root causes of the

17

Martha Bragin

Table 7 Community Coping Mechanisms

In prior community In camp

Festivals for births, weddings and important holidays Not occurring

Collective mourning ceremonies: last for 12 days and involve the entire community including expressions of grief and concrete assistance

Needs strengthening

Consultation with priests, utilization of holy water and amulets In place

Consultation with elderly and wise persons In place

Music and dancing at home Not occurring

Story telling and beating the drum Not occurring

Children’s games Need strengthening

Assistance to disabled by community Need strengthening/ monitoring

Participation in ‘the struggle’ for independence and the building of the new state

Needs strengthening

Mothers participate in coffee ceremony/share worries and concerns Needs strengthening

Community members participate in ‘the struggle’ for freedom and independence

Needs strengthening

School Emergency school not available

03_24_Bragin 07-03-2005 11:27 Pagina 17

symptoms programmatically, rather than the symptoms themselves. This was espe- cially practical as treatment of symptoms requires specialists available in each camp over time; correcting many of the root caus- es of those symptoms was practical, inex- pensive, and it worked!

The program The funding provided for this program was short term, as it was meant to alleviate con- ditions for war affected children and ado- lescents while negotiations to settle the con- flict continued. The hope was that residents would soon be able to return to their homes. Therefore, long term interventions

actually designed to improve children’s age appropriate functioning were not utilised. Rather interventions were chosen that would shore up and restore community coping mechanisms in order to foster fami- ly resilience. No program was to be put into place that would not be sustainable should the war end and families return home in the short term. The assessment team called the community leaders together. They brought representa- tives of the mothers, fathers, teachers, reli- gious leaders, business and health care pro- vides, as well as representatives of the youth association to hear a presentation of the results of the assessment. Most immediately

18

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Table 8 Daily activities boys

In village In camp

In school Out of school All boys in camp

Wash Wash Wash

Pray Pray Pray

Breakfast Breakfast Breakfast

School Herding Collect fuel/ help sisters with water

Lunch Lunch Lunch

Study Play (while with the animals and other boy herders)

Rest

Work/help father with chores Bring the herd home Idle

Dinner Dinner Dinner

Stories/sleep Stories/ sleep Sleep

03_24_Bragin 07-03-2005 11:27 Pagina 18

19

Martha Bragin

Table 9 Daily activities/girls

In village In camp

In school Out of school All girls in camp

Wash Wash Start fetching water and fuel before sunup

Breakfast Breakfast Start fire/heat water

Clean after breakfast Clean after breakfast Wash/help siblings wash Breakfast

School Bake enjera Bake enjera

Lunch Prepare other food Prepare sauce

Clean house Lunch Lunch

Study Wash clothes Make coffee for mother

Dinner Clean house Wash clothes

Join mother and other women/sing and beat the drum

Fetch fuel and water Take care of siblings/heat supper

Stories to smaller children with older women

Take care of siblings/help with supper

Sleep (many difficulties—fear of snakes and scorpions/bad dreams)

Sleep Join mother and other women/sing and beat the drum

Tell stories to smaller children with older women

Sleep

03_24_Bragin 07-03-2005 11:27 Pagina 19

wanted long term development assistance, and a date was made for a separate meeting with the appropriate contacts. What the community agreed, interestingly enough, was that they sought a minimum of resources to be spent on these projects, as that would leave more available for post war development. They then sat down to agree on what they could do, as parents, teachers, and young people, to improve family life in the camp. A community child protection committee was formed to monitor and evaluate the progress of program implementation as well as the results. This committee was to work without material incentives, as it was felt that material incentives would skew the results of their work. However, in part this was possible in Eritrea because of the strong tradition of participating in commu- nity activities as volunteers - this was con- sidered part of their contribution to ‘the struggle.’ The committee, which consisted of two rep- resentatives of mothers, fathers, teen boys, teen girls, younger boys, younger girls, a male and female teacher, a doctor, a health worker, and representatives of the youth union and camp government, was estab- lished. They received training and supervi- sion from the Ministry of Labour and Social Affairs. Activity 1: Emergency Schools. Emergency (tent based) schools were set up for all seven year olds and all school going primary aged chil- dren2. Activity 2: Youth association to engage young peo- ple. Young people were put to work camou- flaging tents. Storytelling contests and drumming contests were organized for girls. Prizes were allotted for grandmothers who trained young girls to participate. Footballs were distributed and matches were organized by adolescents for younger

children. Art supplies were distributed, and art activities and games were organized for non-school going children. Activity 3: Support for mothers at home. Women’s association would support coffee ceremonies by providing free coffee. Older boys in the youth association would involve young children in afternoon activities while women and older girls participated in coffee ceremonies. Activity 4: Support for mourning and help for wid- ows and disabled. A council of elder men was established to arrange for memorials. The council was given supplies that they could use to help orphans, widows, the sick and disabled. Staff from the Ministry of Labour and Social Affairs, along with a representative of the Eritrean Relief and Rehabilitation Commission (ERREC) visited weekly to provide training, observe, advise and assess the progress of the program. The problem of the water points remained intractable. Girls were up before dawn to collect fuel and water due to both the scarci- ty of and intense competition for those commodities. UNICEF had provided the existing water points, but felt that providing more was inadvisable as it was hoped that the population would go home in just a few months. The feeling was water points were best provided in the villages where people lived permanently and where they were critical for the prevention of disease.

The evaluation The child protection committee was in charge of ongoing support for the program as well as evaluation. The program was designed to target the specific coping mech- anisms that the community named as not in place. Therefore, it is not surprising that these mechanisms were those addressed by the program and in place at its conclusion.

20

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

03_24_Bragin 07-03-2005 11:27 Pagina 20

21

Martha Bragin

Table 10 Community coping mechanisms

Festivals for births, weddings and important holidays

Need strengthening/ no weddings

Collective mourning ceremonies: last for 12 days and involve the entire community including expressions of grief and concrete assistance

In place

Consultation with priests, utilisation of holy water and amulets

In place

Consultation with elderly and wise persons In place

Music and dancing at home In place

Story telling and beating the drum In place

Children’s games Strengthened

Assistance to disabled by community Strengthened

Participation in ‘the struggle’ for independence and the building of the new state

Participation up

Mothers participate in coffee ceremony/share worries and concerns

In place

School Emergency school available

In prior community In camp after the program

Table 11 Daily activities/boys

In school Out of school All boys in camp

Wash Wash Wash

Pray Pray Pray

Breakfast Breakfast Breakfast

School Herding Collect fuel/ help sisters with water/ some attend school

Lunch Lunch Lunch

Study Play (while with the animals and other boy herders)

Participate in youth activities

In village In camp after the program

Work/help father with chores Bring the herd home Play

Dinner Dinner Dinner

Stories/sleep Stories/ sleep Stories/sleep

03_24_Bragin 07-03-2005 11:27 Pagina 21

The child protection committee was able to collect a different random sample of community participants, and to revisit the questions. Table 10-13 reflect their responses. It may be that the positive results came from the overall sense of excitement and improved well being that the activities cre- ated. Such a qualitative study does not give a measure of causality. By using the CPET the camp community was able to feel that the programs for children were theirs, and that they were able to find ways

to improve their children’s well being even in time of war and displacement.

Limitations It should be noted that we did not replicate the activity matrix because, being short term, the activities open to adolescents for personal action were created by the pro- gram, and adolescent participants were involved in the activities that the program had created. It would have been very inter- esting to see whether, following the pro- gram’s ending, these youth continued to

22

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Table 12 Daily activities/girls

In school Out of school All girls in camp

Wash Wash Start fetching water and fuel before sunup

Breakfast Breakfast Start fire/heat water

Clean after breakfast Clean after breakfast Wash/help siblings wash

Breakfast

School Bake enjera School or youth union activities

Lunch Prepare other food Same as above

Clean house Lunch Lunch

Study Wash clothes Make coffee for mother

Dinner Clean house Wash clothes

Join mother and other women/sing and beat the drum

Fetch fuel and water Take care of siblings/heat supper

Stories to smaller children with older women

Take care of siblings/help with supper

Stories with older women for siblings

Sleep Join mother and other women/sing and beat the drum

Join mother and other women sing and beat the drum

Stories to smaller children with older women

Sleep

Sleep

In village In camp after program

03_24_Bragin 07-03-2005 11:27 Pagina 22

create channels of activity for themselves. It would have been highly recommended to learn whether the programs produced last- ing success in restoring children’s develop- mental trajectory over time. However, fund- ing was not available to continue the pro- gram. Therefore the recommended follow- up post tests were not conducted.

Conclusion Our short-term intervention, improving the quality of children’s daily lives, restoring community coping mechanisms that had served well over time, and creating new ones where they were necessary, appeared to have proved helpful. The community members reported that their babies were feeding better as mothers were more relaxed and that young children utilized the opportunity to play games appropriate to their age group and became less fearful of being away from mother for short times. School aged children continued to have night-

mares but did have fewer somatic complaints and were able to concentrate at school. Adolescents reported feeling more hopeful as they participated in community activities. More important, it showed that this simple tool for community based monitoring and evaluation of a psychosocial program could be utilized, with little funding and under difficult circumstances. It is hoped that this model will be replicated in other circumstances with the results made available, so that the viability of this method of monitoring and evaluation can be reported in other circumstances.

References Anthony, E. & Cohler C. (Eds) (1987). The

Invulnerable Child. New York: Guilford.

Boyden, J. & Gibbs, S. (1997). Children of War:

Responses to Psychosocial Distress in Cambodia. No.

97.2.Geneva: The United Nations Research

Institute for Social Development.

23

Martha Bragin

Table 13 War related changes in children after the program

Age of child Observations

0 – 2 years As mothers feel better, babies reported feeding and sleeping better.

2 years - 5 years Independence and fear of noises still a problem. Sleep rituals, stories singing help with sleep

6 years - 12 years Nightmares, sleepwalking, night terrors, headaches and stomach aches are less frequent (but continue) as they are more positively occupied and moms are calmer. No longer run out of the camp. Those in school concentrate better, all say they enjoy the games that they play – play helper rather than soldier

12 years - 15 years Nightmares, sleepwalking, night terrors reported to be less frequent. However more are attending school and playing and as they do this they report feeling more hopeful and positive, say they don’t think consciously about revenge and violence – play sports

15 years - 18 years Young people and their parents report a greater sense of hope about their future, and a sense that they are part of an important struggle.

03_24_Bragin 07-03-2005 11:27 Pagina 23

Bragin, M. (2001). Evaluating Psychosocial

Programs For Children Affected By Armed

Conflict: A Community Based Approach. Mind

and Human Interaction, (12), 289-304.

Bragin, M., Tackle T., and Zerai, W. (1999).

Children in Eritrea Affected by the Border Conflict: A

Psychosocial Needs Assessment. Asmara: MOLHW

and UNICEF.

Eth, S., & Pynoos, R. (Eds.). (l985). Post traumatic

stress disorder in children. Washington, DC:

American Psychiatric Press.

Garbarino, J., Kostelny, K., & Dubrow, N. (1991).

No place to be a child: growing up in a war zone.

Lexington, MA: Lexington.

Hovens, J., Falger, P., Op den Velde, W., &

Schouten, E. (l992). Occurrence of current

posttraumatic stress disorder among Dutch

World War II Resistance veterans according to

the SCID. Journal of Anxiety Disorder, 6, 147-157.

Keilson, H. (1992). Sequential traumatization in chil-

dren: A clinical and statistical follow-up study of the

fate of Jewish war orphans in the Netherlands.

Jerusalem: Magnes Press.

Kleber, R., Figley, C., & Gersons, B. (Eds.). (1995).

Beyond trauma: Societal and cultural dynamics. New

York and London: Plenum Press

Machel, G. (1996). Impact of Armed Conflict on

Children. New York: UNICEF/UNDPI.

Macksoud, M. (1993). Helping Children Cope With

The Stresses Of War: A Manual For Parents And

Teachers. New York: UNICEF.

Macksoud, M., Aber, L., Dyregor, A., and

Raundalen, M. (1990). Child Behaviour Inventory.

Columbia University: Centre for the Study of

Human Rights, Project on Children in War.

Nikapota, A., & Samarasinghe, D. (l990). Training

manual for helping children in armed conflict. New

York: UNICEF.

Ressler, E., Tortorici, J., and Marcelino, A. (1993).

Children in War: a Guide to the Provision of Services.

New York: UNICEF.

Vanistendael, S. (1996). Growth in the Muddle of Life:

Resilience: Building on People’s Strengths. Geneva:

International Catholic Child Bureau. 1 In subsequent years, the Christian Children Fund, under the direction of Dr. Mike Wessels, developed a mechanism called the ‘child well being committee’ which estab- lishes a community vehicle for child protec- tion and development activities, from psy- chosocial to economic. This model appears to be ideal for implementing community based monitoring and evaluation methods. 2 In Eritrea at that time, all seven year olds were required to begin school, but for the rest of the population, only a small percent- age participated. It was argued that univer- sal free education should be a goal, but that it could only be provided for those who began first grade, as there was still a great shortage of teachers. The new school goers would benefit from universal primary edu- cation, as they grew older.

24

The Community Participatory Evaluation Tool for psychosocial programs: a guide to implementation Intervention 2005, Volume 3, Number 1, Page 03 - 24

Martha Bragin PhD. MSW is a member of the Clinical and Research Faculty at the Columbia University International Program on Refugee Trauma, Adjunct Associate Professor of Psychology at John Jay College of Criminal Justice and a member of the Committee on Psychoanalysis in the Community of the American Psychoanalytic Association. Dr. Bragin is a consultant to international organi- zations on psychosocial reintegration during and after conflict. 353 2nd Street 3R, Brooklyn, New York 11215

03_24_Bragin 07-03-2005 11:27 Pagina 24