Archived page: may contain outdated information!
  • Send page
  • Print page

Bosnia and Herzegovina, Croatia and Federal Republic of Yugoslavia (FRY)/Kosovo

04-11-2002

ICRC community-based mine/unexploded ordnance awareness programme

 Contents:  

    Introduction    |      2. Evaluation Methodology    |      3. Mine Awareness and ICRC in the Balkans    |      4. Effectiveness & Efficiency    |      5. Relevance, Coherence, Coordination and Hand-over    |      6. Recommendations  
 

The evaluation abstract summarizes the main findings, conclusions and recommendations contained in evaluation studies conducted throughout 2001 and 2002. These studies were undertaken by independent experts with no previous involvement in the program activities evaluated. As such, the reports constitute an independent assessment of ICRC’s work. Responsibility for their contents and quality rest with the respective evaluation teams. The views and opinions expressed in the evaluation abstracts are those of the experts and do not necessarily reflect the position of the International Committee of the Red Cross. ICRC is currently reviewing the appropriateness and feasibility of recommendations in terms of future planning and budgeting.

 
 

 REPORT TITLE:  Evaluation of Mine Awareness Programmes  

 REGIONS: Bosnia and Herzegovina, Croatia and Kosovo

 PROGRAMME: Preventive Action

 TARGET POPULATION: Civilians

 LANGUAGE OF REPORT: English

 DATE OF REPORT: June 2002

 EVALUATION TEAM: Emery Brusset, Chris Horwood

   
go to top of page
 
1. Introduction 
 

The International Committee of the Red Cross commissioned an external evaluation of the Mine Awareness Programme (MAP) in October 2001. The purpose of the evaluation was to capture the knowledge generated by the pilot programmes in the Balkans, and provide verifiable information on achievements to date in three of the countries, Croatia, Bosnia & Herzegovina, and the Province of Kosovo for future planning.

This abstract is a synthesis of the main conclusions of three country reports, each of which can be read separately. The findings presented here are common to the three country cases, and are summarized to facilitate their application to other ICRC operations. Specific country information can be found in the annexes.

The evaluation was undertaken between December 2001 and April 2002. All the main stakeholders in Geneva and the three countries were visited, and 4,550 interviews were carried out on random samples of persons drawn from the general population, and groups most at risk.

This unique research opportunity permitted the evaluation to focus its analysis on impact. It also generated an exceptional amount of information on the situation and ICRC response in these three countries.

 
go to top of page
   
2. Evaluation Methodology 
 

The terms of reference point to two key outcomes of the evaluation: to document the impact of the programme, and to inform the process of phasing out and transfer of remaining responsibilities in the three countries.

The three programmes covered a wide array of activities. The evaluation focused on verifiable changes, which occur as a result of the programmes. The analysis reflects two areas of expertise: the first in surveys of societal impact of the mine awareness programme conceived as a public information campaign (ensured by Mr Chris Horwood); and the second in management of international institutions and the Red Cross Movement (ensured by Mr Emery Brusset).

The field visits included a total of 24 days to cover the institutional issues, and 42 days for the surveys. Three teams of field researchers were recruited and trained locally for the study. The assignment was undertaken in two successive visits to each country, so as to correspond with the design and compilation of the survey data.

The team used the following five techniques to collect information:

  1. Countrywide survey questionnaires: (some standardised others targeting special groups):In total 4750 questionnaires were completed.

  2. Focus group meetings (with special groups such as school children, hunters, fishermen as well as RC Youth Movement members) to verify the information gained from the quantitative surveys.

  3. Random interviews an d semi-structured interviews of rural farmers, town inhabitants, children, and of the main actors (see annex 3 " Persons Met " ) in particular ICRC/Operational National Societies/NGOs/Mine Action Centres and NGOs. This led to an analysis of key narrative themes, a qualitative methodology based on mapping convergent or contradictory accounts, and verifying the contradictory information through access to new sources, checked for possible bias.

  4. Desk review of documentation from ICRC and other agencies, collected during three successive visits in Geneva and two visits in country.

  5. Direct observation, such as viewing campaign material, expositions, videos and media spots, etc.

 This is the most detailed and wide ranging quantitative survey conducted so far in mine awareness in the humanitarian sector .

 
go to top of page
   
3. Mine Awareness and ICRC in the Balkans 
 

The use of mines has increased over time since their introduction in the late 19th century. Since 1945 the extension of non-international armed conflicts has led to an increase in mine risk, as the proliferation of cheap landmines became widely available in the 70s and 80s during the Cold War period of proxy wars and internationally supported national rebel movements.

Mines, and more generally explosive remnants of war, fall within the category of weapons whose effects are particularly cruel, lasting, and indiscriminate. In December 1997, 123 states signed the Convention on the Prohibition of Anti-personnel Landmines, called the Ottawa Treaty (122 ratifications in April 2002). The ICRC has determined that the use of landmines against civilian populations is a violation of the customary rules of international humanitarian law.

Mine awareness (MA) is a new sector of humanitarian activity, having begun in the early 1990s in Afghanistan and Cambodia and then other affected countries, and for ICRC in 1996 in Bosnia. MA is defined here as a public communication campaign (sometimes at the community level and sometimes national) aimed at changing the knowledge, attitudes and behaviour of a population towards mines, unexploded ordnance (UXO), and other explosive remnants of war.

MA implies a high degree of network based project cooperation with a variety of social information actors, many of which have not been traditional partners of the Red Cross. However the existence of Red Cross branches at local level in most countries has given the ICRC a unique advantage in its involvement in this field. The national Red Cross societies effectively provide ICRC with a socially codified and often highly considered community of volunteers in every country of operation.

The period concerned by the evaluation begins in January 1998 just after the signing of the Ottawa Treaty (and in August 1999 in Kosovo, date of the initiation of the programme). The general aim of the ICRC had been increasingly to support a comprehensive response to the scourge of mines and unexploded ordnance. Mine awareness complements the legal work carried out for the banning of anti-personnel mines, and is a growing but complementary addition to the primary aim of mine victim medical assistance.

Under the 1998 " Appeal for Assistance to Mines Victims " , a total of CHF 18,659,642 were spent, of which physical rehabilitation, surgical treatment and the Special Fund for the Disabled received CHF17, 038,596. The advocacy campaign, linked to the Ottawa Treaty, received CHF1, 262,305, and MA CHF1, 099,849 or just under 6% of the total. By 2000 this had barely shifted to a total mines related expenditure of CHF 29,290,070, of which CHF2, 095,326 was spent on MA (7%).

In 1998 80% of MA spending was allocated to Bosnia and Herzegovina (the other countries being Azerbaijan, Croatia, Angola and Sudan, in that order), while in 2000 70% of the total was still being shared between Bosnia, Croatia and Kosovo. This illustrates the importance of the Balkan programmes in shaping MA within ICRC.

The importance of the legal work receded worldwide from 1998, after the signing of the Ottawa Treaty. While the MA budget has doubled, legal and advocacy spending shrank by two thirds. This illustrates the growing importance given to national level campaigning.

Within ICRC mine awareness is now managed by the Division of Communication as a form of preventive action, itself defined as " pre-emptive steps taken to prevent or reduce the suffering of people directly affected by armed violence, carried out in peacetime and in times of war " [1 ] . The support to the country programmes in the Balkans is provided by a Geneva based Sector Coordinator, and by a Regional Advisor. Three MAP teams manage the programmes in each Delegation (Sarajevo) or Mission (Zagreb and Pristina), composed of two to five individuals. These are supported by Field Officers who liaise with teams of Mine Awareness Instructors, who are Red Cross volunteers based in the Municipalities and conducting MA (at all times totalling more than 80 in each of the country cases). In Bosnia and Croatia, ICRC also funds coordinator positions within the national Red Cross structures.

 Each MA programme is composed of a combination of national and local television and radio spots (in the latter two cases except for Kosovo), presentations, multi-media exhibitions, special events, and poster and leaflet distributions. The general objectives have changed over the years and across countries, but the overall purpose remains the reduction in the rates of mine and UXO related incidents. This is served through general public information, building up the capacity of the national Red Cross, and advocacy aimed at national authorities, civil society and other humanitarian mine actors.  

 
go to top of page
   
4. Effectiveness & Efficiency 
 
Effectiveness 
 

 Effectiveness in terms of reduction in incidents :   Measuring effectiveness is done against stated objectives, and it has not been possible for the evaluation to define how much the ICRC MA programmes contributed to the reduction of accidents. However if one defines the long term objective in a slightly more limited manner as ‘generating an information flow which can influence knowledge, attitudes and practi ces’, then the programmes have been effective [2 ] . The nature of this information flow becomes clearer when looking at the specific objectives. In Croatia the aims have evolved from influencing the accident statistics to creating an enabling environment in which the message will be relayed and sustained by a wide variety of actors. In Kosovo ICRC reinforced the two-way flow of information, through the establishment of a village-based volunteer structure. In Bosnia risk information was communicated actively to a community of actors and authorities, and to the population.

 Effectiveness in terms of exposure of the population : This evaluation finds that the level of coverage in the three countries in relation to the dissemination of mine awareness information is very high. In mine affected and non-mine affected areas virtually all people, adult and children have been exposed to mine awareness, in some form, in recent years. This is predominantly due to the prevalence of mass media and people’s access to televisions and radios. In mine-affected areas in Croatia, a large proportion of the inhabitants have been exposed to different forms of mine awareness through the CRC/ICRC posters and leaflets and presentations by the CRC mine awareness instructors. In Bosnia and Kosovo other agencies have also contributed to this result. 

 Effectiveness in terms of changes in KAP: The evaluation concludes on the basis of an extensive survey that the performance of MAP in the Balkans has been highly effective in targeting key populations at risk, and generating a very satisfactory level of ‘mine smart’[3 ] knowledge leading to changes of attitude and behaviour. The constraints were significant, considering the existing threat and the adverse effects of civil war (such as divided Red Cross societies, politically divergent views about mine awareness). The programme in Ko sovo has attained slightly lower levels of mine smart knowledge, but on the other hand it has ensured a more effective two-way flow of information to and from the communities about threats and risks.

 Effectiveness in terms of Cooperation: The ability of the ICRC to generate capacity within the national society has been good at the Municipal and at times Branch levels, but can be described as a national success mostly in Croatia. In part the explanation rests on local capacity, but also the formulation of objectives, which do not define, in a manageable way, the institutional impact of MA. The benefits of MA are not well captured in the Planning for Results process, as it tends to divide Cooperation from Communication, and concentrate on target populations rather than institutional outcomes.

 
Efficiency 
 

 Red Cross resource management: Considerable efficiency, drive and local knowledge are gained from the established RC networks and the practice of RC volunteers. The permanent ICRC staff working on what are nationwide campaigns is limited (2 persons full time and 5 field officers part time in Croatia currently, 1 full time in Kosovo with support from field officers). Although costs have been further contained in Kosovo by not paying village liaison volunteers at all, this programme is also affected by a lower level of performance as measured by actual changes in behaviour in the population. There is generally a good retention of knowledge within ICRC, teams are well focused, and good planning in relation to needs. The methods of dissemination are innovative, and well adapted to their context. However the support functions (regionally and at headquarters) are very understaffed, increasing the isolation of the MAPs in each countr y. This is aggravated by the tendency of programmes to function in relation to isolated, compartmentalised objectives.

    

 Cost benefit analysis: There is little monitoring of impact, which means that too much energy goes into following up on a considerable range of small projects. No benchmarks for phasing out or handing over are defined, and the programmes risk running for as long as ICRC is willing to seek funding. The emphasis given equally in the course of programmes to one project or the other may be misplaced. Some indicative cost benefit analysis shows, for example, that for the same impact, television can be twenty times cheaper than theatre, or five times cheaper than posters and publications.

One could extrapolate the cost of 1% of the population exposed to mine awareness for each medium from the tables in the country reports. In the case of Croatia (which is the most suitable case study as in this country ICRC has been the sole MA operator for the most part) the analysis yields the following results for example:

Activity

DEM

CHF

Television/radio

906

687

Posters and publications

4,159

3,153

MAIs

9,624

7,294

Theatre

17,500

13,265

Although these figures should be taken with extreme care, they would however seem to highlight an order of magnitude in terms of assumed benefits stemming from each medium. The figures provided by the general population are matched by other groups surveyed, and comparable ratios have been found in the three countries.

This efficiency is further increased for ICRC when one takes into consideration the capacity of local acto rs to make their own contribution to match the ICRC one. The national Croatian television broadcaster HTV for example reports that over the period November 2001 to January 2002 its 90 broadcasts of ICRC spots, each lasting 30 seconds, cost the equivalent of 570,000 DEM (these spots are shown on prime time). Over the average three-month period since 1999 ICRC has spent 7,250 DEM for the TV broadcasts.

The overall spending of CHF 2 Million per annum globally on Mine Awareness, of which 80% has been spent in the Balkans, is not high considering the impact in terms of changes in attitudes and behaviour (from 30% to 95% changes among the different target groups and the three countries, although factors other than ICRC must be taken into account). The ICRC programmes are structured very differently from those of other agencies (UNICEF uses the Ministry of Education as the main partner, for example), and no comparable impact assessments have been carried out which could allow reliable efficiency comparisons to other agencies.

 Multiplier effects: The absence of benchmarking (in terms of phasing in or out) and of impact indicators in the programmes has meant that opportunities for achieving greater impact have been missed by concentrating on potentially less effective media. An excessive autonomy of the MA teams within ICRC and the mine action system leads to an inability to communicate within ICRC the opportunities and benefits accruing from implementation (providing entry points for medical and psycho-social or economic security programmes for example, or strengthening protection and cooperation). At the same time ICRC is given priority to a high level of interface with the civilian population and national Red Cross in a legitimate (for the public) field of intervention, thus gaining in access and credibility.

ICRC programmes have been good in terms of content and focus, and in terms of taking advantage of the Red Cross Movement strengths. On the other hand the programmes lack steering from the point of view of the outcomes and the desired changes in the context. This passivity towards the external world may not be easily perceived by looking at the activities and structure, but stands out clearly when reviewing these against measured impact.

 
go to top of page
   
5. Relevance, Coherence, Coordination and Hand-over 
 
Relevance and Coherence: 
 

 Need: The rate of incidents relating to mines and UXOs dropped considerably in the three countries over the evaluation period, but the reduction was never as high as could be expected considering the very high level of threat. This may partly be attributed to MA, which is certainly perceived by partners and the population as an important element of safety and well-being.

    

 Mandate: MA is consistent with the core work of the ICRC. It reinforces the promotion of international norms, victim assistance and protection, and cooperation. While the activities have remained rather restricted to communicating messages to the population, and in the case of Kosovo eliciting information from the population to better organise mine action responses, the evaluation finds that ICRC should formalise its community based approach, and possibly expand into highly focused clearance. This involvement was argued in 1998 " End of Mission Report: September 1996 – August 1998 " , Laurence Desvignes, ICRC Sarajevo on the strength of protection. The example of the light and rapid system for Explosive Ordnance Destruction commissioned on specific actions by ICRC The Swiss Federation of Demining (now called Swiss Federation for Mine Action) placed a team on call in Kosovo for ICRC tasks. This work is being evaluated internally by ICRC to draw out lessons for the future. in Kosovo has proven to be beneficial to villagers.

 
Coordination and Hand-over: 
 

 Coordination: balanced and efficient interaction with other mine actors and especially the Red Cross has taken place in all countries, and the relative marginalisation of MA in mine action, the reluctance of public authorities towards MA, and the weakness of the national Red Cross societies in this field, have been effectively tackled. Some degree of sustainability has been ensured, particularly in Croatia, where much increased priority and interest is given to MA by key players, generating an environment where some donor funding would have a very large impact.

 Transfer: The programme has gradually developed an analysis of the activities of other actors in mine action and successfully defined a strategy and role for ICRC. If the partners are able to be operational, ICRC is in a position to hand over to them a community liaison function, valuable material and a demand for more MA from the population. For reasons which are beyond the control of the MA p rogrammes and which relate to each country’s political and financial situation, the National Societies are not able to be operational, with the possible exception of Croatia.

The ownership of the programmes by the National Societies has not been well secured by the ICRC. This is due to the concentration of the National Societies on the core priorities of the Red Cross (assistance, disaster response, support to social cases), as well as the financing difficulties that these Societies face in the Balkans. It is partly due also to the highly driven and autonomous nature of the ICRC teams, where in some cases even the networks of volunteers have been more closely identified with the ICRC than with the National Society. Credit must be given however to the very long-term capacity building commitment of the ICRC, which in some cases was closely integrated in the Cooperation programme.

 
go to top of page
   
6. Recommendations 
 

ICRC's intervention in anti-personnel mines is highly coherent with its mandate and capacities, and the impact has justified the investment.

There are however opportunity costs in terms of efficiency. Programmes are not well supported and too compartmentalised. Poor monitoring of outcomes leads to poor prioritisation. Furthermore contributions MA could make to the general work of ICRC are not taken advantage of.

These defects can be eliminated by a clearer design of objectives and more systematic us e of population surveys, leading to a more solid basis for decision making. The following recommendations follow this three point structure.

 
6.1. Redefinition of objectives 
 

 The evaluation would recommend understanding the general objective as "generating an efficient risk information capacity". This information would run between the society and national and international mine actors, with the ultimate aim of improving safety. This conceptual shift would unify the existing institutional and the population oriented aims of the programme. It would also allow other ICRC programmes to take better advantage of the work carried out by the MAPs.  

6.1.1. The results achieved by ICRC are not necessarily reductions in the rates of accidents, as many other variables intervene which could theoretically even increase the incident rate while the programmes are effective. A more limited general objective is required, defining an area that ICRC can influence more directly. The Specific Objectives of the ICRC are better indications of what the organisation is aiming at, but need to be specifically marshalled around a uniform outcome, ideally sustained over time by other actors, as intended in Croatia.

6.1.2. The work of ICRC allows the population to better understand the risks (defined as the potential for negative or positive outcomes from a particular course of behaviour) by relaying the messages communicated; and to better manage the threat (marking and eliminating the explosives) by calling on national and international resources for mine action. It seems advisable for ICRC to proceed with a selective involvement in EOD/clearance, possibly through an external standby mechanism. This threat and risk reduction is clearly aligned with IHL, and ICRC's role in relation to war: promotion of international standards, generating political interest in these standards, and protecting the population from the consequences of war.

6.1.3. ICRC can contribute to information flow through volunteer networks attached to Red Cross structures, public communication campaigns, database management, and making a link to the central services of the national Red Cross and the mine action and medical assistance community (in particular the Mine Action Centres). Within ICRC this should lead to further development of the support structures in Geneva.

6.1.4. There are other benefits accruing from MAP beyond risk and threat management, and these relate to the unique ability to become an interface between the population and the Red Cross Movement. The information content of RC volunteer networks could be extended to support systematically the promotion of international norms, and protection of the civilian population. The networks could be tuned to serve the national Red Crosses in their core activities, and be offered as an ICRC contribution to the national Societies. 

    

6.1.5. To adopt this conceptual redefinition (MA deals with efficient information transfers, not directly with safer behaviour) will have consequences for the ICRC programmes. If people learn the core messages of mine awareness from occasional exposure to TV spots supported by a presentation, ICRC may wish to spend less resources on more colourful but time consuming methods. The objective of dissemination does not seem to lead to a wide range of multi media initiatives, but rather to a better study of the dynamics of a population's perceptions.

 
6.2. Residual responsibilities in the Balkans: 
 

 Results show a need to continue a steady but more limited MA programme in each country, focused on active cooperation with the national Red Cross, and with a few activities, such as national media and specialised presentations. This needs to be more limited in Kosovo and Croatia than in Bosnia.  

6.2.1. Where the population has a TV culture or where mass media is regularly used by a community, ICRC needs to concentrate resources to harness the power of mass media for risk management. A sustained low cost media campaign would be justified, in particular through the continued use of high quality material produced.

6.2.2. Mass media needs to be supported by selected programmes of presentation and interaction appropriate to different people within each community. Specialised presentations will be much more efficient if these are developed for specific population groups.

6.2.3. Capitalising on the RC network of volunteers, the ICRC must continue ensuring a minimum level of interest in MA, so responding to the expressed needs of the population and partners for continued assistance in this area. It would seem natural for the capacity issues to be handled in the future as part of a comprehensive programme by the Cooperation programmes.

 
6. 3. Indicators for engagement and disengagement:  
 

 ICRC must develop a monitoring capacity based on the methods developed for the present evaluation, and set indicative targets to intervene in, or phase out from, the achievement of institutional sustainability and optimal mine awareness in the population. There are opportunities currently in which ICRC could carry out a baseline survey before mine action begins in a country, an innovative approach with significant value added in terms of reporting and prioritising.  

6.3.1. ICRC should develop criteria for mine awareness interventions that reflect ICRC programme objectives as well as the level of need in affected countries. Such a ‘policy’ should ideally be based on needs, ICRC/RC capacity, and other internal ICRC objectives. It may be advisable to review the global programme budget and the mine affected countries in the world, and establish a priority need rating. This is not to suggest Kosovo did not need intervention, for example, but ICRC has not been active in seriously mine affected countries, and yet is active with mine action in countries far less affected. Equally ICRC may have partial mine awareness programmes (data gathering only or some victim assistance but no mine awareness etc…). As in Kosovo, ICRC should not necessarily respond in the same way as the rest of the mine action community who may be operating on different criteria, and/or seeking to achieve different objectives.

6.3.2. In an effort to understand the dynamics between knowledge and behavioural change in relation to risk management, ICRC should develop an on-going project monitoring function in its mine awareness programmes. This would not be complicated and would over time give added value to the mine action community as well as directly inform their mine awareness programmes in terms of methodology, targeting, sustainability and impact expectations.

6.3.3. To avoid the possibility that the mine awareness message is not being correctly taken up by target populations and especially the Mine Awareness Instructors liaising between ICRC/RC and the communities, ICRC needs to develop an internal monitoring system. If village volunteers are not correctly learning the basic safety messages this will be noticed during the project and remedial steps can be taken. It is too late to test this during an evaluation.

6.3.4. A crucial part of risk management with regards to mine awareness has to be the targeting and engaging of the most vulnerable sections of the population. In the case of mines and UXO the vulnerable groups are men and ICRC should put more energy into exploring how they can ‘reach’ this group. Too long have mine action agencies focused on child/youth outreach.

 
 

 Notes  

1. " Emergency Appeals 2001 " , Programme Descriptions, page 19.

2. It is generally accepted that mine clearance is less effective in reducing accident rates as it is by nature slow and localised.

3. Mine smart knowledge is a phrase used in this report to capture the degree to which people have absorbed the messages of MA. It refers to their level of knowledge but does not necessarily presuppose that it leads to a change in behaviour.