...

usb - Valid International

by user

on
22

views

Report

Comments

Transcript

usb - Valid International
SQUEAC in routine monitoring of CMAM programme
coverage in Ethiopia
By Lily Schofield, Selome Gizaw Lalcha and Terefe Getachew
Lily Schofield has worked in many countries in Africa and Asia as a nutrition
consultant. She has been involved in many coverage assessments of selective
feeding programmes since 2007. She is currently working as the Evaluation and
Research Support Advisor, Concern Worldwide Kenya.
Selome Gizaw Balcha has a BA in Sociology and Social Anthropology and works as
a Senior Social Development Officer for the national CMAM programme, Concern
Worldwide Ethiopia.
Terefe Getachew has an extensive background in community mobilization and has
participated in multiple nutrition surveys and assessments. He is currently working
as a Senior Social Development Officer for the national CMAM programme,
Concern Worldwide Ethiopia.
The authors would like to thank the Woreda Health Offices and Health Centre Staff
and all Health Extension Workers and Community Volunteers who took part in the
survey, as well as the mothers and caregivers who participated as respondents.
The development of community-based management of acute malnutrition (CMAM)
was a significant advance in the treatment of severe acute malnutrition and
associated child mortality. One of its primary innovations was the decentralization
of services closer to beneficiary homes by treating children through out-patient
therapeutic programmes (OTPs) rather than as inpatients. This translated into the
potential for significantly higher coverage because travel time for beneficiaries was
shortened and community mobilization improved1 . However, in order to achieve
this potential, regular monitoring of coverage levels and barriers to coverage are
needed so that community outreach and mobilization strategies can be adapted
accordingly.
The challenges of monitoring programme coverage
Measuring coverage of CMAM programmes presents special
challenges when compared with other health services. Even during
acute food emergencies, severe acute malnutrition (SAM) only
afflicts a small percentage of the total under 5 population.
Therefore, a random sample of all children under 5 is unlikely to
include enough SAM children to estimate CMAM coverage with any
precision. Secondly, traditional survey methods that rely on
sampling proportional to population size may bias coverage
estimates upwards, as areas of high population are more likely to be
close to main roads and health centres (HCs)2 . In response to these
challenges, an alternative survey method based on area sampling
was developed for assessment of CMAM coverage. This method,
called Centric Systematic Area sampling (CSAS), overcomes the
biases described above. However it remains a resource and time intensive methodology, typically requiring
12 to 18 enumerators for 7 to 10 days of data collection and significant logistical support. It is therefore a
poor tool for routine monitoring of coverage, as only the most well resourced programmes can afford to
conduct regular CSAS surveys.
The challenge of monitoring coverage becomes more acute when an integrated CMAM programme is
considered. While many of the early CMAM programmes were implemented by non-governmental
organisations (NGOs) in emergency settings, more and more countries are beginning to integrate the
treatment of SAM into basic preventative and curative health services offered through Ministry of Health
(MoH) facilities3 . These programmes possess a high potential for sustainability, but are also resource
constrained both in terms of staff and finances. The result is that in most integrated CMAM programmes,
coverage is never assessed and so the proportion of SAM children being missed is never known. This also
means the community mobilization cannot be evaluated and improved.
Valid International and partners recently developed a new methodology (SQUEAC) aimed at integrated CMAM
programmes that is less resource intensive and requires only basic technical skills. This methodology is based
on a combination of qualitative and quantitative data collection to triangulate information about coverage and
barriers to access. This information is then confirmed through small area surveys that used Lot Quality
Assurance Sampling (LQAS)4 techniques to confirm or refute initial hypotheses about adequacy of coverage.
Pilot of new methodology in Ethiopia
Ethiopia has long been at the forefront of developments in CMAM. One of the first places where CMAM
programmes were initially trialled and the approach developed,5,6 Ethiopia is currently working to integrate
CMAM into primary health care facilities around the country. As of the beginning of 2008, CMAM services were
offered in tertiary health facilities in four regions of the country. Given the scale of integrated CMAM in
Ethiopia, there is an immediate need to identify ways of monitoring coverage on a regular basis that can be
carried out by the MoH with limited or no external support.
This article describes a pilot test of SQUEAC in one region in Ethiopia to determine the feasibility and
functionality of SQUEAC as a tool for use by the MoH to regularly monitor coverage of integrated CMAM
services.
The SQUEAC pilot was undertaken in 23 health centres (HCs) across 8 woredas (districts) in Tigray Region,
Ethiopia in November, 2008. A second round of SQUEAC surveys were conducted in the same HCs in July,
2009.
Study area
Tigray region is the Northern most region of Ethiopia bordering Eriteria on the East, Sudan on the West and
Afar and Amhara regions to the South. A mountainous region, the primary livelihoods are subsistence
agriculture, animal husbandry and wage labour. There is also significant reliance on safety nets and food aid
among the poorer segments of the population7 .
CMAM activities in Tigray Region began in June 2007 with initial
training and support provided by Concern Ethiopia. The
programme was launched in nine HCs in eight woredas
throughout the region. This was soon expanded to 10 more HCs
in 10 additional woredas. In 2009, the regional health bureau
(RHB) began further decentralization through a programme of
on-the-job training. At the time of the first round coverage
assessment, OTPs were functional in 81 HCs in 38 woredas
across the region.
Pilot Methodology
Collecting information of home
location of admission and defaulters
from OTP cards (Hewane HC)
The SQUEAC assessment was based on a two stage sampling
method. In the first stage, routine programme data such as
admissions, recovery rate, defaults, etc, were used to identify a region of each HC catchment area that was
likely to have low coverage. This data was also used to generate a list of reasons why coverage may be low
in this area.
In the second stage, a small area sample in the suspected low coverage area served to confirm or refute the
original hypothesis that coverage was low. Low coverage was defined as below the appropriate SPHERE
standard for selective feeding programmes (50% for rural areas, 70% for urban areas and 90% for camps).
Teams also collected qualitative information8 from key informants on perceptions of the programme,
malnutrition and reasons for non attendance to further triangulate data.
During the first round of SQUEAC, a cascadetraining model was used. MoH staff from all participating HCs as
well as representatives from the woreda health offices received a two-day training on the theory and practice
of SQUEAC. HC staff then trained Health Extension Workers (HEWs) and Community volunteers (CVs) from
their catchment area in data collection for the small area surveys. During the second round a one-day
refresher orientation was given to MoH staff from town administrations, woreda health offices and HCs.
Selection of woredas/Health Centres for pilot
All woredas with >50% of their area falling within the catchement of a HC offering OTP services were included
in the pilot. Within these woredas, all HCs offering CMAM and who had received support from Concern's
National CMAM programme participated in the assessment.
Sampling for small area surveys
Once the suspected area of low coverage was identified during the first stage of SQUEAC, a random sample
was taken from the villages in that area for the small area sample. Estimates of malnutrition prevalence and
average village size were used to calculate the number of villages to sample in order to find a minimum of 10
SAM cases9 . Experience from the first round showed that due to large village sizes in Tirgray, sub-villages (a
smaller population unit whose boundaries are locally known, but often are not listed in government records)
were a more useful sampling unit. Subvillages were therefore used as the sampling unit in all HCs during the
second round.
Within village sampling was done in one of two ways in the first round; active and adaptive case finding10 or
house-to house screening of all children 6 to 59 months in selected villages/subvillages. Based on first round
experience, it was decided that house-to-house screening was the most effective method. HEWs and CVs
were already familiar with this technique from immunisation campaigns and could complete a village/subvillage in a reasonable timeframe. House-to-house was therefore used by all teams in the second round.
SAM was defined according to programme admission criteria as a child whose mid upper arm circumference
(MUAC) was less than 11 cm and/or the presence of bilateral pitting oedema.
Data collection
Individual HCs were in charge of organizing their teams for data collection. On each team, a HEW was in
charge of leading the team and recording all data on the data collection forms. The CVs on the team were
responsible for MUAC measurement and oedema assessment.
For all SAM cases found that were not currently attending OTP, a questionnaire was administered to the
caretaker to determine the reasons for non-attendance. All uncovered cases were also referred to the
nearest OTP for treatment.
Data analysis
Data analysis was conducted using a simple compilation sheet that summed the information gathered from all
villages within a HC catchment area. The total number of SAM cases found in all villages surveyed and the
total number of covered cases was tallied. The achieved sample size (the number of SAM cases actually
found in the survey) was then used to calculate a decision value for the survey. The following rule of thumb
formula was used to calculate the decision value:
D= [N x p/100]
where p was the target coverage proportion (for example 50% in rural areas).
If the number of covered SAM cases was greater than the decision value, then coverage in the area surveyed
was classified as above the target. If the total number of covered cases was less than or equal to the decision
value, then coverage was classified as below the target. Defaulted cases and relapsed cases were considered
uncovered because they were not attending OTP at the time of the survey.
Results from qualitative data collection (focus groups and semi-structured interviews) were summarised by
main points. These points were compared to the results of the structured questionnaires collected during the
small area surveys. The information from both the qualitative data collection and the small area
questionnaires were used to identify barriers to coverage. Action points to address these barriers were then
outlined by health centre staff.
Evaluation of the pilot
To assess the success of the pilot and the usefulness of the methodology for MoH use in Tigray region, a
series of debriefings were held to capture lessons learned. First, the staff at several HCs were interviewed
about their experience and opinions of the SQUEAC methodology and process. Secondly, a debriefing was
held with Concern staff who supervised the survey and lastly, a debriefing was held with representatives from
the federal and regional MoH and Concern nutrition staff to present results and brainstorm about next steps.
A follow-up debriefing was held after completion of the second round with Concern staff who oversaw
implementation of the second round.
The pilot test was largely successful with all 23 participating HCs able to obtain information about CMAM
coverage and barriers to services during the first round and 22 HCs in the second round.
Implementation
Implementation of the entire SQUEAC assessment took six days in the first round and five days in the second
round, including training but excluding travel days. Fieldwork for small area surveys and qualitative data
collection took two days within each health centre area. During the first round, a large number of data
collectors were used to minimize data collection time. However it became clear that these high numbers were
not required for efficient data collection. Therefore numbers were reduced significantly in the second round
(see Figure 1), with no significant negative impact on data collection time or quality.
Outcomes
In the 23 HC catchments surveyed in the first round, all except one were below their target coverage level
(50% for rural areas and 70% for one town administration) (Table 1). In the second round, three additional
HCs had increased their coverage to above their target level.
Table 1: Results for standard indicators measured on IYCF practices
First round - November 2008
Second round-July 2009
Health
centre
#
cases
#
covered
Decision
value
Coverage
#
cases
#
covered
Decision
value
Coverage
E/Bahre
22
6
11
<50%
16
4
8
<50%
Abi Adi
9
1
4
<50%
7
2
3
<50%
Wukro
Mariam
19
2
9
<50%
11
2
5
<50%
Chila
9
0
4
<50%
7
2
3
<50%
Mayasmi
20
1
10
<50%
18
2
9
<50%
Hayelom
9
0
4
<70%
19
1
9
<70%
E/Homus
28
6
14
<50%
2
1
1
<50%
Wukro town
4
0
2
<50%
3
1
1
<50%
Dinglet
13
2
6
<50%
0
0
-
<50%
Zala
26
7
13
<50%
12
1
6
<50%
Sobeya
16
4
8
<50%
18
3
9
<50%
Fatsi
15
0
7
<50%
18
3
9
<50%
Zalambesa
4
0
2
<50%
5
4
2
<50%
Freweni
14
1
7
<50%
5
2
2
<50%
Abi Kebeles
22
9
11
<50%
NA
NA
NA
<50%
Dengolat
11
0
5
<50%
6
2
3
<50%
Mehoni
5
4
2
<50%
5
3
2
<50%
Kukuftu
4
2
2
<50%
8
5
4
<50%
Hreko
7
1
3
<50%
4
2
2
<50%
Korem
8
3
5
<70%
0
0
-
<70%
Hiwane
7
1
3
<50%
7
0
3
<50%
Adi Godum
3
1
1
<50%
6
4
3
<50%
Ado Keyeh
4
1
2
<50%
5
0
2
<50%
NA=data collection postponed due to schedule conflict with training for community based-nutrition
The main reasons for non-coverage were largely the same in both rounds and are presented in Figure 2. In
both rounds, lack of awareness of the CMAM programme was the primary barrier to coverage, followed
closely by distance and lack of awareness about the signs and symptoms of malnutrition. While awareness of
CMAM and of malnutrition requires further improvement, there is evidence that the action steps implemented
by the ministry following the first round had a positive impact. The percentage of mothers with severely
malnourished children surveyed who did not know about the programme decreased from 39.2% to 21.7% in
the second round. Similarly, the percentage of caregivers who did not think their child was malnourished
decreased from 14.5% in the first round to 10.9% in the second round (Figure 2).
Uptake by MoH actors/Sustainability
There was a high level of interest and investment by the MoH staff, particularly at a regional and woreda
level. The identification of barriers to coverage and implementation of action steps to address these barriers
was prompt. Before commencement of the second round, Concern held discussion with the woreda health
offices about decreasing the financial and logistic support in the second round and eliminating financial
support in the third round. There was a high level of ownership of the process at the woreda level, though HC
staff still expressed a need for external support to complete any future assessments.
There was a low rate of staff turnover in the seven months between the first and second rounds, facilitating a
rapid refresher for HC staff in the second round. Unfortunately, low staff turnover is not the norm for Ethiopia.
Increased inputs in the form of refresher training may be required before successive rounds of SQUEAC, if it
is used in other areas of the country.
Discussion
The methodology was well understood by the MoH and was successfully implemented in a short period of
time. SQUEAC assessments required much less time and resources than the traditional CSAS method. As
evidenced by the positive changes seen in coverage in several of the surveyed areas, the assessment also
provides information on barriers to access and coverage that can be translated into effective remedial
actions.
Challenges
A challenge that emerged in the second round was when an area sample found no severely malnourished
children. Because the methodology identifies a relatively small area for sampling, focusing time and
resources on suspected areas of low coverage carries the risk, particularly in areas were SAM prevalence is
low, that no children eligible for OTP will be found. If no children are identified, then coverage cannot be
classified as above or below the target. One possible solution to this would be to select backup villages in
areas were the SAM prevalence is low. If the first sample did not find any SAM cases, then sampling could
continue through the replacement villages. While this would require more time for data collection it would
ensure that coverage could be classified.
While the methodology worked well in rural areas, it was less robust in the urban context. Three of the 23
HCs included in this pilot were town administrations, which cover a very small geographic area but have high
population densities relative to the rural areas. The urban centres also tended to have lower malnutrition
rates, which had implications for sample size. Because of this, identifying a sub-area of low coverage within
the town administrations was not meaningful. In the three towns included, the entire town was taken as the
sampling frame. Even when the entire town was taken as a sampling frame, identifying meaningful sampling
units was difficult. Divisions within towns were largely administrative and did not correspond to population
groupings, mixing both business districts and homes. Trend analysis and beneficiary home mapping were also
more difficult to interpret in towns as only a handful of cases attending OTP at the town HCs came from the
town itself. The majority of attendees came from outside the catchment area of the HC.
This pilot also highlighted that given the health workers limited experience with qualitative data methods such
as focus groups and key informant interviews, a key component that was missed during the first round
assessment was adequate training and guidance on conducting the qualitative part of the assessment. This
was addressed in the second round by developing a short worksheet. This worksheet helped ensure
uniformity of qualitative data collection and helped enumerators plan the qualitative work.
Even with these challenges, the assessments provided clear indications of where the community mobilisation
systems were breaking down. Based on this, health staff were able to define immediate actions steps to
improve coverage.
Conclusions
This methodology has the potential to become a useful tool for routine management and monitoring of CMAM
activities by the MoH. There was a high level of ownership at the woreda level and HC staff were able to carry
out the assessments and utilise the resulting information to address barriers to coverage.
While resource requirements for these assessments were minimal compared to alternative coverage survey
methods, some costs remain. Whether these costs can be successfully incorporated into MoH budgets and a
regular schedule of coverage assessments maintained without external support remains to be seen. However,
this pilot clearly demonstrates that providing simple, easy to use tools for programme assessment that can be
implemented with limited resources and external support is key to improving the quality of integrated CMAM
programmes.
For more information, contact: Lilly Schofield, [email protected]
1
Collins S, Dent N, Binns P, Bahwere P, Sadler K, Hallam A. Management of severe acute malnutrition in
children. Lancet 2006. Dec 2; 368(9551): 1992-2000.
2
Myatt M, Feleke T, Sadler K, Collins S. A field trial of a survey method for estimating the coverage and
selective feeding programmes. Bull World Health Organ 2005 Jan; 83(1): 20-6.
3
Gatchell V, Forsythe V, Thomas PR. The sustainability of community-based theraputic care (CTC) in nonemergency contexts Food Nutr Bull 2006 Sep 27; 27(3):S90-S98.
4
Myatt M. SQUEAC: Low resource method to evaluate access and coverage of programmes. Field Exchange
2008 Jun; 33: 3-5
5
Collins S, Sadler K. Outpatient care for severely malnourished children in emergency relief programmes; a
retrospective cohort study. Lancet 2002 Dec 7; 360: 11770-11773.
6
Chaiken M, Deconinck H, Degefie T. The promise of a community-based approach to managing severe
malnutrition: A case study from Ethiopia. Food Nutr Bull 2006 Jun 27;27(2): 95-104.
7
Tigray Region Livelihoods Profiles: Regional Overview. Livelihood Integration Unit. DPPA, January 2008.
8
Though ideally qualitative data collection happens before areas are chosen for small area surveys, because
of the impending EOS screening and other schedule constraints, qualitative data collection and small area
surveys were conducted simultaneously.
9
Personal communication with M. Myatt 10 See footnote 2.
Taken from Field Exchange Issue 38, April 2010
http://fex.ennonline.net/38/routine.aspx
© 2010. ENN is a registered charity in the UK no. 1115156, and a limited company no. 4889844.
Fly UP