IHS Rapid HFA discussion. NOT FOR CIRCULATION.

(initial publication: Sep 13, 2023. last update: 2023-10-01 16:29:49)

Summary

BACKGROUND
For a rapid assessment with remote data collection (either via phone interviews or self-report), we must reduce/maintain the interview time/length. However, we often end up having a series of detailed questions. Questionnaires that look good in theory do not necessarily generate useful data. Moreover, unrealistically detailed and/or long questionnaires negatively affect data quality in phone interviews. There are two specific problems that we must address in the rapid assessment. We explore the questions with empirical data from existing HFAs.

Problem 1. Response categories are too many and difficult to understand.
Question: Can we simplify response categories for availability of medicines and availability of equipment?
Answer: Yes, because the detailed categories are rare, and inclusion/exclusion of those categories will not change estimates meaningfully. Therefore, no overall insights will change even with simple yes/no options for availability questions. See Part 1 for details.

Problem 2. The list of medicines, commodities, and equipment is too long.
Question: Can we reduce the number of tracer items?
Answer: Yes, a set of 10 questions appears to be equally efficient in classifying facilities with categories - as long as we are clear that the purpose of HFA is not to check availability of all individual essential medicines. See Part 2 for details.

Part 1. Response categories

Based on empirical data, we want to see to what extent we need to assess detailed availability. of medicines and equipment in rapid phone HFA. We assess distribution of facilities falling into detailed categories that may not be suitable for the rapid assessment. Examples of such categories are:
- Relatively rarely reported responses (e.g., equipment available but not functional) - Categories that are not necessarily clear in terms of programmatic utility (e.g., medicine never available/procured)

Two data sources
A. SPA - data collected and observed by surveyors through facility visits
B. COVID-19 HFA - data collected/reported through phone interviews (similar methods with the proposed rapid HFA)

1.1 Medicine availability categories in SPA

At least one unexpired unit observed - Yes
Observed, but expired - Yes, but
Reported to be available, but not observed - Yes, but
Not available today - No
Never available - Never
Missing responses (e.g., the assessed facility do not store or provide any medicines) - Missing For the analysis, ‘Yes, but…’ includes #2 and #3

With missing

Without missing

1.2 Equipment availability categories in SPA

Lab equipment
1. equipment used and observed + in working order - Yes
2. equipment used and observed + not in working order - Yes, but
3. equipment used reported, not seen - Yes, but
4. equipment used, not available today - No
5. not used - Never
6. Missing responses - Missing
For the analysis, ‘Yes, but…’ includes #2 and #3
Other equipment in general OPD
1. observed, functioning - Yes
2. reported functioning - Yes
3. observed, not/dk if functioning - Yes, but
4. reported, not/dk if functioning - Yes, but
5. not available - No
6. Missing responses - Missing
For the analysis, ‘Yes’ includes #1 and #2. ‘Yes, but…’ includes #3 and #4

With missing

Without missing

1.3 Medicine availability categories COVID-19 (combined) HFA

Currently available - Yes
Currently unavailable - No
Never available - Never
Missing responses (e.g., the assessed facility do not store or provide any medicines) - Missing

With missing

Without missing

1.4 Equipment availability categories COVID-19 (combined) HFA

Currently available and at least one functional - Yes
Currently available but none functional - Yes, but
No - No
Missing responses - Missing

With missing

Without missing

Part 2. List of tracer items

Assumptions:
We ask about availability of various medicines not necessarily to know availability of individual drugs. Rather, we want to know:
1. On average how many are available among the select medicines (e.g., percent maximum score) - knowing that often facilities do not have all medicines assessed, and/or
2. Where do facilities fall in terms of broad categorization (quin tiles) to see if a cluster of facilities need more intensive support (e.g., geographic clustering, type of facilities)

For the last two objectives, we do not need to have a long list of medicines. Our goal is to find a “sweet spot” that reasonably satisfy all objectives. i.e., the shorter the list, the better - if the analytic value is similar.

In this exercise, we create and compare various metrics, using different sets/lists of medicines. Selection of medicines can be based on:
* typical availability of different medicines (include those with wide ranges)
* medicines distribution paths, if different (include childhood vaccines vs. contraceptives).

Data source: SPA - data collected and observed by surveyors through facility visits
Data: A total of 65 medicines were assessed in all three countries: Haiti, Malawi, and Tanzania. In addition, 17 equipment were assessed in general OPD areas in all three countries. This analysis uses only those medicines that were relatively commonly asked across all three surveys.

2.1. Exploratory analysis: range/distribution of medicine availability

Average of ‘national’ estimates

Detailed results by country and facility type

2.2. Exploratory principal component analysis (PCA) results by country

Exploratory results suggest that, in all three countries, PCA based on indices may not be good to asses underlying character/concept behind availability of medicine items. For example, only a small portion of variation is explained by the first component, around ~20%-25%. In any case…

Haiti

Malawi

Results not shown. Similar with Haiti results.

Tanzania

Results not shown. Similar with Haiti results.

2.3. Comparing indices

TWO approaches to create summary metrics
1 sum scores: number of tracer items available out of the total number of tracer items (scaled to 0-100)
2 pca index: index created from PCA, using different sets of medicines

FOUR sets of tracer medicines
1: All 65 common medicines available in the three surveys
2: Randomly selected 20 medicines
3: Randomly selected 10 medicines
4: Specific 10 medicines that are available at low-, medium- (30%-60%), and high-levels Option A
4: Specific 10 medicines that are available at low- and high-levels Option B
5: Specific 10 medicines that are available at low- and high-available Option B

List of 10 Option A medicines (average availability (%), across three country SPA data)
- beclomethasone inhaler (3.7)*
- simvastatin (high cholesterol) (6.7)*
- insulin injections (lente) (diabetes) (13.7)*
- hydrocortisone (31.7)
- salbutamol inhaler (32)
- magnesium sulphate injection (39)
- gentamycin injection (52.3)
- amoxicillin syrup/suspension (71.7)*
- doxyclycline (71.7)*
- paracetamol tablets (82.3)*

List of 10 Option B medicines (average availability (%), across three country SPA data)
- beclomethasone inhaler (3.7)*
- calcium gluconate injection (6.7)
- simvastatin (high cholesterol) (6.7)*
- insulin injections (lente) (diabetes) (13.7)*
- dexamethasone injection (22.3)
- amoxicillin syrup/suspension (71.7)*
- doxyclycline (71.7)*
- oral rehydration salts (ors) sachets (73.7)
- amoxicillin tablet (76.3)
- paracetamol tablets (82.3)*

Note: * Medicines included in both Option A and Options B

Quintiles

How comparable relative categories are - with different sets of medicines? Let’s say “comparable” means being in the lowest quintile, middle three quintiles, or the top quintile.

Percent of facilities in comparable quintiles, by country and the list of tracer meds
approach	survey	all.meds	random.20.meds	random.10.meds	specific.10.meds.A	specific.10.meds.B
Simple sum	Haiti_2017	reference	90.3	81.2	79.9	90.3
Simple sum	Malawi_2013	reference	92.9	89.4	91.5	92.0
Simple sum	Tanzania_2014	reference	87.5	81.3	84.8	88.2

Percent of facilities in comparable quintiles, by country and the list of tracer meds
approach	survey	all.meds	random.20.meds	random.10.meds	specific.10.meds.A	specific.10.meds.B
PCA	Haiti_2017	reference	85.6	73.1	67.5	34.7
PCA	Malawi_2013	reference	80.9	72.3	68.6	66.9
PCA	Tanzania_2014	reference	88.7	64.8	47.0	30.2

Detailed look at rank

END OF NOTE