Assessment at the macro level: using the CHIS assessment tool¶
A short example can be used to highlight how the assessment tool can be used for assessing the macro, policy level.
A country is in the process of examining its community health services, and in that regards wants to assess the strengths and weaknesses of the overall CHIS before planning specific interventions. The first step they take is to form a steering committee that will oversee the assessment and analyse the results. The steering committee conducts a stakeholder identification exercise to see who should take part in the assessment, and aiming for a rather small group due to logistical reasons, the identify the following.
A community health worker and a representative from a community health committee are selected to bring in the viewpoint of the community itself. They are selected since they have experience with how the data collection and feedback is working between the CHW and the facilities to which they report, and how the CHW interacts with the community at large.
An information officer from a facility is included since this person handles the flow of data between the CHWs and the wider health management information systems, as well as having knowledge of the facilities' challenges with medical stock distribution.
Each district has a Community Health program officer, and the participation of one of them is secured to get the input from this level. This person will, in addition to managing the community health services in the district, also liaise between the CHWs and the other health program managers that are based in the district headquarters, so this is considered adequate participation from the district level.
From the national level, the head of the community health programme, as well as the IT technician, is joining. Finally, a representative from a large international NGO, who are supporting the CHWs in some of the districts, is invited.
Conducting the assessment¶
The steering committee, consisting of staff at the community health programme, then invites the identified stakeholders to a one-day workshop in the capital. Due to logistical challenges, the community, facility, and district representatives are selected from the same district, though the group would ideally bring in representatives from at least two districts. However, the NGO agrees to invite one more CHW from another district as they are piloting new processes around CHW reporting there.
The group goes through the tool question by question in plenary, since they are not so many. This is also a chance for those who are familiar with the issue in question to inform the others on the team.
An example of how they answered one of the questions now follows. The group is discussing the question: Do traditional health providers report through the CHIS? The scoring scheme for this question is:
| Highly adequate | Adequate | Present but not adequate | Not adequate at all | | Items | 3 | 2 | 1 | 0 | |:-|:-|:-|:-|:-| | Do traditional health providers report through the CHIS? | Yes, all traditional health providers working at the community level report their activities to the national CHIS. | Most traditional health providers working at the community level report their activities to the national CHIS. | Some traditional health providers working at the community level report their activities to the national CHIS. | Traditional health providers are not part of the CHIS. |
First, the head of the community health program informs the team that, no, this is not taking place. They have tried to incorporate data from traditional birth attendants (TBA) in the CHW reporting, but due to the lack of formalized acknowledgment and collaboration, the TBAs see little incentive in reporting neither to the CHWs or the facilities. This is something they would consider, but for now they would need to give this question a score of 0.
However, both the attending CHWs can then attest to that, sometimes, they do collect data from the TBAs. They know at least one TBA themselves, who is based in one of the villages they serve. They have a good relationship, and sometimes communicate with the TBA to confirm a visit there since there is no community health committee in this particular village. When they so do, they can inform that they get relevant data on the TBA activities, and include in their reporting if this is appropriate. However, one of the CHWs says that this is included, and thus part of, her report, while the other says she is adding it on the side of the paper as performed by the TBA.
The team then decides that a score of 1 is appropriate for this question, but make a note in the scoring sheet that how this TBA reporting is done varies and is not standardized at all.
Interpreting the results¶
And so they go through the questions of the assessment tools. In the end, they have a list of scores for all the questions, as well as a number of notes clarifying certain issues. The scoring will itself highlight the strengths and weaknesses of the CHIS, but the steering committee is then charged with further analyzing the data and deciding on the priorities. For instance, since they don't have electronic reporting from CHWs, they realize an improvement in score on automatic reminders when reporting is slow is far away. So even if they set the score to 0 here, this is not seen as a major weakness that needs to be addressed at the present moment.
A report with the findings and such interpretations of the results is then shared with the team who conducted the assessment, to get feedback. They decide to use a three-level prioritization scheme (high, medium, low priority) for all of the questions, and present it at the next executive meeting for all health programs, as well as sharing it with health partners.