Table 1 Issues identified and suggestions offered by stakeholders of the user-testing study
Issues and Suggestions | Seriousness | Toolkit Development Team Actions |
|---|---|---|
Baseline Assessment Tool (BAT) | ||
Clearer instructions are needed to complete the BAT | XXX | We have written an instruction manual (Additional file 3) on how to use the toolkit. |
The purpose of the Recommendations spreadsheet and Outputs sheets were difficult to understand | XXX | We have revised the heading of this sheet to ‘Recommendations mapping’ to make its purpose clearer. Similarly, we have changed the headings of the Output sheets to be descriptive rather than numerical; for Output 1 (country-specific ANC package), we have removed the row requiring input on ‘Interpersonal support’ from each of the 8 contacts and, instead, inserted a single input column for these data to avoid repetition; for Output 2 (SWOT analysis), we have modified it from focusing on innovations to focusing on new and updated recommendations, which the Ministry of Health will have to coordinate. Additionally, we added a column for ‘ongoing implementation and research efforts’ and have highlighted that the SWOT analysis relates to any new and updated recommendations. |
In the French version, the links (conditional formatting) between the Situational analysis sheet and the Recommendations sheet are not functional | XXX | We have fixed this technical issue. |
The BAT does not capture ground level issues, e.g. related to minority populations or field workers | XXX | We have added a column to the Population statistics section (Item 2.8) of the situational analysis tool tab to provide for regional or population variations in the indicators; we have also added item 3.5 as follows: “Please describe any equity issues affecting health service coverage and quality”. Additionally, we have revised the SWOT analysis (Output 2) to better capture ground level issues. |
The formatting was frustrating and needs to be improved | XX | We have modified the formatting to make it more user friendly; however, we realize that Excel is not the best medium for the BAT and we are planning to convert it to a website/HTML format. |
Recommendations should be linked to an implementation plan | XX | Implementation guidance is the next step in the process; issues related to implementation are likely to be country specific and how to address them will depend on the toolkit outputs, which require finalisation after the stakeholder meetings. |
Some interventions and outcomes are missing from the Recommendations sheet | X | We have added these where relevant. |
The BAT would be better as a word processing document than a spreadsheet | X | We understand that word documents are helpful to some people; however, as they do not facilitate analysis, we believe that a spreadsheet format is justified. The planned HTML version of the toolkit will be more user friendly. |
Certain terminology is considered not widely used | X | We have clarified the problematic terms used or have provided alternative terms, e.g. community health worker or accredited social health activist. |
Slide document | ||
The logic models are quite complicated | XX | We have revised the logic model graphics to make them more accessible to users. |
Slides had a lot of information | XX | In the new instruction manual, we have suggested that the meeting organisers share the Slidedoc® (as a pdf booklet) with stakeholders prior to the meeting so they can have time to review the full contents of the Slidedoc® prior to the meeting’s presentation. |
Need to improve the French translation | X | We hired a professional translator to edit and improve the French version. |
Tailor pictures to setting or make the pictures more diverse | X | Organisers can substitute the pictures in the Slidedoc® as necessary to represent the national context, or conversely make the pictures more diverse to represent a global audience; we have modified the Slidedoc® in accordance with the latter. |
Stakeholder meetinga | ||
The meeting was too short | NA | Organisers should consider this suggestion, depending on the time and resources available. |
Groups were too big | NA | Organisers could consider smaller groups, e.g. a group size of 10 instead of 20. This would mean that there would be four groups instead of two; as the feedback session would take longer (while each group presents their results) this would have implications for the meeting duration. If smaller groups are preferred, ensure that a selection of different stakeholders are represented within each group. |
Have another room available for group work | NA | Organisers should consider this suggestion, as having adequate space for group work is important to facilitate open discussions. |
Grass roots issues should be discussed | NA | Organisers should ensure that implementers and service users are among the invited stakeholders and that their voices are heard in the discussions. Suggestions for representatives to be invited are detailed in the instruction manual. |
Include implementation case studies and experiences from other countries | NA | We have now included the experiences of two country case studies in this paper (Boxes 1 and 2) and organisers might wish to refer to these in their presentations. |
Presentations could be shortened | NA | Organisers might prefer to keep presentations brief to maximise the time for group work. |