Table 3 An example of an approach to the formative evaluation of policy dialogues

• Dialogues are characterised by twelve features and these are the focus of the questions in the formative evaluation survey. A dialogue:

   • Addresses a high-priority issue

   • Provides an opportunity to discuss different features of the problem, including (where possible) how these affect particular groups

   • Provides an opportunity to discuss three options to address the policy issue

   • Provides an opportunity to discuss key implementation considerations

   • Provides an opportunity to discuss who might take what action

   • Is informed by a pre-circulated evidence brief that mobilises both global and local research evidence about the problem, three options to address the problem, and key implementation considerations

   • Is informed by a discussion about the full range of factors that can inform how to approach the problem, options to address the problem, and the implementation of these options

   • Brings together many parties who would be involved in, or affected by, future decisions related to the issue

   • Ensures fair representation among policymakers, stakeholders, and researchers

   • Engages a facilitator to assist with the deliberations

   • Allows for frank, off-the-record deliberations by following the Chatham House Rule: "Participants are free to use the information received during the meeting, but neither the identity nor the affiliation of the speaker(s), nor that of any other participant, may be revealed", and

   • Does not aim for consensus

• For each design feature the survey asks:

   • How useful did they find this approach on a scale ranging from 1 (Worthless) to 7 (Useful)?

   • Comments and suggestions for improvement?

• The survey also asks:

   • How well did the policy dialogue achieve its purpose, namely to support a full discussion of relevant considerations (including research evidence) about a high-priority issue in order to inform action on a scale from 1 (Failed) to 7 (Achieved)?

   • What features of the dialogue should be retained in future?

   • What features of the dialogue should be changed in future?

   • What others can do better or differently to address the high-priority issue and what they personally can do better or differently?

   • Their role and background (so that the McMaster Health Forum can determine if different groups have different views about, and experiences with, the dialogues)

• The McMaster Health Forum also plans to conduct brief follow-up surveys six months after a dialogue, with the objective of identifying what, if any, actions have been undertaken by dialogue participants and what, if any, impacts have been achieved. Here again, participation is voluntary, confidentiality is assured, and anonymity safeguarded

• The Evidence-Informed Policy Networks (EVIPNet) operating in Africa, Asia and the Americas plan to use a similar approach in the formative evaluation of their policy dialogues