Table 8 Methods evaluated based on properties of good policy support
From: From insight network to open policy practice: practical experiences
Method | Quality of content | Relevance | Availability | Usability | Acceptability | Efficiency |
|---|---|---|---|---|---|---|
Co-creation | + Participants bring new info (2, 3, 25, 26) | + Additional file 1: New questions are identified during collaborative work (6, 11) | + Draft results raise awareness during work (2, 8, 27) | ? Readers ask clarifying questions and learn and create understanding through collaboration | + Participants are committed to conclusions (2, 8, 27) | ? Collaboration integrates communication to decision-makers and stakeholders (users) into the making, which saves time and effort |
Open assessment | + It combines functionalities of other methods and enables peer-reviewed assessment models (4, 5, 16) | + End-user discussions improve assessment (16, 26, 27) | ? It is available as draft since the beginning | + Standard structure facilitates use (8, 9) | + Openness was praised (3, 8, 9, 21) | + Scope can be widened incrementally (12–16) |
Insight network | + It brings structure to assessment and helps causal reasoning (8, 9, 11, 16, 17) | + It helps and clarifies discussions between decision-makers and experts (8, 9) | - | ? Readers see what is excluded | ? It helps to check whether important issues are missing | - |
Knowledge crystal | + They streamline work and provide tools for quantitative assessments (e.g. 3, 23, 24) | + They clarify questions (1, 6) | ? It is mostly easy to see where information should be found | ? Summaries help to understand | ? They make the intentionality visible by describing the assessment question | + Answers can be reused across assessments (12–16, 23–24) |
Web-workspace | + Its structure supports high-quality content production when moderated (8, 9) | + It combines user needs and open policy practice (8, 9) | + It offers an easy approach to and archive of materials (16, 21, 23, 26) | + The user needs guided the functions developed (8) | - | ? It offers a place to document shared understanding and distribute information broadly |
Structured discussion | + It helps to moderate discussion and discourages low-quality contributions (2, 30) | + It guides focus on important topics (16, 30) | - | ? Threads help to focus reading | + User feedback has been positive: it helps to focus on key issues (8, 30) | ? Structure discourages redundancy |
Open policy ontology | - | + It gives structure to insight networks and structured discussions (8, 16, 30) | - | ? Ontology clarifies issues and relations | - | - |
Value profile and archetype | - | + Value profiles help to prioritise (8) | - | ? Voting advice applications may offer an example | ? Stakeholders’ values are better heard | ? Archetypes are effective summaries |
Paradigm | ? It motivates clear reasoning | ? It systematically describes conflicting reasonings | - | - | ? Stakeholders’ reasonings are better heard | ? It helps to analyse inferences of different groups |
Analysis of destructive policies | - | + It widens the scope (3, 8) | - | ? It emphasises mistakes to be avoided | ? Focus is on everyone’s problems | ? Lessons learned can be reused in other decisions |
Suggestions by open policy practice | Work openly, invite criticism; use tools and moderation to encourage high-quality contributions (Table 1) | Acknowledge the need for and potential of co-creation, discussion, and revised scoping; invite all to policy-support work; characterize the setting (Table 4) | Design processes and information to be open from the beginning; use open web-workspaces (Table 5) | Invite participation from the problem owner and user groups early on; use user feedback to visualise, clarify and target content (Table 6) | Be open; clarify reasoning; acknowledge disagreements; use the test of shared understanding (Table 2) | Combine information production, synthesis and use to a co-creation process to save time and resources; use shared information objects with open license, e.g. knowledge crystals |