Table 3 Detailed judgments in Evidence to Decision (EtD) frameworks

Is the problem a priority?^a

• Are the consequences of the problem serious (i.e. severe or important in terms of the potential benefits or savings)?

• Is the problem urgent? [not relevant for coverage decisions]

• Is it a recognised priority (e.g. based on a political or policy decision)? [Not relevant when an individual patient perspective is taken]

How substantial are the desirable anticipated effects?

• Judgments for each outcome for which there is a desirable effect

How substantial are the undesirable anticipated effects?

• Judgments for each outcome for which there is an undesirable effect

What is the overall certainty of the evidence of effects?

• See GRADE guidance regarding detailed judgments about the quality of evidence or certainty in estimates of effects

Is there important uncertainty about or variability in how much people value the main outcomes?

• Is there important uncertainty about how much people value each of the main outcomes?

• Is there important variability in how much people value each of the main outcomes? [not relevant for coverage decisions]

Do the desirable effects outweigh the undesirable effects?

• Judgments regarding each of the four preceding criteria

• To what extent do the following considerations influence the balance between the desirable and undesirable effects:

 - How much less people value outcomes that are in the future compared to outcomes that occur now (their discount rates)

 - People’s attitudes towards undesirable effects (how risk averse they are)

 - People’s attitudes towards desirable effects (how risk seeking they are)

How large are the resource requirements?^a

• How large is the difference in each item of resource use for which fewer resources are required?

• How large is the difference in each item of resource use for which more resources are required?

What is the certainty of the evidence of resource requirements?^b

• Have all-important items of resource use that may differ between the options being considered been identified?

• How certain is the evidence of differences in resource use between the options being considered? (see GRADE guidance regarding detailed judgments about the quality of evidence or certainty in estimates)

• How certain is the cost of the items of resource use that differ between the options being considered?

• Is there important variability in the cost of the items of resource use that differ between the options being considered?

Are the net benefits worth the incremental cost?^a

• Judgments regarding each of the six preceding criteria

• Is the cost-effectiveness ratio sensitive to one-way sensitivity analyses?

• Is the cost-effectiveness ratio sensitive to multi-variable sensitivity analyses?

• Is the economic evaluation on which the cost-effectiveness estimate is based reliable?

• Is the economic evaluation on which the cost-effectiveness estimate is based applicable to the setting(s) of interest?

What would be the impact on health equity?^a,b

• Are there groups or settings that might be disadvantaged in relation to the problem or options that are considered?

• Are there plausible reasons for anticipating differences in the relative effectiveness of the option for disadvantaged groups or settings?

• Are there different baseline conditions across groups or settings that affect the absolute effectiveness of the intervention or the importance of the problem for disadvantaged groups or settings?

• Are there important considerations that should be made when implementing the intervention in order to ensure that inequities are reduced, if possible, and that they are not increased?

Is the intervention acceptable to key stakeholders?^a

• Are there key stakeholders that would not accept the distribution of the benefits, harms and costs?

• Are there key stakeholders that would not accept the costs or undesirable effects in the short term for desirable effects (benefits) in the future?

• Are there key stakeholders that would not agree with the values attached to the desirable or undesirable effects (because of how they might be affected personally or because of their perceptions of the relative importance of the effects for others)?

• Would the intervention adversely affect people’s autonomy?

• Are there key stakeholders that would disapprove of the intervention morally, for reasons other than its effects on people’s autonomy (e.g. in relation to ethical principles such as no maleficence, beneficence or justice)?

Is the intervention feasible to implement?^a

For decisions other than coverage decisions:

• Is the intervention or option sustainable?

• Are there important barriers that are likely to limit the feasibility of implementing the intervention (option) or require consideration when implementing it?

For coverage decisions:

• Is coverage of the intervention sustainable?

• Is it feasible to ensure appropriate use for approved indications?

• Is inappropriate use (indications that are not approved) an important concern?

• Is there capacity to meet increased demand if covered?

• Are there important legal or bureaucratic or ethical constraints that make it difficult or impossible to cover the intervention?