Expert Opinion

(1)

Dal grading al GRADE

Nicola Magrini

Area Valutazione del Farmaco

Agenzia Sanitaria e Sociale Regionale, Bologna

WHO Collaborating Centre for Evidence Based Research Synthesis and Guideline Development

(2)

1. Linee-guida e raccomandazioni

evidence-based: definizioni e ruolo

(3)

Definizione di linea-guida e raccomandazioni

• "Guidelines are recommendations intended to assist providers and recipients of health care and other stakeholders to make informed decisions.

Recommendations may relate to clinical interventions, public health activities, or government policies."

WHO 2003, 2007

(4)

Distinguere fra qualità delle evidenze e forza delle raccomandazioni

• Si tratta di due informazioni complementari in buona parte indipendenti

• La prima valuta la qualità metodologica delle prove disponibili per stimare gli effetti del

trattamento (benefici e rischi)

• La forza considera:

– la fattibilità, la trasferibilità

– i benefici e rischi attesi e la loro rilevanza

– le implicazioni organizzative, economiche, sociali e finanziarie (quindi rispetto al contesto) dell’interventi

(5)

Che cosa è la qualità delle evidenze in una linea-guida

In the context of making recommendations:

• The quality of evidence reflects the extent of our confidence that the estimates of an effect are

adequate to support a particular decision or

recommendation.

(6)

Forza della raccomandazione

“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable

effects of a management strategy outweigh undesirable effects.”

• Strong or conditional

• Forte e …

(7)

Vecchi sistemi di grading

• La qualità delle evidenze rifletteva

sostanzialmente il disegno degli studi e la loro conduzione

• La forza delle raccomandazioni dipendeva dal disegno e in parte dalla qualità metodologica degli studi … quindi …

• Ridondanze e ripetizioni da un lato ma anche mancate considerazioni della entità dell’effetto, della eterogeneità tra gli studi e della

trasferibilità dei risultati …

(8)

(9)

Raccomandazioni e evidenze

• Per poche raccomandazioni chiave:

– Reflect and discuss on relevant outcomes

– Search for and retrieve all available evidence – Identify relevant SRs

– Formally assess quality of evidence

– GRADE (systematic and transparent approach)

– But also: Panel composition and role of chair

– Manage CoI

(10)

The scope

• Small is beautiful (S. Hill)

• Who is the target user of the guideline

• Who it applies to

• What is covered?

– Eg diagnosis and treatment of diabetic retinopathy

• Develop key questions (<10-20…..)

(11)

2. Elaborazione di linee-guida

Il gruppo di lavoro (Panel)

(12)

How to improve guideline production

Attuali limiti:

• Governance and composition of the guideline committee

(“what is to be decided is often already decided with the selection of the deciders”)

• Unanimity in guideline

(not a natural component in research)

• Lack of independent review

(outside the accepted procedures of scientific publications)

• CoI

(13)

Group composition

• One systematic review (Murphy et al. 1998)

• Composition of panel influences recommendations

– Members of a specialty are more likely to advocate techniques that involve their specialty

• Balanced groups

– Select the appropriate group leader

• Necessary technical skills

– including information retrieval, systematic reviewing, health economics, group facilitation, project management, writing and editing

• Include or have access to content experts

• No SR on how to obtain consultation, but logical reasons support this

• Up to 15 members

(14)

Composizione del GdL/Panel

• „Include all who are affected“

- To identify the right questions

- To identify areas of suboptimal care

- To identify feasibility of recommendations

• Consequences

- Definition of Standards of Care

- Ownership to improve implementation

(15)

Composizione del GdL/Panel in RER

• Cosa significa includere i competenti/responsabili …. Non esattamente „all who are affected“

- Andare verso gruppi di competenti (non necessariamente i responsabili/primari)

- Gruppi stabili versus gruppi ad hoc

- Separare meglio fase di elaborazione da quella di disseminazione introduzione /implementazione? Quale formazione specifica?

• Consequences

- Alta variabilità tra gruppi e tra prodotti

- Rapporti con istituzioni (e Soc. Scient.) variabili

(16)

Expertise needed in the group

• Medical content:

health care professionals

• Values and preferences:

patients / carers / community

• Support staff /Secretariat:

metodologi, clinici part-time (epidemiologists, health economists), … administrative support

(17)

3. Perché il GRADE

(18)

Which approach?

Evidence Recommendation

• B Class I

• A 1

• IV C

Organization

AHA

ACCP

SIGN

Recommendation for use of oral anticoagulation

in patients with atrial fibrillation and rheumatic

mitral valve disease

(19)

LG e sintesi evidenze

• La parte di qualità e sintesi delle evidenze

– Essenziale e importante

– Puo essere commissionata esternamente a un gruppo o dalla segreteria scientifica di supporto – Non coinvolge direttamente il Panel (se non per

commenti e integrazioni)

– È certamente la piu lunga e impegnativa assieme

alla stesura del testo finale

(20)

Study design is important

Early systems of grading the quality of evidence focused almost exclusively on study design

Randomised trials provide, in general, stronger evidence than observational studies:

–RCTs start at High Quality

–Observational studies start at Low Quality

However, other factors may decrease or increase

the quality of evidence

(21)

Quality assessment criteria

Quality of evidence

Study design Lower if Higher if

High Randomised trial

Moderate

Low Observational

study Very low

Study quality:

-1 Serious limitations -2 Very serious limitations

-1 Important inconsistency

Directness:

-1 Some

uncertainty -2 Major

uncertainty

-1 Sparse or imprecise data

Strong association:

+1 Strong, no plausible

confounders +2 Very strong, no major threats to validity

+1 Evidence of a Dose response gradient

(22)

Factors that may decrease the quality of evidence

Study limitations (risk of bias)

Inconsistency among studies

Indirectness of evidence

Imprecise results

Reporting bias

(23)

Vecchie gerarchi troppo semplici:

l’importanza della competenza clinica

STUDY DESIGN

Randomized Controlled Trials

Cohort Studies and Case Control Studies

Case Reports and Case Series, Non-systematic observations

BIAS

E xp e rt O p in io n

(24)

Strong / weak recommendation Quality

assessment criteria

Ratings of outcomes

Quality assessment

criteria

Quality of evidence:

estimates of benefits &

harms and risk of bias,

Risk-benefit profile

evaluation for different subgroups

Evidence

Emilia Romagna New Cancer Drugs Rec.

GRADE steps and polls

• We (in Italy) adapted GRADE by:

– Rating/voting also the evaluation of the benefit-risk profile (favourable, uncertain, unfavourable)

– Defining an expected use indicator

– Incorporating the results of polls in the final document

(25)

4. dalle evidenze alla

raccomandazione

(26)

Getting from evidence to recommendations - GRADE

Recommendations are judgments:

– Quality of evidence

– Trade off between benefits and harms – Values and preferences

– Resource use

But judgments need to be based on the best

available evidence and transparent

(27)

Che cosa determina la direzione e la forza di una raccomandazione

Factors that can strengthen a recommendation

Comment

Quality of the evidence The higher the quality of evidence, the more likely is a strong

recommendation.

Balance between desirable and undesirable effects

The larger the difference between the desirable and undesirable

consequences, the more likely a strong recommendation warranted. The

smaller the net benefit and the lower certainty for that benefit, the more likely weak recommendation warranted.

Values and preferences The greater the variability in values and preferences, or uncertainty in values and preferences, the more likely weak recommendation warranted.

Costs (resource allocation) The higher the costs of an intervention

(28)

• Desirable outcomes

– lower mortality

– reduced hospital stay

– reduced duration of disease – reduced resource expenditure

• Undesirable outcomes

– adverse reactions – costs of treatment

• Every decision comes with desirable and undesirable consequences

Scelta degli outcomes …

prima di leggere gli studi

(29)

GRADE:

recommendation – quality of evidence

Clear separation:

1) 4 categories of quality of evidence: ⊕⊕⊕⊕

^(High)

,

⊕⊕⊕ 

^(Moderate)

, ⊕⊕ 

^(Low)

, ⊕ 

^{(Very low)}

?

– methodological quality of evidence – likelihood of bias

– by outcome and across outcomes

2) Recommendation: 2 grades – conditional (aka weak) or strong (for or against an intervention)?

– Balance of benefits and downsides, values and

(30)

Che cosa è la qualità delle evidenze in una linea-guida

In the context of making recommendations:

• The quality of evidence reflects the extent of our confidence that the estimates of an effect are

adequate to support a particular decision or

recommendation.

(31)

Forza della raccomandazione

“The strength of a recommendation reflects the extent to which we can, across the range of patients for whom the recommendations are intended, be confident that desirable

effects of a management strategy outweigh undesirable effects.”

• Strong or conditional

• Forte e …

(32)

Implications of

a strong recommendation

• Patients: Most people in this situation would want the recommended course of action and only a small proportion would not

• Clinicians: Most patients should receive the recommended course of action

• Policy makers: The recommendation can be adapted

as a policy in most situations

(33)

Implications of

a conditional/weak recommendation

• Patients: The majority of people in this situation

would want the recommended course of action, but many would not

• Clinicians: Be more prepared to help patients to make a decision that is consistent with their own values/decision aids and shared decision making

• Policy makers: There is a need for substantial

debate and involvement of stakeholders

(34)

6. How to improve transparency in going

from evidence to recomendations

(35)

Contesto, evidenze, quesito, esiti

Raccomandazione

Entità dei benefici e dei rischi

Rapporto beneficio/rischi Accettabilità e preferenze

Costi e uso risorse

(36)

Going from evidence to recommendation

risk benefit profile, values and preferences, costs (1/3)

(37)

Going from evidence to recommendation

risk benefit profile, values and preferences, costs (2/3)

(38)

Going from evidence to recommendation

risk benefit profile, values and preferences,

costs and fesibility (3/3)

(39)

7. Il processo di formulazione della raccomandazione

I fattori legati al Panel e al contesto

… e il CoI

(40)

Il processo: come renderlo costruttivo e trasparente?

• Group members are heterogeneous and might have different objectives

• Chair facilitates rather than leads the group

• Common understanding of goal, tasks and ground rules

• Similar level of required know-how and skills

• Sufficient technical support in research synthesis

(41)

Panel: balanced participation and formal agreements

• Key task of chair

• Drafting of manuscript

• Formal consensus processes -

Delphi Method

- Nominal group process

(42)

How to present controversies

• Lay out the controversies

• Describe the evidence

• Ask members to focus on the agreed upon

evidence and the factors leading to a decision

• Ask whether there still is disagreement

• Vote

– Make voting explicit and transparent (ways of

(43)

Esempi di raccomandazioni che hanno usato il GRADE

Examples of flexibility and

transparency

(44)

LG RER 2012 e GRADE

• GREFO farmaci oncologici

• diabete IGEA, Incretine,

• Farmaci biologici (anti-TNF, …) molti, vari e diversi

• Diabete gestazionale,

• Gravidanza fisiologica In corso:

• Nuovi anticoagulanti?

• … nuovi farmaci anti HCV,

(45)

LG RER 2012 e GRADE …

se siete interessati potremmo farne 1 alla volta

1. GREFO farmaci oncologici 2. diabete IGEA,

3. Incretine,

4. Farmaci biologici (anti-TNF, …) molti, vari e diversi 5. Diabete gestazionale,

6. Gravidanza fisiologica

7. Farmaci antiaggreganti IMA/SCA pre e in-hospital In corso:

8. Nuovi anticoagulanti?

9. … nuovi farmaci anti HCV,

10. … anche altri temi NON farmacol

(46)

GRADE ed esperienze RER/ITA

• Via via che su alza il livello di committenza della LG, più il

GRADE sempre essere utile … come strada e linguaggi comuni (fornisce un buon framework)

• MA in RER …

• Sembra mancare spesso un interesse dei Panelisti (vi è anzi il timore) a trovare una posizione nuova e originale