Critical appraisal for a Randomized trial

Type:

Individual Clinical Study

<Author Last Name First and Middle Initial, et al Year.[ref] Title of Study ( STUDY NAME). >

Example:

Cortes JE, et al. 2018 [1] Randomized comparison of low dose cytarabine (LDAC) with or without glasdegib in patients with newly diagnosed acute myeloid leukemia or high-risk myelodysplastic syndrome.

Appraisal Conclusions: <Provide summary results of the primary endpoint.> This study was appraised as a high quality study as there were no major or moderate concerns. OR This study was appraised as low quality due to one major study concern: high differential attrition. < To note: First reference of brand name include brand (generic) here, then refer to brand name only for FDA-approved strengths and generic only for non-FDA approved strengths.>

Study Grade:

(Refer to Appendix E for Reference Tool)

< High, Moderate, Low>

Appraisal Elements

Description

Concerns that Impact Confidence in Results or Quality

Funding Source

<Manufacturer or Funding Source>

Example:

Eli Lilly

<Minor: Manufacturer-funded. Conflict of interest and bias cannot be ruled out.>

< Moderate: The manufacturer retained final control of the data and/or manuscript. Potential for bias.>

< Moderate: Funding source was not disclosed.>

Drug/Condition

<Brand Name> <vs. placebo> in <disease>

Example:

Doptelet vs. placebo in adults with thrombocytopenia associated with liver disease prior to an elective procedure

< Moderate: The comparator is not appropriate, treatment effect may be biased and favor intervention.>

< Moderate: The drug dose or formulation is not currently FDA-approved.>

Study Design

<Phase 1, 2, or 3, global, multicenter, randomized, double-blind, double-dummy, placebo-controlled, parallel group study>

< Moderate: Open-label study with objective or semi-objective endpoint. Bias cannot be ruled out.>

< Moderate: Single-arm study with no comparator. Magnitude of benefit cannot be determined.>

< Major: Open-label study with subjective or unvalidated endpoint. Likely potential for bias.>

< Major: Non-randomized study; magnitude of benefit is unknown.>

1° Endpoint

· <Describe>

<Minor: Semi-objective endpoint; bias cannot be ruled out.>

< Moderate: Unvalidated, clinically-relevant, objective or semi-objective endpoint; potential for bias.>

< Moderate: Subjective, clinically-relevant, validated endpoint; potential for bias.>

< Moderate: The endpoint is a patient-reported outcome that may be subject to bias.>

< Moderate: Composite endpoint may lack transparency in reporting key measures; potential for bias.>

< Major: Endpoint determined post-hoc; likely potential for bias.>

< Major: Endpoint is not relevant to the outcomes of interest. Benefit cannot be determined. Likely potential for bias.>

< Major: Unvalidated, subjective, clinically-relevant endpoint; likely potential for bias.>

Significant 2° Endpoints

· <Describe>

· <Describe>

None

Population Size

<# of patients>

<Minor: Small sample size; bias cannot be ruled out.>

< Moderate: Results may not be generalizable to the greater population; potential for bias.>

Significant Population

Characteristics

Inclusion Criteria:

· <Describe>

· <Describe>

Exclusion Criteria:

· <Describe in bullets or commas if more than 5 items to save space>

· <Describe>

<Minor: Canadian or European study; bias cannot be ruled out.>

<Minor: Results may not be generalizable to the greater population; bias cannot be ruled out.>

< Moderate: Baseline characteristics significantly different between treatment groups; potential for bias.>

< Moderate: The population includes individuals outside the United States; bias cannot be ruled out.>

< Moderate: Results may not be generalizable to the greater population; potential for bias.>

Treatment Allocation

· Brand Name, Dose, Route of Administration, Duration

· Placebo or Comparator (if applicable), Dose, Route of Administration, Duration

< Moderate: Treatment dose/duration is inappropriate for study endpoint; potential for bias.>

< Moderate: Blinding may be compromised; potential for bias.>

< Major: Treatment dose/duration is inappropriate for study endpoint; likely potential for bias.>

< Major: Blinding may be compromised; likely potential for bias.>

Efficacy Results

<Table out the results: use the colors seen below; calculate NNT, if appropriate>

<Minor: Magnitude of benefit cannot be determined due to single-arm trial design.>

< Moderate: Efficacy results are reported; however, significance of results is unknown. Potential for bias.>

< Major: Statistical test was inappropriate/incomplete for dataset; likely potential for bias.>

< Major: Efficacy results are not reported; likely potential for bias.>

Safety Results

<Describe most common adverse events, serious adverse events, adverse events leading to treatment discontinuation: Max 2-4 sentences>

<Minor: Safety results are not reported; bias cannot be ruled out.>

Analysis Type

<Describe: Max 1-2 sentences>

Example:

The population used for efficacy analyses included a full analysis set of all patients who were randomized. Of the 231 patients randomized, 231 patients made up the full analysis dataset.

Example:

Efficacy analyses were performed in the intention-to-treat population. 132/132 patients (100%) were evaluated for the primary endpoint.

Example:

A modified intent-to-treat (ITT) analysis was employed for primary efficacy superiority analyses, which included all patients treated with ≥ 1 dose of study drug (4 patients were excluded). Example:

Analysis type was not reported.

<Minor: Up to 5% of patients excluded from primary efficacy analysis; bias cannot be ruled out.>

< Moderate: A Per-Protocol (PP) analysis and Intention-to-Treat (ITT) analysis are not performed for the non-inferiority trial; potential for bias.>

< Moderate: 5% to 20% of patients excluded from the primary efficacy analysis; potential for bias.>

< Major: A Per-Protocol (PP) analysis is not performed for the non-inferiority trial; potential for bias.>

< Major: At least 20% of patients excluded from the primary efficacy analysis; likely potential for bias.>

< Major: Analysis type was not reported.>

Subject Disposition

<Describe: Max 1-2 sentences>

Overall subject disposition:

Differential subject disposition:

Example:

Overall subject disposition: 6/258 patients (2.33%) prematurely discontinued the study.

Differential subject disposition: 2/83 patients (2.41%) in the Zeposia 1.0 mg group vs. 2/87 (2.30%) of patients in the Zeposia 0.5 mg group vs. 2/88 patients (2.27%) in the placebo group discontinued the study.

Example:

Subject disposition was not reported.

<Minor: Moderate overall attrition; however, it was accounted for in trial methodology. Bias cannot be ruled out.>

To note: add one overall level of concern: select the most severe concern.

< Moderate: Subject disposition is not reported; results are not likely to favor the intervention; potential for bias.>

< Moderate: Moderate differential attrition; potential for bias.>

< Moderate: High overall attrition; however, it was accounted for in trial methodology. Potential for bias.>

< Moderate: Moderate overall attrition; potential for bias.>

< Major: Subject disposition is not reported; results are likely to favor the intervention; likely potential for bias.>

< Major: High differential attrition; likely potential for bias.>

< Major: High overall attrition; likely potential for bias.>

Power

<Describe: Max 1-2 sentences>

Example:

100 patients in each cohort were required to achieve 90% power to detect an absolute difference of 35% between the Doptelet response rate and the projected 18% placebo response rate.

Example:

Power was not reported.

None: Results are statistically significant.

< Major: Power was not reported; sample size may not be sufficient to achieve 80% power; likely potential for bias.>

< Major: Sample size was not sufficient to achieve 80% power; likely potential for bias.>

< Major: Non-inferiority margin is too large to detect a difference; likely potential for bias.>

Randomization

<Describe: Max 1-2 sentences >

Example:

Patients were randomized in a 4:3:4 ratio to one of three treatment arms: baricitinib + MTX, baricitinib monotherapy, or MTX monotherapy. Randomization was stratified by geographic region and presence or absence of joint erosions on centrally read baseline radiographs.

<Minor: Randomization methodology cannot be appraised; bias cannot be ruled out.>

< Moderate: Single arm study; magnitude of benefit is unknown.>

< Moderate: Randomization methodology does not mitigate discrepancies in baseline characteristics; potential for bias.>

< Major: Non-randomized study; magnitude of benefit is unknown.>

Blinding

<Describe: Max 1-2 sentences >

Example:

Investigators and patients were not blinded to treatment.

Example:

A double-blind method was used for retapamulin versus placebo comparison.

<If open label study: See study design>

< Moderate: Open label study with objective or semi-objective endpoint. Bias cannot be ruled out.>

< Moderate: Blinding may be compromised; potential for bias.>

< Major: Open label study with subjective or unvalidated endpoint. Likely potential for bias.>

< Major: Blinding may be compromised; likely potential for bias.>

Other Key Study Concerns

<Describe: Max 1-2 sentences>

Example:

Study is not published. Data are derived from the prescribing information or Clinical Trials.gov.

Example:

Concomitant treatment with stable doses of nonsteroidal anti-inflammatory drugs (NSAIDs), analgesics, and/or oral corticosteroids (≤ 10 mg/day of prednisone or equivalent) was permitted.

< Moderate: Study protocol was amended ad-hoc; results not likely to be impacted. Potential for bias.>

< Moderate: Unpublished study; however, methodology can be appraised. Potential for bias.>

< Moderate: Concomitant medications likely confound results.>

< Moderate: Washout periods may be inadequate to mitigate bias.>

< Major: Study protocol was amended ad-hoc; results are likely to be impacted. Potential for bias.>

< Major: Trial was ended prematurely; likely potential for bias.>

< Major: Unpublished study; methodology cannot be appraised. Likely potential for bias.>