How to read a forest plot

from https://s4be.cochrane.org/blog/2016/07/11/tutorial-read-forest-plot/

Purpose of Forest Plots

• Summarize Data: Forest plots graphically summarize data from multiple studies addressing the same question.
• Facilitate Comparison: They allow for the comparison of results across studies, providing an overall picture of the evidence, even if individual studies differ in findings.
• Identify Patterns: They help detect patterns, overall trends, and the consistency of results (heterogeneity) across studies.

Key Components of a Forest Plot

1. Axes

• Horizontal Axis:
  • Represents the effect size or statistic measured in each study (e.g., Odds Ratio [OR], Relative Risk [RR], Absolute Risk Reduction [ARR], Standardized Mean Difference [SMD]).
  • The statistic displayed can be relative (e.g., OR, RR) or absolute (e.g., ARR, Mean Difference). The choice affects the null value on the plot.
• Vertical Line (Line of Null Effect):
  • Relative Statistics (e.g., OR, RR): Null effect = 1 (indicates no association or effect).
  • Absolute Statistics (e.g., ARR, Mean Difference): Null effect = 0 (indicates no difference).

2. Study Representation

1. Components of a Study Line

• Black Box (Point Estimate):
  • Represents the primary result or effect size of the study (e.g., relative risk, odds ratio).
  • Size of the Box: Proportional to the study’s sample size.
    • Larger Box: Indicates a study with more participants (greater weight in the analysis).
    • Smaller Box: Represents a study with fewer participants (less weight in the analysis).
• Horizontal Line (95% Confidence Interval [CI]):
  • The line extends horizontally from the point estimate and represents the range within which we are 95% confident the true effect size lies.
  • The ends of the line mark the boundaries of this range.

2. Interpreting Study Lines Relative to the Null Effect Line

• Null Effect Line:
  • This is the vertical line on the forest plot.
  • The position of the null effect value depends on the type of statistic:
    • Relative measures (e.g., OR, RR): Null value = 1 (indicating no difference).
    • Absolute measures (e.g., ARR, mean difference): Null value = 0.
• Position of Study Lines:
  • To the Left or Right of the Null Line:
    • Indicates whether the study favors the intervention or the control.
    • Interpretation depends on the context (e.g., reducing risk, increasing likelihood of an event).
  • Crossing the Null Line:
    • If a study’s horizontal line crosses the null effect line, it includes the null value in its 95% CI.
    • Not Statistically Significant: Crossing the null line implies that the result might not be different from the null (no effect).

3. Importance of Study Size and Precision

• Bigger Studies (More Participants):
  • Typically have narrower confidence intervals.
  • Represented by a larger black box and shorter horizontal line.
  • These studies are less likely to cross the null effect line, indicating a higher likelihood of statistical significance.
• Smaller Studies (Fewer Participants):
  • Usually have wider confidence intervals.
  • Represented by a smaller black box and longer horizontal line.
  • More likely to cross the null effect line, reflecting greater uncertainty and reduced statistical significance.

Summary Points

• Black Box Size: Indicates sample size (bigger box = larger study).
• Horizontal Line (95% CI): Reflects precision (narrower line = greater precision).
• Crossing the Null Line: Implies lack of statistical significance.
• Relative vs. Absolute Measures: Null effect line value differs based on the type of statistic (1 for relative measures, 0 for absolute measures).

3. The Diamond

• Diamond Shape at the Bottom of the Plot:
  • diamond is the most important aspect of a forest plot.
  • Represents the combined result of all studies.
  • Vertical Points of the Diamond:
    • Represents the combined point estimate.
  • Horizontal Tips of the Diamond:
    • Represents the 95% confidence interval for the combined result.
  • Interpretation:
    • If the diamond’s CI crosses the line of null effect, the combined result is not statistically significant.
    • This indicates that, when considering all data, there is still a possibility that there is no effect.

4. Author and Year of Studies:

• Purpose: The far-left column of a forest plot typically lists the name of the lead author of each included study, alongside the year of publication.
  • Importance: This provides transparency regarding the source of the data being summarized. It allows for an assessment of study recency and context, helping reviewers gauge whether findings are potentially outdated or highly relevant to current practice.
  • Context: Including these details ensures traceability, facilitating further exploration of the original studies if necessary.

5. Event and Total Columns (n/N Format):

• Description: Two columns immediately to the left of the plot present data in an “n/N” format:
  • ‘n’ (Event Count): Represents the number of individuals in a group (treatment or control) who experienced the specified outcome.
  • ‘N’ (Total Count): Indicates the total number of participants in that group.
  • Example Context: The first column usually refers to the treatment group, showing the proportion of treated individuals who had the outcome versus the total treated. The second column refers to the control group with analogous data.
• Usefulness:
  • This arrangement allows for a quick, numerical comparison of event rates between the treatment and control groups, providing context for the graphical representation of results in the plot.
  • It offers insight into absolute event rates and the relative size of each study’s groups, adding weight to the interpretation.

6. Figure 7 – Point Estimate and 95% Confidence Interval (Right Column):

• Description: The far-right side of the forest plot provides numerical values for each study’s point estimate (e.g., risk ratio, odds ratio) and the corresponding 95% confidence interval (CI).
  • Point Estimate: This is a measure of effect size or association from each individual study.
  • 95% CI: Indicates the range within which we are 95% confident the true effect lies. A narrow CI suggests precision, while a wider CI indicates more uncertainty.
• Importance:
  • This column serves as a numerical summary of what is depicted graphically by the lines and boxes in the forest plot, offering an alternative, detailed view for those who prefer to analyze numbers directly.
  • The presence of this data also helps in identifying studies with high variability or those with stronger precision around their effect estimates.

7. Subtotal and Summary Statistics (Diamond Representation):

• Description: The diamond shape at the bottom of the forest plot represents the pooled estimate of effect across all included studies. Key elements highlighted in this section include:
  • Subtotal Line: Displays the cumulative totals of participants in the treatment and control groups across all studies.
  • Diamond Center: Represents the overall effect size (point estimate) derived from pooling all studies.
  • Diamond Width: Reflects the 95% CI around the pooled estimate.
• Utility:
  • Provides a visual summary of the overall intervention effect across all studies.
  • Enables a quick assessment of statistical significance and direction of the overall effect.
  • The boldness of the summary statistics underscores their role in drawing conclusions about the effectiveness or harm of the intervention being studied.

8. Heterogeneity of Studies (Figure 9):

• Definition: Heterogeneity refers to the variability or differences in results across the individual studies included in the meta-analysis. Ideally, all studies testing the same intervention should yield consistent results.
• Causes of Heterogeneity:
  • Variability may arise from clinical differences (e.g., patient populations, intervention delivery), methodological differences (e.g., study design, bias), or random variation.
• Statistics to Assess Heterogeneity:
  • I2 Statistic (I-Squared): Quantifies the percentage of total variation across studies due to heterogeneity rather than chance.
    • Rule of Thumb:
      • I2 < 25%: Low heterogeneity (desirable).
      • 25%-50%: Moderate heterogeneity.
      • >50%: High heterogeneity (requires cautious interpretation).
  • Chi2 (Chi-Square): Tests whether observed differences in results are due to chance but is often less informative alone.
  • Z-Statistic: Assesses the overall effect estimate but is less commonly emphasized compared to I2.
• Importance:
  • High heterogeneity (I2 > 50%) suggests substantial inconsistency across studies, possibly driven by factors beyond random chance.
  • Identifying and interpreting heterogeneity allows reviewers to decide if combining results into a single pooled estimate is valid and meaningful.
  • Moderate to high heterogeneity may require subgroup analyses or meta-regression to explore potential sources of variation.

Holistic View: The forest plot itself, combined with all the supplementary data (author/year, event totals, point estimates, confidence intervals, and heterogeneity assessments), provides a comprehensive tool for summarizing and interpreting evidence from multiple studies.

Concluding Insights: The pooled data (diamond) represents the “bottom line” on the effectiveness or harm of an intervention, but only when interpreted alongside context-providing elements such as heterogeneity and individual study results.

Summary of Key Points on Forest Plots:

• Individual Study Representation:
  • Each horizontal line on a forest plot corresponds to the result of an individual study.
  • The box represents the point estimate (e.g., effect size) for the study, and the line shows the 95% confidence interval (CI) around that estimate.
• Interpretation Relative to the Vertical Line:
  • The vertical line (line of no effect) typically represents the null hypothesis (e.g., risk ratio of 1 for relative measures or difference of 0 for continuous measures).
  • Results to one side of the line indicate either a positive or negative effect depending on the direction of the comparison (e.g., favoring treatment or control).
  • Crossing the Vertical Line:
    • If a study’s CI crosses the vertical line, it indicates that the null value is within the 95% CI.
    • This suggests no statistically significant difference was observed between the treatment and control groups for that individual study.
• Diamond at the Bottom of the Plot:
  • The diamond represents the overall or pooled result from all included studies in the meta-analysis.
  • Horizontal Points of the diamond mark the limits of the 95% CI for the combined estimate.
  • The same interpretation rules apply: if the diamond crosses the vertical line, the overall result is not statistically significant.
• Heterogeneity (I2 Statistic):
  • I2 Value indicates the degree of inconsistency among the studies included.
  • Interpretation:
    • I2 > 50%: Suggests substantial heterogeneity, meaning that variation across studies may not be due to chance alone, warranting cautious interpretation.
    • Lower I2 values indicate greater consistency.
• Further Analysis (Cochrane Review):
  • The reference to the Cochrane Review suggests thorough, high-quality evidence synthesis that emphasizes robust methodology, often using forest plots for meta-analytic results.