from https://s4be.cochrane.org/blog/2016/07/11/tutorial-read-forest-plot/

Purpose of Forest Plots

• Summarize Data: Forest plots graphically summarize data from multiple studies addressing the same question.
• Facilitate Comparison: They allow for the comparison of results across studies, providing an overall picture of the evidence, even if individual studies differ in findings.
• Identify Patterns: They help detect patterns, overall trends, and the consistency of results (heterogeneity) across studies.

Key Components of a Forest Plot

1. Axes

• Horizontal Axis:
  • It represents the estimated effect from each study.
  • The effect size could be relative (e.g., Odds Ratio [OR], Relative Risk [RR]) or absolute (e.g., Absolute Risk Reduction [ARR], Mean Difference).
  • The position of each study’s confidence interval (CI) along the axis indicates the magnitude and direction of the effect.
• Vertical Line (Line of Null Effect):
  • This line represents the point of no effect, meaning there is no difference between the compared groups.
  • For relative measures (e.g., OR, RR):
    • The null value is 1 (i.e., an OR or RR of 1 means no association between the exposure and outcome).
    • If the confidence interval crosses 1, the result is not statistically significant.
  • For absolute measures (e.g., ARR, Mean Difference):
    • The null value is 0 (i.e., a mean difference of 0 means no difference between groups).
    • If the confidence interval crosses 0, the result is not statistically significant.

2. Study Representation

1. Components of a Study Line

• Black Box (Point Estimate):
  • Represents the primary result or effect size of the study (e.g., relative risk, odds ratio).
  • Size of the Box: Proportional to the study’s sample size.
    • Larger Box: Indicates a study with more participants (greater weight in the analysis).
    • Smaller Box: Represents a study with fewer participants (less weight in the analysis).
• Horizontal Line (95% Confidence Interval [CI]):
  • Extends left and right from the box.
  • Shows the range within which the true effect is likely to lie 95% of the time.
  • Shorter line = More precise result (less uncertainty).
  • Longer line = Less precise result (more uncertainty).

2. Interpreting Study Lines Relative to the Null Effect Line

• Null Effect Line:
  • This is the vertical line on the forest plot.
  • Shows the value where there is no difference between the two groups.
  • The position of the null effect value depends on the type of statistic:
    • Relative measures (e.g., OR, RR): Null value = 1 (indicating no difference).
    • Absolute measures (e.g., ARR, mean difference): Null value = 0.
• Position of Study Lines:
  • To the Left or Right of the Null Line:
    • Indicates whether the study favors the intervention or the control.
    • Interpretation depends on the context (e.g., reducing risk, increasing likelihood of an event).
  • Crossing the Null Line:
    • If a study’s horizontal line crosses the null effect line, it includes the null value in its 95% CI.
    • Not Statistically Significant: Crossing the null line implies that the result might not be different from the null (no effect).

3. Importance of Study Size and Precision

• Bigger Studies (More Participants):
  • Typically have narrower confidence intervals.
  • Represented by a larger black box and shorter horizontal line.
  • These studies are less likely to cross the null effect line, indicating a higher likelihood of statistical significance.
• Smaller Studies (Fewer Participants):
  • Usually have wider confidence intervals.
  • Represented by a smaller black box and longer horizontal line.
  • More likely to cross the null effect line, reflecting greater uncertainty and reduced statistical significance.

Summary Points

• Black Box Size: Indicates sample size (bigger box = larger study).
• Horizontal Line (95% CI): Reflects precision (narrower line = greater precision).
• Crossing the Null Line: Implies lack of statistical significance.
• Relative vs. Absolute Measures: Null effect line value differs based on the type of statistic (1 for relative measures, 0 for absolute measures).

3. The Diamond

• Diamond Shape at the Bottom of the Plot:
  • diamond is the most important aspect of a forest plot.
  • Represents the combined result of all studies.
  • Vertical Points of the Diamond:
    • Represents the combined point estimate.
  • Horizontal Tips of the Diamond:
    • Represents the 95% confidence interval for the combined result.
  • Interpretation:
    • If the diamond’s CI crosses the line of null effect, the combined result is not statistically significant.
    • This indicates that, when considering all data, there is still a possibility that there is no effect.

4. Author and Year of Studies:

• Purpose: The far-left column of a forest plot typically lists the name of the lead author of each included study, alongside the year of publication.
  • Importance: This provides transparency regarding the source of the data being summarized. It allows for an assessment of study recency and context, helping reviewers gauge whether findings are potentially outdated or highly relevant to current practice.
  • Context: Including these details ensures traceability, facilitating further exploration of the original studies if necessary.

5. Event and Total Columns (n/N Format):

• Description: Two columns immediately to the left of the plot present data in an “n/N” format:
  • “n” (Event Count):
    • Number of individuals in a group (treatment or control) who experienced the outcome of interest (e.g., a heart attack, infection, recovery).
  • “N” (Total Count):
    • Total number of participants in that group.
  • Example Context: The first column usually refers to the treatment group, showing the proportion of treated individuals who had the outcome versus the total treated. The second column refers to the control group with analogous data.

Example:

Study	Treatment Group (n/N)	Control Group (n/N)
Study A	50/200 (25%)	75/200 (37.5%)
Study B	30/150 (20%)	45/150 (30%)

In Study A, 50 out of 200 people in the treatment group experienced the event (25%), while 75 out of 200 in the control group did (37.5%).

Usefulness:

• This arrangement allows for a quick, numerical comparison of event rates between the treatment and control groups, providing context for the graphical representation of results in the plot.
• It offers insight into absolute event rates and the relative size of each study’s groups, adding weight to the interpretation.
  • A study with very few total participants (small N) has less weight in the analysis.
  • A study with a large N is more influential in the meta-analysis.
  • The black box (point estimate) on the forest plot is based on this data.
  • The size of the box depends on the number of participants (N), giving more weight to larger studies.

6. Figure 7 – Point Estimate and 95% Confidence Interval (Right Column):

• Description: The far-right side of the forest plot provides numerical values for each study’s point estimate (e.g., risk ratio, odds ratio) and the corresponding 95% confidence interval (CI).
  • Point Estimate: This is a measure of effect size or association from each individual study.
  • 95% CI: Indicates the range within which we are 95% confident the true effect lies. A narrow CI suggests precision, while a wider CI indicates more uncertainty.
• Importance:
  • This column serves as a numerical summary of what is depicted graphically by the lines and boxes in the forest plot, offering an alternative, detailed view for those who prefer to analyze numbers directly.
  • The presence of this data also helps in identifying studies with high variability or those with stronger precision around their effect estimates.

7. Subtotal and Summary Statistics (Diamond Representation):

• Description: The diamond shape at the bottom of the forest plot represents the pooled estimate of effect across all included studies. Key elements highlighted in this section include:
  • Subtotal Line: Displays the cumulative totals of participants in the treatment and control groups across all studies.
  • Diamond Center: Represents the overall effect size (point estimate) derived from pooling all studies.
  • Diamond Width: Reflects the 95% CI around the pooled estimate.
• Utility:
  • Provides a visual summary of the overall intervention effect across all studies.
  • Enables a quick assessment of statistical significance and direction of the overall effect.
  • The boldness of the summary statistics underscores their role in drawing conclusions about the effectiveness or harm of the intervention being studied.

8. Heterogeneity of Studies:

• Heterogeneity refers to how much the results of individual studies differ from one another in a meta-analysis. Ideally, studies testing the same intervention should have similar results, but in reality, variations occur due to differences in:
  • Clinical factors (e.g., different patient populations, intervention methods).
  • Methodological factors (e.g., study design, bias, sample size).
  • Statistical variability (random chance).
• Statistics to Assess Heterogeneity:
  • I2 Statistic (I-Squared):
    • Quantifies the percentage of variation across studies that is not due to chance.
    • Rule of Thumb for I² Interpretation:
      • I² < 25% → Low heterogeneity (desirable, studies are consistent).
      • 25% – 50% → Moderate heterogeneity (some variation exists).
      • >50% → High heterogeneity (significant differences, requires careful interpretation).
    • Example:
      • If I² = 60%, it means 60% of the variation in results is due to real differences between studies, not just random chance.
  • Chi2 (Chi-Square):
    • Tests whether differences between studies are due to chance.
    • p-value < 0.05 suggests that the variation is significant.
    • Limitations: It’s affected by the number of studies—small studies may show low heterogeneity even if differences exist.
  • Z-Statistic
    • Assesses the overall pooled effect size.
    • Less commonly used compared to I² because it doesn’t directly measure differences between studies.
• Importance:
  • Identifying and interpreting heterogeneity allows reviewers to decide if combining results into a single pooled estimate is valid and meaningful.
    High heterogeneity (I² > 50%) means the studies show inconsistent results, making the overall pooled estimate less reliable.
  • Low heterogeneity (I² < 25%) means the studies are similar, and their combined results are more trustworthy.
  • When heterogeneity is high, researchers may:
    • Perform subgroup analyses (e.g., separating by age groups, dosages, study designs).
    • Use random-effects models instead of fixed-effects models.
    • Explore potential sources of variation (e.g., different patient populations).

Holistic View: The forest plot itself, combined with all the supplementary data (author/year, event totals, point estimates, confidence intervals, and heterogeneity assessments), provides a comprehensive tool for summarizing and interpreting evidence from multiple studies.

Concluding Insights: The pooled data (diamond) represents the “bottom line” on the effectiveness or harm of an intervention, but only when interpreted alongside context-providing elements such as heterogeneity and individual study results.

example: Forest Plot with High (I² ≈ 75%) and Low Heterogeneity (I² ≈ 10%)

Here are two forest plots illustrating low vs. high heterogeneity:

1. Forest Plot with Low Heterogeneity (I² ≈ 10%)

• Effect sizes are consistent (all studies report similar OR values around 1.0).
• Confidence intervals (CIs) overlap significantly, indicating little variation between studies.
• The meta-analysis result is stable, meaning combining these studies is valid.
• Interpretation: Low heterogeneity suggests strong consistency between studies.

2. Forest Plot with High Heterogeneity (I² ≈ 75%)

• Effect sizes vary widely (some studies favor treatment, others show no effect or even harm).
• Confidence intervals do not overlap well, showing major variability between studies.
• The meta-analysis summary is less reliable, as results are too different.
• Interpretation: High heterogeneity means substantial variation, requiring deeper investigation (e.g., subgroup analysis).

Key Takeaways

• Low heterogeneity (I² < 25%) → Studies are consistent and results can be confidently pooled.
• High heterogeneity (I² > 50%) → Studies differ significantly, requiring caution when interpreting pooled results.

---

Summary of Key Points on Forest Plots:

• Individual Study Representation:
  • Each horizontal line on a forest plot corresponds to the result of an individual study.
  • The box represents the point estimate (e.g., effect size) for the study, and the line shows the 95% confidence interval (CI) around that estimate.
• Interpretation Relative to the Vertical Line:
  • The vertical line (line of no effect) typically represents the null hypothesis (e.g., risk ratio of 1 for relative measures or difference of 0 for continuous measures).
  • Results to one side of the line indicate either a positive or negative effect depending on the direction of the comparison (e.g., favoring treatment or control).
  • Crossing the Vertical Line:
    • If a study’s CI crosses the vertical line, it indicates that the null value is within the 95% CI.
    • This suggests no statistically significant difference was observed between the treatment and control groups for that individual study.
• Diamond at the Bottom of the Plot:
  • The diamond represents the overall or pooled result from all included studies in the meta-analysis.
  • Horizontal Points of the diamond mark the limits of the 95% CI for the combined estimate.
  • The same interpretation rules apply: if the diamond crosses the vertical line, the overall result is not statistically significant.
• Heterogeneity (I2 Statistic):
  • I2 Value indicates the degree of inconsistency among the studies included.
  • Interpretation:
    • I2 > 50%: Suggests substantial heterogeneity, meaning that variation across studies may not be due to chance alone, warranting cautious interpretation.
    • Lower I2 values indicate greater consistency.
• Further Analysis (Cochrane Review):
  • The reference to the Cochrane Review suggests thorough, high-quality evidence synthesis that emphasizes robust methodology, often using forest plots for meta-analytic results.

---

examples:

Forest Plot (Odds Ratio Example)

Key Features of the Plot:

1. Each horizontal line represents a study, showing the confidence interval (CI).
2. The blue dots indicate the effect size (OR) for each study.
3. The vertical dashed line at OR = 1 represents the null effect (no difference between groups).
   • Studies whose confidence intervals cross 1 are not statistically significant.
4. The red dot represents the meta-analysis summary effect (combined effect size).

Interpretation of the Example:

• Some studies (e.g., Study 2) show an OR above 1, suggesting higher odds of the event in the treatment group.
• Others (e.g., Study 4) have an OR below 1, indicating a protective effect of the treatment.
• Study 3 and Study 5 cross the null line (OR=1), meaning their results are not statistically significant.
• The meta-analysis summary (red dot) suggests an overall OR of 0.92, indicating a small protective effect.

Forest Plot (Absolute Risk Reduction Example)

Key Features of the Plot:

1. Each horizontal line represents a study, showing the confidence interval (CI).
2. The green dots indicate the effect size (ARR%) for each study.
3. The vertical dashed line at ARR = 0 represents the null effect (no difference in absolute risk).
   • Studies whose confidence intervals cross 0 are not statistically significant.
4. The red dot represents the meta-analysis summary effect (combined effect size).

Interpretation of the Example:

• All studies show a negative ARR, meaning the intervention reduces absolute risk.
• Some studies (e.g., Study B) have a CI crossing 0, meaning their results are not statistically significant.
• The meta-analysis summary (red dot) suggests an overall ARR of -2.7%, meaning the intervention reduces the absolute risk of the event by 2.7%.
• If we calculate the Number Needed to Treat (NNT):
  • NNT = 1/ARR = 1/0.027 ≈ 37
    • This means 37 patients need to be treated to prevent one adverse event.

This example helps differentiate absolute measures (ARR) from relative measures (e.g., OR, RR) by focusing on the actual reduction in risk rather than comparing risks between groups

Forest Plot (Mean Difference Example)

Here is an annotated forest plot example using Mean Difference (MD).

Key Features of the Plot:

1. Each horizontal line represents a study, showing the confidence interval (CI).
2. The purple dots indicate the effect size (Mean Difference) for each study.
3. The vertical dashed line at MD = 0 represents the null effect (no difference in means).
   • Studies whose confidence intervals cross 0 are not statistically significant.
4. The red dot represents the meta-analysis summary effect (combined effect size).

Interpretation of the Example:

• All studies show a negative mean difference, suggesting the intervention reduces the measured outcome (e.g., weight loss in kg).
• Some studies (e.g., Study Y) have a CI crossing 0, meaning their results are not statistically significant.
• The meta-analysis summary (red dot) suggests an overall MD of -2.2 kg, meaning the intervention results in an average weight loss of 2.2 kg compared to the control group.

---

Key Takeaways Across Forest Plots:

• Relative measures (OR, RR) compare event probabilities between groups.
• Absolute measures (ARR) show actual risk reduction.
• Mean Difference (MD) quantifies changes in continuous variables (e.g., weight loss, blood pressure).

When to Use Relative vs. Absolute Measures in a Forest Plot

Measure	Definition	Best Used For	Null Value	Interpretation
Odds Ratio (OR)	Compares odds of an event occurring in two groups	Case-control studies, logistic regression, meta-analysis	1	OR > 1: Increased odds in treatment group; OR < 1: Decreased odds
Relative Risk (RR)	Compares probability (risk) of an event in two groups	Cohort studies, randomized controlled trials (RCTs)	1	RR > 1: Increased risk in treatment group; RR < 1: Reduced risk
Absolute Risk Reduction (ARR)	Shows actual difference in event rates between two groups	Clinical impact, decision-making, calculating NNT	0	ARR > 0: Treatment reduces risk; ARR < 0: Treatment increases risk
Number Needed to Treat (NNT)	The number of patients needed to treat to prevent one event	Practical application in healthcare, cost-effectiveness	—	Lower NNT = More effective treatment
Mean Difference (MD)	Compares continuous outcomes between groups (e.g., weight, blood pressure)	Clinical trials, meta-analysis of continuous outcomes	0	MD > 0: Increase in outcome; MD < 0: Decrease in outcome

---

When to Use Each Measure:

1. Use OR when event rates are low or in case-control studies
   • Example: Risk of rare cancer in smokers vs. non-smokers.
2. Use RR when absolute risk data is available
   • Example: Risk of stroke in patients taking a new anticoagulant.
3. Use ARR to assess real-world clinical impact
   • Example: If ARR = 5%, then 5 out of 100 patients benefit from the treatment.
4. Use NNT for treatment decisions
   • Example: If NNT = 20, then 20 patients need to be treated to prevent one event.
5. Use MD for continuous outcomes
   • Example: Blood pressure reduction (e.g., -5 mmHg with a new antihypertensive drug).