Reading Biotech Trial Results: Study Design and Effect Size Before the p-Value
When a biotech company announces trial success, examine patients, controls, prespecified endpoints, effect size, and safety. Learn to distinguish statistical significance from clinical relevance in investment materials.
Start by checking which prespecified primary endpoint was met, in which patient population, and against which control group. A p-value alone cannot establish effect size, safety, or approval prospects. Also check whether favorable findings were selectively presented from multiple analyses.
Read the study's research question before the press release
A trial is designed to answer who received which treatment, compared with what, and over what period. Interpretation depends on randomization, blinding, a standard-of-care or placebo control, and patients' previous treatments. If the analyzed population differs from total enrollment, check why patients were excluded. The FDA's E9(R1) guidance describes an approach that clearly defines the treatment effect of interest and consistently determines how events such as treatment discontinuation are handled in design and analysis. When reading investment materials, this also helps you check whether only favorable results excluding dropouts have been presented.
Separate significance, effect size, and multiple comparisons
A p-value is neither the probability that a drug will succeed nor a measure of effect size. With a large sample, a small difference can be statistically significant; with a small sample, a meaningful difference can be difficult to estimate precisely. Read effect sizes together with confidence intervals. If a trial missed its primary endpoint but showed improvement in a subgroup or one of several secondary endpoints, check whether the analysis was prespecified. The FDA explains that testing multiple endpoints requires controlling the risk of false positive conclusions. Record exploratory signals separately from confirmatory conclusions.
The additional steps between safety findings and commercial success
Even strong efficacy figures can have different treatment value if serious adverse events, discontinuations, or management burdens are substantial. Check the denominator and observation period for adverse event rates and differences from the control group. How much an improvement in a surrogate endpoint translates into longer survival or better daily functioning is a separate question. Announcing trial results, obtaining regulatory approval, and generating prescriptions and revenue are distinct stages. Examine the convenience of competing treatments and manufacturing capacity as well, and do not declare superiority by placing numbers from different trials side by side without aligning patient populations.
Worked example
Hypothetical example: If an event occurs in 40 of 200 treated patients and 60 of 200 control patients followed for the same period, event rates are 20% and 30%, respectively. The absolute difference is 10 percentage points, and the relative reduction is (30%-20%)÷30%=about 33.3%. These express the same data using different reference points. These numbers alone cannot establish statistical significance, a survival-analysis hazard ratio, or the likelihood of approval.
What to check
- Do the primary endpoint and analyzed population match the prespecified plan?
- Have you checked the control group, patient population, and observation period?
- Have you examined effect size, confidence intervals, and multiple comparisons?
- Have you read serious adverse events and dropouts alongside efficacy?
Sources and primary materials
This is educational material about research methods. For an actual company decision, verify the filings and primary sources for the relevant period.