10.4081/btvb.2022.30
Scientific literature is overflowing of significance testing and p-values. P-value states how discordant the observed finding is with a null hypothesis. P&lt;0.05 indicates that an association greater than that detected would happen less than 5% of the time under a null hypothesis of no association [...].
application/pdf
2785-5309
30
1
en
Licia Iacoviello
https://creativecommons.org/licenses/by-nc/4.0
Bleeding, Thrombosis and Vascular Biology
Editorial
Moving beyond p-value
Text.Serial.Journal
https://www.btvb.org/btvb
1
Invited Editorials
2022-04-15
2022-03-24
2022-03-22
2022-05-12
Bleed Thromb Vascul Biol
Licia Iacoviello
2022/04/15
10.4081/btvb.2022.30
Department of Epidemiology and Prevention, IRCCS Neuromed, Pozzilli; Department of Medicine and Surgery, University of Insubria, Varese
2785-5309
1
Editorial
en
Moving beyond p-value
https://www.btvb.org/btvb/article/download/30/18
1
                                                                                                                         EDITORIAL

Moving beyond p-value
Augusto Di Castelnuovo,1 Licia Iacoviello2,3

Mediterranea Cardiocentro, Napoli, Italy; 2Department of Epidemiology and Prevention, IRCCS Neuromed, Pozzilli, Italy;
1

Department of Medicine and Surgery, University of Insubria, Varese, Italy
3




                                                                              trusted in the past that p-values from our tests had pro-
INTRODUCTION                                                                  vided such “reassuring” probability. Unfortunately, this is
    Scientific literature is overflowing of significance test-                not the case: p-value is not the false positive risk.
ing and p-values.1 P-value states how discordant the ob-                          This is not the unique misuse/misinterpretation of the
served finding is with a null hypothesis. P<0.05 indicates                    p-value.1
that an association greater than that detected would hap-
pen less than 5% of the time under a null hypothesis of
no association. As a widespread belief, it is thought that                    STATISTICAL INFERENCE IN THE 21st CENTURY




                                                                                               ly
p-value provides the probability that chance alone pro-
                                                                                  In 2019, the American Statistical Association pub-




                                                                                        on
duced the detected association. This is not true.1 P-value
                                                                              lished a special issue containing 43 papers,3 which ex-
is the probability of the data (D) assuming that the null
                                                                              haustively discussed the topic, and tried to provide
hypothesis (H) is true. In formal statistical language, p-
                                                                              alternatives to go beyond p-value. In an accompanying


                                                                                  e
value=p(D|H). The probability that chance alone creates
                                                                              editorial,4 the perils of misuse and misinterpretation of p-
                                                                             us
the observed association is p(H|D) (=the probability of the
                                                                              values and significance testing were well expressed: a)
null hypothesis given the data).2
                                                                              don’t base your decisions merely on whether or not an as-
    Imagine you obtained a result for a coagulation test as-
                                                                             al
                                                                              sociation or an effect was found to be “statistically signif-
sociated with thrombosis. What you now want to know is
                                                                              icant”; b) don’t believe that an association exists (or is
                                                          ci

the probability that given the finding (“the test is associ-
                                                                              null) just because it was (it was not) statistically signifi-
ated”), the null hypothesis (“there is no association”) is true;
                                                        er



                                                                              cant; c) don’t believe that your p-value gives you the prob-
this probability is also known as “False Positive Risk” and
is equal to p(H|D). Lower this probability is, more confident                 ability that chance alone produced the observed
                                                       m




you are in the correctness of the test conclusion.                            association; d) don’t conclude anything about scientific
                                                                              or practical importance of your data only based on statis-
                                            om




    I am afraid that a lot of us, including myself, have
                                                                              tical significance (or lack thereof).
                                                                                  Following these convincing and authoritative pro-
                                     -c




                                                                              nouncements by statistician’s community, several Jour-
                                                                              nals changed their guidelines for statistical reporting.5,6 A
                            on




    Correspondence: Augusto Di Castelnuovo, Mediterranea Cardio-
    centro, Via Orazio, 2, 80122 Napoli, Italy.                               firm claim for retiring of statistical significance charac-
    E-mail: dicastel@ngi.it                                                   terises these updating.
                       N




                                                                                  This is also the line of this editorial, which sketches
    Key words: Editorial; p-value.
                                                                              the statistical guidelines of Bleeding, Thrombosis and
    Contributions: The authors contributed equally.                           Vascular Biology Journal.

    Conflict of interest: The authors declare no potential conflict of in-
    terest.
                                                                              DON’T
    Funding: None.                                                                What not to do? We have never to assume that there
    Received for publication: 24 March 2022.                                  is ‘no association’ or ‘no difference’ or ‘no effect’ just be-
    Accepted for publication: 29 March 2022.                                  cause a p-value is larger than a threshold such as 0.05 (or,
                                                                              in the same way, because a confidence interval includes
    This work is licensed under a Creative Commons Attribution Non-
                                                                              zero or one –depending on the metric).3
    Commercial 4.0 License (CC BY-NC 4.0).
                                                                                  We have not to conclude that two studies are in con-
    ©
     Copyright: the Author(s), 2022                                           flict because one had a statistically significant result, and
    Licensee PAGEPress, Italy                                                 the other did not. For example, the risk factor alpha is as-
    Bleeding, Thrombosis and Vascular Biology 2022; 1:30
    doi:10.4081/btvb.2022.30
                                                                              sociated with a certain outcome with an identical point es-
                                                                              timate in both studies A and B (Figure 1), and confidence

                                                Bleeding, Thrombosis and Vascular Biology 2022; 1:30
4                                                 A. Di Castelnuovo, L. Iacoviello




Figure 1. Relative risks and 95% confidence intervals (invented data) for two risk factors (alpha and beta) in two different studies (A
and B). Black squares represent the relative risk (RR); horizontal lines denote the 95% confidence intervals (95%CI).




                                                                                                 ly
intervals largely overlap: the two studies are not divergent           same outcome, the risk factor alpha (observed effect equal




                                                                                         on
in their conclusions, even if the association is ‘significant’         to 300%) is compatible with: a) a risk greater than 50%
in study A but not in study B.                                         for the large majority of the interval; b) a risk between 0%
    We have to stop the use of p-values in a dichotomous               to 50% for another portion of the interval and c) a null


                                                                                     e
way and must retire statistical significance.6                         risk or protection for a very small fraction of the interval.
                                                                            us
                                                                       Concluding that beta is a risk factor since p<0.05 but
                                                                       alpha is not as p>0.05 is unreasonable.
                                                                     al
DO                                                                          The magnitude of the effect (or association or differ-
                                                                       ence) is very important. Focusing on statistically signifi-
    Unfortunately, there is no single solution for awe-
                                                        ci

                                                                       cant but small effects (therefore most of the times having
somely replacing of significance testing and p-value. Sev-
                                                      er


                                                                       negligible clinical impact) and ignoring large effects (po-
eral alternatives have been proposed, and it is expected
                                                                       tentially of clinical interest) because the latter are not sta-
that many of them will be increasingly used for statistical
                                                     m




                                                                       tistically significant is an improper approach.
inference in scientific research. Detailed discussion of
                                            om




these alternatives is outside the scope of this editorial. The
reader will find plenty of information in other sources.3,4
                                                                       A FINAL QUESTION
                                      -c




                                                                           If I were on a crashing airplane, I would rely more on
MEASURE OF THE EFFECT AND HYPOTHESIS                                   a pilot’s manoeuvre that would reduce the risk of crashing
                               on




TESTING                                                                by a non-statistically significant 90% rather than one that
                                                                       would reduce the risk by a significant 10%.
    A vital thing to argue is that the measure of the effect
                          N




                                                                           And you?
is more important that the hypothesis testing. Magnitude,
precision, direction, plausibility, consistency, repeatability
and clinical or practical utility have to be the key features
to be investigated for an effect (or association, or differ-           REFERENCES
ence), much more than a p<0.05. Note that only precision                1. Wasserstein R, Lazar N. The ASA’s statement on p-values:
and, to a lesser extent, magnitude, are linked to p-values.                context, process, and purpose. Am Stat 2016;70:129-33
    We should be more confident with uncertainty.4 We                   2. Krueger JI, Heck PR. Putting the p-value in its place. Am
have to report point estimate (=observed effect) and con-                  Stat 2019;73:122-8.
fidence intervals, and describe the practical inferences                3. Statistical inference in the 21st Century: a world beyond
suggested by all values within the interval, especially the                p<0.05. Am Stat 2019;73.
                                                                        4. Wasserstein RL, Schirm AL, Lazar NA. Moving to a world
point estimate and the limits. We must become familiar
                                                                           beyond p<0.05. Am Stat 2019;73:1-19.
with the fact that all values within the interval are com-              5. Harrington D, D’Agostino RB Sr, Gatsonis C, et al. New
patible with the data. For example, in the study B (Figure),               guidelines for statistical reporting in the journal. N Engl J
the risk factor beta is associated with an increased risk                  Med 2019;381:285-6.
(for a certain outcome) ranging from 10% to 50% (point                  6. Amrhein V, Greenland S, McShane B. Retire statistical sig-
estimate equal to 30%). In the same study and for the                      nificance. Nature 2019;567:305-7.


                                       Bleeding, Thrombosis and Vascular Biology 2022; 1:30
