Here are three action items that you can take to the bench with you to prevent p-hacking and enhance the reliability of your conclusions: But awareness of this phenomenon is critically important, both to prevent bias in our own research and to improve our ability to interpret the results of others. It’s likely that we all have, save those who were extremely well-trained from day one. The point of this article is not to bash everyone who has subconsciously p-hacked at some point in their life. But this is wrong with the purpose of science being to establish what is true, p-hacking muddies the waters, wastes time and resources, and may (in extreme cases) diminish the public’s belief in science. Indeed, it’s tempting to see p=0.06 and to alter the statistical test, test a few transformations, or exclude that one point that didn’t follow your trend. There’s no great reason for this to be the case instead, he (and others) posit that this is evidence of how widespread hacking borderline p-values has become. One of the authors of the study that brought researcher degrees of freedom to the forefront, Uri Simonsohn, points out that p-values just under 0.05 are extremely over-represented in the published data. A great and timely tool to visualize this is available at 538 Science. By simply manipulating researcher degrees of freedom, even absolutely negative data can produce a p-value under 0.05 an incredible 61% of the time. These include when to stop collecting data, whether or not your data will be transformed, which statistical tests (and parameters) will be used, and so on. P-hacking is typically done through manipulation of “researcher degrees of freedom,” or the decisions made by the investigator. The term p-hacking, coined in 2014 by Regina Nuzzo in Nature News, describes the conscious or subconscious manipulation of data in a way that produces a desired p-value.

Rather, I want to talk about something which we all have experience with, to some degree or another: data dredging, or p-hacking.

Moreover, the p-value cannot directly speak to the strength of evidence, which can be better inferred when considering effect size, prior probability, and experimental reproducibility.īut this isn’t an article about the p-value, per se. There are important qualifications to p-value interpretation. Contrary to popular interpretation, the p-value doesn’t indicate the likelihood that the observed result was due to chance. P-value abuse directly contributes to one of the biggest problems facing the scientific community: the prominence of false-positive results in the published literature.

Data dredging/ p-hacking