Dear OB Colleagues,
There's been quite a bit of controversy about "Hypothesizing after Results are Known" (HARKing). The following article available at http://www.hermanaguinis.com/pubs.html provides empirical results about when, why, and how different forms of HARKing are good, bad, or inconsequential for the credibility of research results:
The Abstract is below. We look forward to your reactions to this article and further discussion about this important issue.
All the best,
Herman Aguinis, Ph.D.
Vice President-Elect, Academy of Management
Avram Tucker Distinguished Scholar and Professor of Management
The George Washington University School of Business
2201 G Street, NW
Washington, DC 20052
ABSTRACT The practice of hypothesizing after results are known (HARKing) has been identified as a potential threat to the credibility of research results. We conducted simulations using input values based on comprehensive meta-analyses and reviews in applied psychology and management (e.g., strategic management studies) to determine the extent to which two forms of HARKing behaviors might plausibly bias study outcomes and to examine the determinants of the size of this effect. When HARKing involves cherry-picking, which consists of searching through data involving alternative measures or samples to find the results that offer the strongest possible support for a particular hypothesis or research question, HARKing has only a small effect on estimates of the population effect size. When HARKing involves question trolling, which consists of searching through data involving several different constructs, measures of those constructs, interventions, or relationships to find seemingly notable results worth writing about, HARKing produces substantial upward bias particularly when it is prevalent and there are many effects from which to choose. Results identify the precise circumstances under which different forms of HARKing behaviors are more or less likely to have a substantial impact on a study's substantive conclusions and the field's cumulative knowledge. We offer suggestions for authors, consumers of research, and reviewers and editors on how to understand, minimize, detect, and deter detrimental forms of HARKing in future research.