Introducing “Opening Up”

Chris Castille Nicholls State University

Hi there! Chris Castille here. I’m an I-O psychologist from Nicholls State University, working in the College of Business Administration. I’ve been asked to serve as the editor of TIP’s newly created “Opening Up” column. This column was created in coordination with SIOP’s Open Science and Practice (OSP) Committee to foster a meaningful and constructive dialogue regarding open science in I-O psychology as it is applied in I-O research and practice.

So, how did I get here and why am I doing this? Although there are many historical facts that brought me to open science, I’d like to share two with you. The first occurred during my doctoral studies at Louisiana Tech. My cohort was tasked with reading what I would argue is the most controversial high-profile psychology study that has been published in the past 20 years: Daryl Bem’s study, published in the Journal of Personality and Social Psychology, which claimed to provide experimental evidence of psychic abilities like ESP, telepathy, and clairvoyance (hence the controversy). Though my cohort generally dismissed the findings as both theoretical and statistical nonsense, our professor (Jerome Tobacyk) pushed us to consider how Bem’s methods (e.g., power analysis, experimental design, statistical approach) compared to others we commonly trusted from our own experience and literature. At the time (and at first blush), his methods appeared simpler (e.g., only t-tests were used) and more straightforward than similar designs from studies we respected and trusted. This dialogue planted a seed of doubt: If we were so willing to dismiss Bem’s case for ESP in spite of the apparent rigor and simplicity of his methods, what other hypotheses should we be willing to jettison from our own evidence base?

The second event occurred years later, when that first seed of doubt received sunlight in the form of a catalog of damning facts gathered by cognitive neuroscientist Chris Chambers. In his book The 7 Deadly Sins of Psychology: A Manifesto for Reforming the Culture of Scientific Practice (2017), Chris examines the state of scientific psychology and provides a wealth of evidence that calls the validity of many claims in psychology into question. His alternative explanations: a set of sins that include bias (e.g., negative results are often excluded from the literature), degrees of freedom in analyzing one’s data (e.g., interrogating the data until a statistically significant and thus publishable probability value is found), use of unreliable methods (e.g., preference for conceptual rather than direct replication), and not making one’s work reproducible in some form.

Although his sins could read as an indictment of many researchers, Chambers pointed to the system within which research takes place as being at fault (e.g., a culture of “publish or perish”). Within this system, top-tier publications can be valued more in practice than the mission that motivates these publications in the first place: furthering science and evidence-based practice. To the extent that this is true, the system can distort the scientific record, which in turn can threaten the improvement and integrity of future research and practice.

Interestingly, although Chambers provides little detail on the scope of the problem for our specific area of I-O psychology, others have moved to assess our literature to see what, if any, foundations are weaker than we’d like them to be.

I share these two facts because they offer a glimpse into my motivation for taking on the task as editor of “Opening Up” as well as what I hope to do with this column. I took up this role because I believe I could help foster a productive dialogue regarding open science and practice in I-O psychology. To this end, the column will aim to do the following:

• Raise awareness regarding the “replication crisis” in psychology and science writ large, with an emphasis on its meaning and implications for I-O psychology
• Provide tips for fostering more open and transparent behavior in research and practice (e.g., education about preregistration)
• Discuss productive initiatives that are occurring across various scientific disciplines (e.g., registered reports, the Society for the Improvement of Psychological Science, the Psychological Science Accelerator)
• Measuring the openness and accessibility of I-O psychology research (e.g., counting the proportion of studies that are registered reports, sharing research materials and data)
• Considering the unique implications of open science for practitioners of I-O psychology (e.g., how preregistration and data sharing might be important for practitioners)
• Share things that you can do to stay informed on open science (e.g., several podcasts have emerged in the wake of the replication crisis and these provide an often entertaining source of content on the crisis, such as The Black Goat; Two Psychologists, Four Beers; Everything Hertz; ReprocubiliTea.

To this end, we’ll accept reader comments that are aimed at fostering a productive conversation regarding enhancing the credibility of I-O psychology by means of opening up our science. For instance, one suggestion for improving I-O psychology put forward by Steven Rogelberg at a SIOP session is that I-O psychology programs consider offering a “replication” thesis track, whereby I-O psych findings (contemporary and classic) undergo direct replication via registered report (i.e., all methodological choices are stored in a database prior to the experiment and can be used to check whether the finished product deviated from established protocols). This type of work could then be published in an online journal SIOP creates explicitly designed for replications. It is worth noting that this is essentially how psychology’s founder, Wilhelm Wundt, started the field. I’m sensing a “back to basics” kind of commentary. We’ll also take on anonymous “Dear the Editor/Open Science Committee” kinds of contributions where an anonymous writer (or writers) can submit concerns or letters to the committee, which will be published in the column.

With that, I’d like to thank the Open Science and Practice Committee for tasking me with editing this column. Thanks also goes out to Mike Morrison for helping me execute this role. I’d also like to thank Fred Oswald and Steven Rogelberg specifically for their enthusiasm for this topic and for asking me to serve as the first editor of “Opening Up.”

Appreciatively,

Chris

References

Bem, D. J. (2011). Feeling the future: Experimental evidence for anomalous retroactive influences on cognition and affect. Journal of Personality and Social Psychology, 100, 407-425.

Chambers, Chris. (2017). The seven deadly sins of psychology: A manifesto for reforming the culture of scientific practice. Princeton, NJ: Princeton University Press.