Opening Up

Christopher M. Castille, Nicholls State University

Using Our Science to Improve Science

Haley R. Cobb, Jack C. Friedrich, & Candice L. Thomas

“What makes [scientific research] difficult is that research is immersion in the unknown.”

- Schwartz, 2008

For some time, open science has been touted as a tool to bolster—or rectify—the credibility of the psychological sciences. Open science “is an umbrella term used to refer to the concepts of openness, transparency, rigor, reproducibility, replicability, and accumulation of knowledge” (Crüwell et al., 2019, p. 1), and those researchers engaging in open science practices may, for example, openly share code and data, make details of study protocol openly available, preregister study designs and hypotheses, or post preprints of their work. Specific to our field, industrial-organizational (I-O) psychology has considered what benefits may come from open science and what practices may be most applicable to our field. However, Castille et al. (2022) were keen to point out that it is challenging to determine which open science practices to adopt or learn. Although these practices are on the rise, adoption broadly has been slow: Over the last 10 years, only 2–2.5% of journal articles in four of the most prominent organizational sciences journals contained open materials or open data (Tenney et al., 2021).

So, why is the uptake of open science slow? Although little research has been done on I-O psychology specifically, we can learn from other fields using open science. For example, scholars—47% of German psychologists in a study by Abele-Brehm and colleagues (2019)—feared that sharing data placed them at a competitive disadvantage. Others worry about their ideas being scooped or errors being detected when data are openly shared (Houtkoop et al., 2018) or that open science will stifle their creativity (Baumeister, 2016). We suggest that, for those scholars who are new to open science or may be hesitant to learn open science practices, incorporating psychological safety into one’s open science practice can challenge these fears. There is ample evidence to suggest that psychological safety can improve work in many ways (e.g., creativity, innovation, information sharing), so why not incorporate these principles into our own work?

Psychological Safety: A Very Brief Overview

Psychological safety can be thought of “as a shared belief that the team is safe for interpersonal risk taking” (Edmondson, 1999, p. 1), which may include sharing ideas, asking questions, voicing concerns, or pointing out mistakes without fear of negative repercussions (e.g., Edmondson & Lei, 2014). Psychological safety is relevant at multiple levels of analysis (e.g., individual, group/team, organizational) and related to a variety of outcomes (e.g., knowledge sharing, creativity, performance; Edmondson & Lei, 2014). There are a number of factors that predict and foster psychological safety in the workplace: Relationships with coworkers, leadership behaviors, and perceptions of employee support are all seen as important determinants of psychological safety (e.g., Carmeli & Zisu, 2009; Newman et al., 2017). For example, individuals are more likely to believe they will be given the benefit of the doubt when their relationships with group members are characterized by trust and respect (e.g., Newman et al., 2017). Additionally, leaders can engage in supportive leadership behaviors, inclusive leadership, being trustworthy, and valuing participation, allowing the leader to act as a model to show followers that it is safe to take risks and communicate openly (e.g., Newman et al., 2017). Finally, employee perceptions of organizational support and trust are related to psychological safety (Carmeli & Zisu, 2009).

We posit that I-O psychology research team members and leaders may benefit similarly by feeling supported by their fellow research team members, exhibiting leadership qualities that support psychological safety, and feeling supported by their organization or department to try new open science practices. We continue this column by suggesting other ways that open science and psychological safety could go together, including suggesting some tips for our readers.           

Psychological Safety and Open Science

Psychological safety in the realm of open science may look similar and could be defined as the belief that one may share ideas, ask questions, voice concerns, point out mistakes, and innovate (e.g., adopt new open science practices) without fear of repercussion from research team members or from oneself. In the open science context, psychological safety could include feeling that colleagues will not ridicule or belittle you for mistakes identified in open materials, taking risks by engaging in new publication practices (e.g., registered reports), and being comfortable with seeking the greater level of feedback that comes with being open. Psychological safety, therefore, could occur for individual researchers (e.g., feeling psychologically safe to learn a new data sharing practice), teams of researchers (e.g., suggesting to a new research team that a research study be preregistered), and across our field more broadly (e.g., being met with understanding when honest mistakes are uncovered in openly shared materials). Shedding the fear of mistakes may result in an increase in openly sharing research materials, identifying errors and fixing them, and may also create more learning opportunities for researchers in general.

For Individuals

For those I-O researchers who have yet to adopt open science practices or are hoping to learn new skills in this domain, doing so within a psychologically safe context should support outcomes such as learning, innovation, and risk taking. One of the chief worries researchers cite when discussing adopting open practices is loss of creativity by adhering to stricter research protocols. Setting aside the broader discussion of whether this is actually the case, psychological safety may support creativity and innovation and even learning. For example, among graduate students, psychological safety is associated with feelings of vitality and greater involvement in creative work (Kark & Carmeli, 2008), and learning through failure can occur through the cultivation of psychological safety (Carmeli & Gitell, 2009)—a necessity, given that there are numerous open science practices and tools, which researchers will inevitably implement with varying degrees of success as they adopt or change them to suit their needs. Individual researchers may recognize psychological safety when they no longer feel afraid to try something new, like adopting a new open science practice, and this experience may also allow them to promote psychological safety within their teams as a leader or as a team member.

For Research Teams

Google’s Project Aristotle highlighted that psychological safety was the most important factor to how their teams innovated (Bergmann & Schaeppi, 2016), suggesting just how powerful psychological safety can be for a team. Within research teams, each scholar can encourage questions, innovation, and openness across the research process. This also means being open to admitting mistakes and correcting them. For example, Strand’s (2023) “Error Tight” protocol was designed to pinpoint, minimize, and correct for errors, and can be adopted by research labs and teams. It is important to note that fostering psychological safety can be a collective effort, just as open science should be. Each member of a research team should experience psychological safety, just as they should also feel responsible to share research protocols and findings more transparently. Within our own research teams, the authors of this column have experienced and promoted psychological safety when using open science practices. That has looked like walking through preregistrations as a team, especially when new team members are unfamiliar with the Open Science Framework, and encouraging questions from all team members, regardless of their experience with open science.

For the Broader I-O Community

The broader scientific community, including ours, has historically not been so conducive to these ideas that we suggest here. For example, the “publish or perish” mentality is thought to contribute to engagement in unethical and consequential research practices, such as hypothesizing after results are known (HARK-ing) or p-hacking (Bedeian et al., 2010; Fanelli, 2010; Gopalakrishna et al., 2022). Additionally, the pressure to publish may lead to research that has little impact on the field and lacks creativity (Miller et al., 2011). Overall, it seems that publish or perish mentality may actually be harmful to the advancement of scientific inquiry and undermine the credibility of research (Simmons et al., 2011). These are all issues that open science and psychological safety have the potential to address; by promoting psychological safety in environments where open science is being practiced, scholars, their research teams, and their communities have the opportunity to learn more, reduce errors, innovate, and build upon each other’s work in more impactful ways. The idea that a researcher—no matter their status—is immune to error or can produce high quality, high impact work at their baseline is an unreasonable expectation and undermines learning, growth, and opportunities to become better scientists and to produce better work in the future (e.g., Livio, 2013). This is especially applicable in classrooms, with students, and when adopting new open science practices that researchers and team members may be unfamiliar with but eager to learn.

Supporting Diversity, Equity, and Inclusion

Although open science has many purported benefits, it is undeniable that embracing open science involves a large degree of risk for many scholars, both real and perceived. Castille et al. (2022) highlighted some issues regarding belongingness and open science, and here, we build on this conversation to suggest that psychological safety can be leveraged to not only make the adoption of open science practices easier but also to make academic research better for all who contribute. Issues of inequity are evident and persistent in the academy (e.g., Gabriel et al., 2023; Ledgerwood et al., 2022), and nontenured faculty are most susceptible to the publish or perish mentality (Miller et al., 2011), having more to lose than tenured faculty when it comes to the issues we have highlighted thus far. Psychological safety may be just one of many ingredients needed to create a more equitable and just academy.

Conversations around supporting psychological safety naturally fit within broader goals of increasing inclusion and belonging within our field and open science. Psychological safety is an integral part of inclusive climates—when people feel safe to share, ask questions, and voice concerns, in a space that is welcoming of their identities and backgrounds, they are likely more open to joining open science communities and engaging in the transparency and error management practices associated with open science. Research by Singh and colleagues (2013) supports this idea: psychological safety is associated with increased performance among racially diverse teams. However, the relationship between psychological safety and diversity, equity, and inclusion (DEI) goals is likely reciprocal: Building psychological safety within open science supports DEI and DEI initiatives and climates supports psychological safety within open science.

There is support for this directionality (i.e., DEI supporting psychological safety) as well: Building inclusive climates is associated with greater identification with the group and an increased willingness to both participate and help the group grow and learn (e.g., Singh et al., 2013). Parallel to the ideas posed by Castille et al. (2022) on how open science can support belongingness and inclusion, research on the intersection of DEI and psychological safety (e.g., Carmeli, 2007; Chen, et al., 2014; Newman et al., 2017) suggests that strong social networks, supportive mentors, and powerful allies facilitate the types of risk taking and community growth that supports open science practice.

We propose that we can both use psychological safety as a tool to help support and build inclusive research spaces, by creating a climate supportive of sharing and growth, and also use existing and ongoing efforts to support DEI to helping to build better psychological safety climates. Efforts to increase psychological safety and those working to increase DEI are working in tandem to support increased belongingness and participation within the open science community.

Tips for Bringing Psychological Safety Into Your Open Science Practice

Readers may be wondering: What can I do to help foster psychological safety in my lab, research teams, and community to promote the adoption of open science? We offer a number of specific recommendations:

• Be open about mistakes, including as a leader (e.g., Liu et al., 2014)
• Focus on the quality of relationships as a research team member (Carmeli & Gitell, 2009)
• Practice inclusive or transformational leadership (e.g., Frazier et al., 2017; Javed et al., 2019)
• Offer support as an organization or journal editor by, for example, creating mentoring programs for open practices (e.g., Chen et al., 2014)
• For further reading on psychological safety, we recommend reviews by Edmondson and Lei (2014), Frazier et al. (2017), and Newman et al. (2017).

Conclusion

No matter how (in)experienced an I-O scholar is, those who want to use more open science practices should be able to do so in an environment that is psychologically safe, where one can openly admit to honest mistakes, take risks, ask questions, and innovate. We hope that we can use I-O science on psychological safety to improve our science more broadly, in this case, by allowing for an easier adoption of open science practices. As we pursue opportunities to use open science practices and learn new ones, let us also feel psychologically safe to do so and to contribute to research that is more robust, reproducible, and transparent.

References

Abele-Brehm, A. E., Gollwitzer, M., Steinberg, U., & Schönbrodt, F. D. (2019). Attitudes toward open science and public data sharing: A survey among members of the German Psychological Society. Social Psychology, 50(4), 252–260. https://doi.org/10.1027/1864-9335/a000384

Baumeister, R. F. (2016). Charting the future of social psychology on stormy seas: Winners, losers, and recommendations. Journal of Experimental Social Psychology, 66, 153–158. https://doi.org/10.1016/j.jesp.2016.02.003

Bedeian, A. G., Taylor, S. G., & Miller, A. N. (2010). Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning & Education, 9(4), 715-725. https://doi.org/10.5465/amle.9.4.zqr715

Bergmann, B., & Schaeppi, J. (2016, July 12). A data-driven approach to group creativity. Harvard Business Review. https://hbr.org/2016/07/a-data-driven-approach-to-group-creativity

Carmeli, A. (2007). Social capital, psychological safety and learning behaviours from failure in organisations. Long Range Planning, 40(1), 30-44. https://doi.org/10.1016/j.lrp.2006.12.002

Carmeli, A., & Gittell, J. H. (2009). High-quality relationships, psychological safety, and learning from failures in work organizations. Journal of Organizational Behavior, 30(6), 709–729. https://doi.org/10.1002/job.565

Carmeli, A., & Zisu, M. (2009). The relational underpinnings of quality internal auditing in medical clinics in Israel. Social Science & Medicine, 68(5), 894–902. https://doi.org/10.1016/j.socscimed.2008.12.031

Castille, C. M., Cobb, H. R., Siegel, J. A., & Thomas, C. L. (2022). Opening up: Tips for fostering belongingness in our scholarly communities while encouraging open science. The Industrial-Organizational Psychologist, 60(1).

Chen, C., Liao, J., & Wen, P. (2014). Why does formal mentoring matter? The mediating role of psychological safety and the moderating role of power distance orientation in the Chinese context. International Journal of Human Resource Management, 25, 1112-1130. https://doi.org/10.1080/09585192.2013.816861 

Crüwell, S., van Doorn, J., Etz, A., Makel, M. C., Moshontz, H., Niebaum, J. C., Orben, A., Parsons, S., & Schulte-Mecklenbeck, M. (2019). Seven easy steps to open science: An annotated reading list. Zeitschrift Für Psychologie, 227(4), 237–248. https://doi.org/10.1027/2151-2604/a000387

Edmondson, A. (1999). Psychological safety and learning behavior in work teams. Administrative Science Quarterly, 44(2), 350-383. https://doi.org/10.2307/2666999

Edmondson, A. C., & Lei, Z. (2014). Psychological safety: The history, renaissance, and future of an interpersonal construct. Annual Review of Organizational Psychology and Organizational Behavior, 1(1), 23-43. https://doi.org/10.1146/annurev-orgpsych-031413-091305

Fanelli, D. (2010). Do pressures to publish increase scientists’ bias? An empirical support from US States data. PLoS ONE, 5(4), e10271. https://doi.org/10.1371/journal.pone.0010271

Frazier, M. L., Fainshmidt, S., Klinger, R. L., Pezeshkan, A., & Vracheva, V. (2017). Psychological safety: A meta-analytic review and extension. Personnel Psychology, 70(1), 113-165. https://doi.org/10.1111/peps.12183

Gabriel, A. S., Allen, T. D., Devers, C. E., Eby, L. T., Gilson, L. L., Hebl, M., Kehoe, R. R., King, E. B., Ladge, J. J., Little, L. M., Ou, A. Y., Schleicher, D. J., Shockley, K. M., Klotz, A. C., & Rosen, C. C. (2023). A call to action: Taking the untenable out of women professors’ pregnancy, postpartum, and caregiving demands. Industrial and Organizational Psychology: Perspectives on Science and Practice, 16(2).

Gopalakrishna, G., ter Riet, G., Vink, G., Stoop, I., Wicherts, J. M., & Bouter, L. M. (2022). Prevalence of questionable research practices, research misconduct and their potential explanatory factors: A survey among academic researchers in The Netherlands. PLOS ONE, 17(2), e0263023. https://doi.org/10.1371/journal.pone.0263023

Houtkoop, B. L., Chambers, C., Macleod, M., Bishop, D. V. M., Nichols, T. E., & Wagenmakers, E.-J. (2018). Data sharing in psychology: A survey on barriers and preconditions. Advances in Methods and Practices in Psychological Science, 1(1), 70–85. https://doi.org/10.1177/2515245917751886

Javed, B., Naqvi, S. M. M. R., Khan, A. K., Arjoon, S., & Tayyeb, H. H. (2019). Impact of inclusive leadership on innovative work behavior: The role of psychological safety. Journal of Management & Organization, 25(1), 117–136. https://doi.org/10.1017/jmo.2017.3

Kark, R., & Carmeli, A. (2008). Alive and creating: The mediating role of vitality and aliveness in the relationship between psychological safety and creative work involvement. Journal of Organizational Behavior, 30(6), 785–804. https://doi.org/10.1002/job.571

Ledgerwood, A., Hudson, S. T. J., Lewis, N. A., Jr., Maddox, K. B., Pickett, C. L., Remedios, J. D., Cheryan, S., Diekman, A. B., Dutra, N. B., Goh, J. X., Goodwin, S. A., Munakata, Y., Navarro, D. J., Onyeador, I. N., Srivastava, S., & Wilkins, C. L. (2022). The pandemic as a portal: Reimagining psychological science as truly open and inclusive. Perspectives on Psychological Science, 17(4). https://doi.org/10.1177/17456916211036654

Liu, S., Hu, J., Li, Y., Wang, Z., & Lin, X. (2014). Examining the cross-level relationship between shared leadership and learning in teams: Evidence from China. Leadership Quarterly, 25(2), 282–295. https://doi.org/10.1016/j.leaqua.2013.08.006

Livio, M. (2013). Don't bristle at blunders. Nature, 497(7449), 309-310. https://doi.org/10.1038/497309a

Miller, A. N., Taylor, S. G., & Bedeian, A. G. (2011). Publish or perish: Academic life as management faculty live it. Career Development International. https://doi.org/10.1108/13620431111167751 

Newman, A., Donohue, R., & Eva, N. (2017). Psychological safety: A systematic review of the literature. Human Resource Management Review, 27(3), 521-535. https://doi.org/10.1016/j.hrmr.2017.01.001

Schwartz, M. A. (2008). The importance of stupidity in scientific research. Journal of Cell Science, 121(11), 1771-1771. https://doi.org/10.1242/jcs.033340

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. https://doi.org/10.1177/0956797611417632

Singh, B., Winkel, D. E., & Selvarajan, T. T. (2013). Managing diversity at work: Does psychological safety hold the key to racial differences in employee performance? Journal of Occupational and Organizational Psychology, 86(2), 242-263. https://doi.org/10.1111/joop.12015

Strand, J. F. (2023). Error tight: Exercises for lab groups to prevent research mistakes. Psychological Methods. https://doi.org/10.1037/met0000547

Tenney, E. R., Costa, E., Allard, A., & Vazire, S. (2021). Open science and reform practices in organizational behavior research over time (2011 to 2019). Organizational Behavior and Human Decision Processes, 162, 218–223. https://doi.org/10.1016/j.obhdp.2020.10.015