Opening Up: Tips for Fostering Belongingness in Our Scholarly Communities While Encouraging Open Science

Christopher M. Castille, Nicholls State University; Haley R. Cobb, Saint Louis University; Jaclyn A. Siegel, San Diego State University; & Candice L. Thomas, Saint Louis University

Author Note: This article was made possible with the support of Larry Williams, director of the Consortium for the Advancement of Research Methods and Analysis (CARMA), who created an Open Science Topic Interest Group (TIG) for the Spring of 2022. Four authors (Chris, Haley, Jaclyn, and Candice) met during the April meeting of the Open Science TIG, where the ideas for this article first emerged.

For this entry into Opening Up, TIP’s column on all things open science, we discuss the relation between two broad domains of interest to scholars in our community: (a) diversity, equity, inclusivity—which we use to define the term “belongingness” as a point of focus—and (b) open science. We refer to the former, belongingness, as a universal need to be accepted, interacted with in an equitable manner, and valued by others (Thau et al., 2007). We refer to the latter, open science, as a movement broadly aimed at encouraging scholars to apply principles, enact scientific values, or use tactics aimed at enhancing transparency and replicability in our science (see Castille et al., 2022). Examples of open science in practice include preregistration of study methods and hypotheses or openly sharing data and code for data analysis, and some of these practices are easier to adopt with certain methodologies and epistemologies than others. We examine the relation between belongingness and open science because over the past several years, scholars have debated whether open science can meaningfully address the needs of individuals from diverse backgrounds (e.g., demographic, such as race, sex, and their intersection; philosophical, such postpositivist, critical theory) or if historical biases and disadvantages will be passed on (unless large scale systemic change takes place). In other words, open science as a movement has sought to increase openness and transparency in scholarly research. However, whether scholars feel included in this movement, and whether those who want to participate in science feel empowered to do so, demands attention (Ledgerwood et al., 2022). For instance, scholars have coined the phrase “bropen science” to characterize certain male scholars whose tactics exclude others from our scientific community (Whitaker & Guest, 2020). Leaders across a variety of scholarly domains agree that there are many praiseworthy aspects to the open science movement (Antonakis, 2017; DeCelles et al., 2021; Eby, 2022; Pratt et al., 2020). We hope to shed light on small ways scholars (as well as the populations they serve) might foster a greater sense of belongingness in our work (or at least provide tactics worthy of debate).

In our aim to foster a sense of belongingness in our academic communities, we refer not only to our scholarly communities (e.g., our academic departments, SIOP, the committees we serve on) but also the communities that we impact or serve (e.g., the populations we sample and notably our participants who provide us with data). In order to create an open science that is truly open and equitable to all, respecting the wishes and prioritizing the safety of underrepresented academics, as well as participants, is essential (see Siegel, 2022). To that end, we offer the following tips.

Be Proactive Within Your Department/College About Flexibly Adopting Open Science, and Ask What Will Be Rewarded

Ask whether open science can be rewarded flexibly. For instance, although open science badges are in vogue in a variety of journals (e.g., Psychological Science), there are areas of scholarship where there are no such signifiers of open science practice, which may unwittingly be construed as less rigorous in our community (e.g., qualitative research; see Pratt et al., 2020; Siegel & LaMarre, 2019). Although we laud efforts to praise open science practice to promote transparent, rigorous, and reproducible research, it is important to keep in mind that open science can take many forms (see Castille et al., 2022). It is possible that practices that reward certain types of scholarship may inadvertently signal what is valued by the academic community. Scholars, professionally motivated to conduct research that is regarded as high quality, may be incentivized to conduct research that “fits the mold” of open science (see Grzanka & Cole, 2021). Therefore, it is essential that we expand the mold and carve out a variety of different options and templates for scholars doing various types of scholarship.

Open science practices can be time intensive and often fall outside of traditional promotion requirements (see Briker & Gerpott, 2022). This often means that when there is high publication pressure, engaging in open science may directly compete with publication (i.e., taking time to provide quality open research products such as code or data may reduce the time to work on other data collections or manuscripts). As a key aim of the open science movement is to separate the process of science (e.g., explaining how decisions were arrived at) from its products (i.e., conclusions, insights, publications; see Grand et al., 2018), it is therefore important, particularly for junior faculty, to ask their superiors if existing structures and decision-making processes (e.g., merit increases, tenure and promotion) will reward a scholar for enacting open-science practices (e.g., preregistering study hypotheses, sharing analysis code). Although practicing open science is not the same as earning badges upon publication (which may not apply to certain areas of scholarship), it is important to consider what will and will not be evaluated favorably.

The combination of time investment for quality open science and lack of inclusion of open science within many organizations’ performance evaluation may be considered as a significant barrier for junior faculty or researchers outside of academia, particularly if open science is represented as all or nothing or as requiring “gold standard” levels of participation to receive recognition. To sustainably incorporate open-science practices into our research, we need to weigh the benefits of particular open-science practices with the logistical demands of our jobs (e.g., promotion, time restraints, or resources). How can we best align our open science practices with our job requirements? How can we engage in open science in a way that both supports the open science principles and our own career trajectories? Conversations with supervisors and mentors about this is a good first step. But it is also important, in building a more inclusive community, that we recognize a more flexible view of open science practices and understand that a researcher's ability to engage in open science is likely influenced by their job level, resources, and demands. 

Notably, this first tip dovetails nicely with our second tip.

Be Selective in Which Open Science Tactics You Adopt

There is a broad buffet of open-science tactics that scholars can choose to enact (or not to; see Castille et al., 2022), some of which can be more time and labor intensive than others (e.g., registered reports, learning a new tool for openly sharing data and reproducible code). Especially as a graduate student, early career researcher, or someone who conducts research with underrepresented methodology or explores phenomena within marginalized or protected populations, it is important to consider how realistic it will be to adopt open science practices. Although the all-or-nothing approach to open science is often championed, scholars face understandable constraints on their ability to engage in open-science practices, and due to misconceptions around open science, scholars also hope to avoid being unduly punished for adopting tactics selectively. For instance, sharing data openly can potentially compromise research ethics and study integrity, particularly for qualitative scholars (Pratt et al., 2020). Though there are efforts to strengthen research practices (e.g., providing templates for preregistering both quantitative and qualitative works), it is worth considering which tactics are worthwhile to pursue or help a scholar to enact their core values (see Castille et al., 2022).

To that end, consider the many tactics available for practicing open science, as well as sharing insights via the published scholarly literature. Although open science may seem rigid and inflexible at first blush, we view open science as a set of tools that allows us to communicate more effectively about the scholarship that we contribute to our fields. To us, this is not rigid and inflexible but rather something that aligns with our values as academic scholars, and we attempt to put values into practice where possible, ethical, and reasonable. We encourage readers to consider adopting an open-science mindset, where one actively considers open-science practices (or values) throughout the research process as well as advocates for open science broadly (Hagger et al., 2022). Being committed to the open-science movement, and understanding how it aligns with one’s values as a scientist, may ease the transition into practicing open science in light of the challenges one may face, thus upholding a sense of belonging despite any hurdles.

As stated, there are many ways to practice open science and publish insights. For instance, several journals have adopted registered reports (where articles are conditionally accepted before data are gathered), including Leadership Quarterly, Journal of Organizational Behavior, Journal of Personnel Psychology, and the International Journal of Selection and Assessment. Although registered reports are a rigorous way of practicing open science, such a high bar need not be met by every scholar or every project for us to progress in our journey of opening up our science. There may be particular areas where a registered report is useful, such as (a) where a high-powered replication study is relevant regardless of the outcome because it provides deeper insight into phenomenon discussed in the literature and (b) for topics that hold society or practical relevance (e.g., employee reactions to artificial intelligence; Briker & Gerpott, 2022). However, as an alternative, several journals use results-blind reviews where only the literature review and methods of a study are evaluated and, if accepted by reviewers, then the findings will be published regardless of whether study hypotheses are supported (e.g., Journal of Business and Psychology, European Journal of Work and Organizational Psychology, Leadership Quarterly, Organizational Research Methods).

Although both registered reports and results-blind reviews are important tactics in the open science skillset, they are merely two tactics we are sampling from the broader buffet. The key idea is to pick something that fits with your aims as a scholar, as well as what is reasonably feasible for you, your research team, your research, and your participants, and to explain or justify why these tactics were chosen.

Find Allies in Your Efforts to Open Up (There May Be More Out There Than You May Think)            

When finding small ways to put widely shared values into practice, it’s important to find allies. In that spirit, we put ourselves to you as colleagues who are willing to support you in your efforts to enact open science, so we encourage you to reach out to us if you are in need of allies. We have also been fortunate to experience the support from other more senior members of our community who have sought to open up our science, notably Fred Oswald, Steven Rogelberg, George Banks, Larry Williams, (who have supported Chris in his work), Cort Rudolph (who has supported Haley Cobb and Candice Thomas), and Lorne Campbell (who has supported Jaclyn Siegel). These scholars have played such a supporting role in our work and have also sought to enhance the uptake of open science practices within our community. Reforming the academy to be more open and transparent while also helping scholars and historically disadvantaged populations is no easy feat. If each of us actively commits to doing something small, then a rising tide will lift all boats. These small efforts become more manageable with the support of senior scholars who can validate our work, as well as dispel myths and rumors about practicing open science. For instance, Briker and Gerpott (2022) find that there are many myths surrounding registered reports (e.g., it is a myth that registered reports are suitable for experimental studies only).

In addition to finding experts who may serve as champions of open science, we also find value in open science communities, such as SIOP’s Open Science and Practice Community, the Society for the Improvement of Psychological Science (SIPS), and ReproducibiliTea (reproducibilitea.org). Connecting with or joining these communities may also help foster a sense of belongingness, as these are established groups that one can be part of to learn more about open science.

Include Members From the Populations We Serve in Your Efforts to Open Up

Thus far, we’ve considered tips for fostering a sense of belongingness in our academic communities (e.g., department, college, university). However, there is an important stakeholder to our work that must be considered when deciding to enact almost any open science tactic: participants from the populations that we serve. Not only must data sharing involve gaining their enthusiastic consent and be considered with regard to impacts on study quality (see Pratt et al., 2020), our participants can be important sources of insight into the very phenomena we wish to probe directly. For example, in a recent qualitative project on lactating employees on which coauthor Candice Thomas is working, de-identified interview transcripts were reviewed prior to publicly sharing and then shared collectively to all study participants. Participants were given the link to the “raw results” and encouraged to review and provide commentary. Through this process, participants engaged in conversation with the researchers and each other about their experiences. By including participants in the data sharing process, we were able to build a richer understanding of the study phenomena and simultaneously foster a sense of community within the population our research is trying to support.

A Note for Practicing Open Science…as a Practitioner

Much of the open science literature centers around academia and academic scholarship. However, a large portion of SIOP members are not, or do not plan to stay, in the academy, and because the central tenet of this article is to support belongingness in open science, we hope to contribute some sense of belongingness to all of our potential readers and not just those in the academy. We also recognize that academic work informs applied work, and bridging the scientist-practitioner gap can be challenging due to a lack of access to academic scholarship (e.g., paywalls). For students intending to pursue careers in practice, being committed to rigorous, transparent, and reproducible research is something in which we believe practitioners can (and should) also engage. We encourage practitioners to consider what open science practices allow them to enact their values and make their work any more rigorous, transparent, and reproducible. Find mentors and communities of support; and consider what is feasible with the work you do. For example, for many, openly sharing data will not be possible (e.g., protected employee data), but sharing reproducible code with colleagues or sharing important descriptive statistics can increase the transparency of one’s work.

A Note on Positionality

We wish to conclude this piece by acknowledging the ways in which our scope as a team broadens and narrows our capacity to meaningfully engage with topics such as diversity and belongingness. Notably, we represent a fairly gender and career-diverse team: Our research team comprises three cisgender women and one cisgender man ranging in career stages from graduate student to assistant professor. However, as an exclusively White group of scholars, we recognize that individuals from minoritized racial backgrounds likely face additional barriers to full engagement and sense of belonging in open science, and academia more broadly, that we as a team cannot conceptualize or fully understand (see Ledgerwood et al., 2022). It is our hope that scholars of color will generously continue to build on these ideas and supplement our commentary with their own. We also hope that our entry into opening up helps our readers, especially those from marginalized backgrounds, find allies. We offer tips for other scholars who, like us, want to play a productive role in fostering a sense of belongingness in our communities. Although we do not pretend to solve all the systemic issues that make a truly open science, we hope that this conversation makes an important contribution in the right direction.

Conclusion

Although open science may not currently be perceived as a highly inclusive endeavor, we think most scholars would agree that so long as the core values that differentiate science from pseudoscience are held, then everyone—regardless of philosophy or background—deserves to belong in this community. Open science advocates may agree that the movement can be perceived as not an inclusive one, but certainly aspire for it to be inclusive. We also encourage our readers to reach out and connect with us. If you wish to contribute or comment on this article anonymously, we offer the following Qualtrics link: https://slu.az1.qualtrics.com/jfe/form/SV_7aFEPWcG30rPzAa

References

Antonakis, J. (2017). On doing better science: From thrill of discovery to policy implications. Leadership Quarterly, 28(1), 5–21. https://doi.org/10.1016/j.leaqua.2017.01.006

Briker, R., & Gerpott, F. H. (2022). Publishing registered reports in management and applied psychology: Myths, truths, and best practices. Manuscript in preparation.

Castille, C. M., Kreamer, L. M., Albritton, B. H., Banks, G. C., & Rogelberg, S. G. (2022). The Open Science Challenge: Adopt one practice that enacts widely shared values. Journal of Business and Psychology, 37, 459-467. https://doi.org/10.1007/s10869-022-09806-2

DeCelles, K. A., Howard-Grenville, J., & Tihanyi, L. (2021). From the editors-improving the transparency of empirical research published in AMJ. Academy of Management Journal, 64(4), 1009-1015. https://doi.org/10.5465/amj.2021.4004

Grand, J. A., Rogelberg, S. G., Banks, G. C., Landis, R. S., & Tonidandel, S. (2018). From outcome to process focus: Fostering a more robust psychological science through registered reports and results-blind reviewing. Perspectives on Psychological Science, 13(4), 448-456. https://doi.org/10.1177/1745691618767883

Grzanka, P. R., & Cole, E. R. (2021). An argument for bad psychology: Disciplinary disruption, public engagement, and social transformation. American Psychologist, 76(8), 1334-1345. https://doi.org/10.1037/amp0000853

Eby, L. T. (2022). Reflections on the Journal of Applied Psychology in times of change. Journal of Applied Psychology, 107(1), 1-8. https://doi.org/10.1037/apl0001000

Hagger, M. S. (2022). Developing an open science “mindset.” Health Psychology and Behavioral Medicine, 10(1), 1-21. https://doi.org/10.1080/21642850.2021.2012474 

Ledgerwood, A., Hudson, S. T. J., Lewis, N. A., Maddox, K. B., Pickett, C. L., Remedios, J. D., Cheryan, S., Diekman, A. B., Dutra, N. B., Goh, J. X., Goodwin, S. A., Munakata, Y., Navarro, D. J., Onyeador, I. N., Srivastava, S., & Wilkins, C. L. (2022). The pandemic as a portal: Reimagining psychological science as truly open and inclusive. Perspectives on Psychological Science. Advance online publication. https://doi.org/10.1177/17456916211036654

Pratt, M. G., Kaplan, S., & Whittington, R. (2020). Editorial essay: The tumult over transparency: Decoupling transparency from replication in establishing trustworthy qualitative research. Administrative Science Quarterly, 65(1), 1-19. https://doi.org/10.1177/0001839219887663

Siegel, J. A. (2022, May 12). Open science, closed doors. https://doi.org/10.31234/osf.io/sv8qh

Siegel, J. A., & LaMarre, A. (2019, October 10). Navigating “publish or perish” as qualitative researchers. Nature behavioural and social sciences. https://socialsciences.nature.com/posts/54648-navigating-publish-or-perish-as-qualitative-researchers

Thau, S., Aquino, K., & Poortvliet, P. M. (2007). Self-defeating behaviors in organizations: The relationship between thwarted belonging and interpersonal work behaviors. Journal of Applied Psychology, 92(3), 840-847. https://doi.org/10.1037/0021-9010.92.3.840

Whitaker, K., & Guest, O. (2020). #bropenscience is broken science. Psychologist, 33, 34-37. https://thepsychologist.bps.org.uk/volume-33/november-2020/bropenscience-broken-science

​​