A Peek Into the Online World: Evaluating the Current State of Online I-O Graduate Programs

Rebecca Grossman, Hofstra University; and Diana Sanchez, San Francisco State University

Table 2

Program Admissions, Curriculum, and Faculty

Note. N = 17 faculty/program directors from master’s online or hybrid programs.

*SIOP data (24 online/hybrid master’s programs, 67 in-person only master’s programs)

Table 3

Student Experience

Note. N = 74 students (31 master’s program students; 43 PhD program students). Responses from 5 students were removed
because they did not report their degree level.

Table 4

Alumni Experience

Note. N = 47 alumni (27 master’s program alumni; 20 PhD program alumni).

Discussion

In this section, we offer commentary on the current state of online programs based on our findings. When possible, we make comparisons between online and in-person programs by drawing from either the database generated through the SIOP website or from previous TIP articles focused on program evaluation. Specifically, we used several of the same survey items used by Vodanovich and colleagues (2018), who surveyed I-O program coordinators, and Acikgoz and colleagues (2018), who surveyed students and alumni. Notably, these prior articles focused on master’s programs only, thus anytime we make comparisons, we do so for only master’s programs, including for data from the SIOP database. Further, these articles did not distinguish between online and in-person programs in their data. However, we compared their lists of programs to our list of online programs and found that only about 10% of programs from our list appeared in their articles, and several of those have both online and in-person programs. Thus, we believe that, in addition to the present work, referencing the contents of these articles is useful for making rough comparisons between online and in-person programs. Although we summarize notable results and highlight some conclusions that can be drawn, we encourage readers to review the tables for more detailed information. 

In Table 1 we identified 54 programs (44 master’s, 9 doctoral, 1 graduate diploma) offered by 46 separate institutions. Twenty-seven of these were represented by our sample (participants could indicate the program with which they were affiliated, if they chose). Although our response rate was low, we are encouraged that our results represent approximately half of current online programs (and considering that many respondents did not identify their program, more programs may be represented than of which we are aware). Most degrees were titled I-O Psychology (22), whereas 12 were psychology degrees with a concentration or specialization in I-O, 6 in Organizational Psychology, and 3 included an I-O concentration with a related degree (e.g., Business Psychology). The remaining 10 were in peripheral fields (e.g., Leadership & Management). Regarding format, 40 programs were online (33 master’s, 7 doctoral), 9 were hybrid (7 master’s, 2 doctoral), and 5 offered both formats (all master’s).

We generally found that websites were transparent about their degree requirements and core curriculum with some exceptions. By reviewing offered coursework, we observed some broad trends. Fittingly, direct I-O degrees tended to offer more I-O specific courses, and, similarly, psychology degrees with I-O concentrations offered more I-O-related courses than programs in peripheral fields. Broad degrees in related fields tended to offer few courses related to I-O (around 4 classes). Some curricula appeared more strongly aligned with SIOP standards, with some specifically stating they use the SIOP training guidelines (SIOP, 2016) to align their courses with training standards (e.g., Eastern Kentucky University, Southern New Hampshire University). This was in contrast to other programs that deviate from coursework norms and standards outlined by SIOP. For example, one program has a course dedicated to the Myers-Briggs Type Indicator, which falls outside of typical I-O training programs.

In Table 2, we presented information about programs’ admissions, curricula, and faculty. Starting with admissions criteria, we included direct comparisons between online and in-person programs. In general, online programs tend to have less stringent admissions criteria, as fewer programs have requirements regarding minimum GRE scores, GPA, previous research experience, and so on, with the exception of personal statements—online programs were more likely to require those. Online programs required a slightly lower GPA (2.96_online vs. 3.02_in-person), had higher acceptance rates (70%_online vs. 47%_in-person), and higher annual enrollments (46_online vs. 11_in-person), though the statistical significance of these differences was not assessed.

Items 13–18 suggest that online programs are providing ample opportunities for applied experiences—the majority of students (93%) are performing internships, and over half of the programs reported having a practicum component. Students are also getting exposure to applied content to some extent in their courses. Notably, no programs report having a designated unit for consulting or grant projects. To make comparisons to the applied score provided by Vodanovich et al. (2018), we followed their methodology and converted items in this dimension to a 0 to 1 scale, then averaged  (see their article for more detail on the scoring methodology). Overall, the applied dimension for these online programs (M = .40, SD = .28) was substantially lower than that reported by Vodanovich et al. (2018) of largely in-person programs (M = .59, SD = .20).

To evaluate curricula, we asked faculty to report on the extent to which various topics previously identified by SIOP’s E&T Committee as competencies for inclusion in I-O programs (SIOP, 2016) are covered in their programs. As can be seen in Table 2, each of the topics were covered to at least some extent in the online programs included in our data. On a scale of 1–3 capturing the extent to which each topic is covered, average scores ranged from 1.50 to 2.90, with workforce aging receiving the least coverage and training and development receiving the most coverage. Although we don’t have data to make comparisons to in-person programs at this degree of granularity, it is encouraging to see that these online programs are covering most primary I-O topics to at least a moderate extent, with none completely absent. We again followed the methodology of Vodanovich et al. (2018) so that comparisons could be made. Our overall curriculum dimension (M = .66, SD = .14) was slightly lower for these online programs than what they reported for largely in-person programs (M = .71, SD = .15).

Faculty-focused questions (items 43, 44; 49–51) showed that online programs tend to have limited numbers of full-time faculty available and rely more heavily on adjunct instructors. Encouragingly, these faculty appear to be fairly active in research and consulting. Items were again converted and combined using Vodanovich and colleagues’ (2018) approach.  Results showed our total (M = .44, SD = .35) for the faculty information/experience dimension was slightly lower for online programs than what has been previously reported for largely in-person programs (M = .52, SD = .17).

Finally, questions about teaching strategies (items 52–62) revealed that faculty in online programs are primarily relying on discussion boards, written materials, and emails, with synchronous forms of individualized communication (e.g., instant messaging) being much less common. Although discussion boards have been touted as a valuable tool for keeping students engaged (Waterhouse, 2005; Wiese et al., n.d.), heavy use of asynchronous approaches may be cause for concern (Laato & Murtonen, 2020). For example, asynchronous discussions have been described as lacking in communication opportunities and potential to be psychologically motivating (Hrastinksi, 2008). Nonetheless, moderate reports of video lectures and meetings indicate that students are getting at least some direct interactions with their professors in these programs.

In Table 3, we presented information pertaining to students’ experiences, distinguishing between master’s and doctoral programs. For three dimensions (viz. program culture, program resources, student satisfaction), we used the same items used by Acikgoz and colleagues (2018) in their article ranking master’s I-O programs to facilitate comparisons. For program culture (items 1–14), students generally had positive reactions to their programs, reporting the highest ratings for being treated fairly, for unhealthy competition (reverse scored), and for the acceptance of students with diverse backgrounds. Scores were generally similar between the master’s and doctoral programs. Program culture scores were generally lower (master’s M = 3.87, SD = 0.21) for online programs than were those reported by Acikgoz and colleagues of largely in-person programs (M = 4.32, SD = 0.29).

For questions pertaining to program resources (items 15–27), most students felt that their program offers them access to the resources they need, with master’s scores generally being higher than doctoral. The most notable ratings were the relatively low scores provided on availability of certifications and training outside of program courses, and for providing funding for professional development. The highest ratings were for access to library resources. The overall resources score (master’s M = 3.16, SD = 0.20) for online programs was lower than that reported by Acikgoz and colleagues (2018; M = 3.98, SD = 0.33) for largely in-person programs.

Student satisfaction (items 28–45) was relatively high, with the lowest satisfaction reported for internships, professional opportunities, and alumni engagement. The highest reported satisfaction was for class size. Overall satisfaction for online programs (master’s M = 3.77, SD = 0.18) was lower than that reported by Acikgoz and colleagues (2018) of largely in-person programs (M = 4.21, SD = 0.32).

For social connectedness, students generally provided the lowest scores on these items. These questions focused on feelings of connection with faculty and other students and how connection is promoted by the program. The lowest ratings were for research connections with faculty beyond coursework and for interactions with students outside of class. The highest ratings were for help from professors on class coursework. Generally, scores were higher from doctoral (M = 2.90, SD = 0.13) versus master’s students (M = 2.56, SD = 0.20). These generally lower ratings show there is an opportunity for online programs to support and promote connectedness between students and with faculty. This may be particularly important because interpersonal skills, which can be developed through social connections, have been identified as major contributors to employability and career success in the field of I-O (Hogan et al., 2013).

Last, in Table 4 we reported the results of our alumni survey. Regarding opportunities and professional development, alumni reported limited, but at least some, involvement in conference presentation and publications, with alumni from PhD programs showing slightly higher averages than those from master’s programs. Encouragingly, it appears that alumni from both levels attended SIOP at least once while in the program, and both reported involvement in numerous applied projects (with PhD alumni reporting approximately double the amount compared to master’s alumni). In stark contrast to the faculty survey, which indicated that over 90% of students perform internships, very few of our respondents said they had an internship while in the program. This discrepancy could indicate online programs are rapidly changing in that area, as alumni were reporting on past experiences and faculty were reporting on current practices. Another possibility is that alumni who did not get internship experience were particularly motivated to respond to this survey.

Alumni from both levels indicated that their programs facilitated their professional development and prepared them to enter the workplace to moderate degrees, but programs appeared to be less effective in terms of helping them develop their professional networks. Nonetheless, alumni reported they were able to obtain employment following graduation fairly quickly, particularly PhD alumni, and obtained starting salaries roughly commensurate with the overall (i.e., across all types of graduates) salary survey (SIOP, 2020). Online programs appear to be better suited for launching alumni into applied- versus academic-focused positions, which is consistent with past research showing strong preferences for candidates from traditional, compared to online, programs for academic positions (e.g., Adams & Defleur, 2005). Further, alumni appear to maintain little involvement with SIOP or publishing research post graduation.

Finally, several items were used to assess alumni satisfaction (items 15–24). Notably, overall satisfaction (M = 3.61, SD = .82) for online programs was lower than that reported by Acikgoz and colleagues (2018) of largely in-person programs (M = 4.10, SD = .31). However, high means on individual items indicate that alumni are proud of their programs and would encourage others to apply to them.

Conclusions

Online graduate programs in I-O were increasing even before the pandemic, and now that all are dipping their toes into the online world to some extent, the time is ripe to take stock of current programs. We presented information pertaining to various aspects of current online programs, revealing many areas where they appear to be excelling (e.g., covering I-O curriculum, access to certain resources) yet others where they may be lagging behind traditional programs, and there is room for improvement (e.g., admissions standards, satisfaction). Notably, there is a lot of variance surrounding these programs, and we also acknowledge our small sample size and apparent lack of representation from many programs, limiting the extent to which we can make broad generalizations. Nonetheless, we believe we have provided a valuable resource and starting point for those interested in learning more. We hope this initial peek into the world of online programs is useful to those in the SIOP community (e.g., prospective students, employers) seeking to understand what it currently means to obtain an I-O degree from an online or hybrid program. We welcome any feedback from readers to inform the ongoing efforts of the Online Program Subcommittee.

Note

^[1] If you have already read “Challenges of Educating and Training I-O Graduate Students Online (Not Another COVID Story)” in this issue, you’ll find the next three paragraphs quite familiar. Feel free to pick up again in the Results section. — Ed.

References

Acikgoz, Y., Huelsman, T. J., Swets, J. L., Dixon, A. R., Jeffer, S. N., Olsen, D. R., & Ross, A. (2018). The cream of the crop: Student and alumni perceptions of I-O psychology master’s degree program quality. The Industrial-Organizational Psychologist, 55(4). https://0-www-siop-org.library.alliant.edu/Research-Publications/TIP/TIP-Back-Issues/2018/April/ArtMID/20647/ArticleID/1397/The-Cream-of-the-Crop-Student-and-Alumni-Perceptions-of-I-O-Psychology-Masters-Degree-Program-Quality

Adams, J., & DeFleur, M. H. (2005). The acceptability of a doctoral degree earned online as a credential for obtaining a faculty position. American Journal of Distance Education, 19, 71–85. https://doi.org/10.1207/s15389286ajde1902_2

Chirikov, I., Semenova, T., Maloshonok, N., Bettinger, E., & Kizilcec, R. F. (2020). Online education platforms scale college STEM instruction with equivalent learning outcomes at lower cost. Science Advances, 6(15), eaay5324. https://doi.org/10.1126/sciadv.aay5324

Hogan, R., Chamorro‐Premuzic, T., & Kaiser, R. B. (2013). Employability and career success: Bridging the gap between theory and reality. Industrial and Organizational Psychology, 6(1), 3–16.

Hrastinski, S. (2008). Asynchronous and synchronous e-learning. Educause Quarterly, 31(4), 51–55.

Laato, S., & Murtonen, M. (2020, April). Improving synchrony in small group asynchronous online discussions. In A. Rocha, H. Adeli, L. P. Reis, S. Costanzo, I. Orovic, & F. Moreira (Eds.), Trends and innovations in information systems and technologies (pp. 215–224). Springer.

Society for Industrial and Organizational Psychology, Inc. (2016). Guidelines for education and training in industrial-organizational psychology. Bowling Green, OH: Author.

Society for Industrial and Organizational Psychology. (2020). Income & employment report. https://0-www-siop-org.library.alliant.edu/Portals/84/PDFs/Surveys/SIOP_TI_Income-and-Employment_Report.pdf?ver=2020-09-22-152339-387

Vodanovich, S. J., Morganson, V. J., & Kass, S. J. (2018, March 26). Ranking I-O master's programs using objective data from I-O coordinators. The Industrial-Organizational Psychologist, 55(4). https://0-www-siop-org.library.alliant.edu/Research-Publications/TIP/TIP-Back-Issues/2018/April/ArtMID/20647/ArticleID/1398/Ranking-I-O-Masters-Programs-Using-Objective-Data-From-I-O-Coordinators

Waterhouse, S. (2005). The power of eLearning: The essential guide for teaching in the digital age. Pearson Education. https://www.pearson.com/us/higher-education/program/Waterhouse-Power-of-e-Learning-The-The-Essential-Guide-for-Teaching-in-the-Digital-Age/PGM11730.html

Wiese, C. W., Shuffler, M. L., Sanchez, D. R., Sanders, A. M. F., & Mendelson, R. (n.d.). Need help transitioning to online classes? SIOP is launching a dynamic resource guide for faculty & students. SIOP. https://0-www-siop-org.library.alliant.edu/Events-Education/Online-Teaching-Survival-Guide