Same Degrees, Different Requirements: The Variety of Comprehensive Requirements (“Comps”) in I-O Psychology Master’s and Doctoral Programs

Rebecca M. Brossoit, Oregon Health & Science University/Colorado State University; Jacqueline R. Wong, Colorado State University; & Gwenith G. Fisher, Colorado State University/Colorado School of Public Health

Contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention or the Department of Health and Human Services.

In 1985, 1999, and 2016, the Society for Industrial and Organizational Psychology (SIOP) published the Guidelines for Education and Training in Industrial-Organizational Psychology (SIOP, 2016). These guidelines assist faculty as they develop and update curricula for I-O graduate programs at both the master’s and doctoral level. Twenty-six competencies, as well as seven training methods for instilling I-O content knowledge (e.g., formal coursework), are included in the guidelines. However, the Guidelines for Education and Training in I-O do not identify the process by which graduate programs ought to evaluate mastery of I-O psychology knowledge and skills.

Comprehensive, preliminary, or qualifying requirements (also known as “comps,” “prelims,” or “quals”) are used by many graduate programs to assess students’ knowledge and skills, qualify students to advance from master’s to doctoral degree training, and evaluate graduate training methods or curricula. These requirements are not unique to I-O psychology programs; many areas of psychology administer comprehensive exams to graduate students (e.g., Kostohryz, 2016). A common goal of I-O psychology graduate training is to develop students as scientist–practitioners with a breadth of knowledge and skills that can be applied across a wide range of jobs and settings. Unlike clinical and counseling psychology, graduate training programs in I-O psychology are not accredited by the American Psychological Association, and I-O psychology program graduates are not all required to pursue licensure for practice. Thus, comprehensive requirements may help ensure that graduate students can demonstrate sufficient mastery of I-O psychology knowledge and skills across the recommended 26 competencies (SIOP, 2016).

Anecdotally, we are aware that various comprehensive requirement formats (e.g., written exams, applied experiences) are used across I-O graduate programs. Although SIOP offers guidelines about education and training content, little has been specified regarding assessment methods. For example, comprehensive requirement(s) or exams are not mentioned as a training method in the Guidelines for Education and Training in I-O (SIOP, 2016). In lieu of formal recommendations, individual programs have developed their own unique comprehensive requirements and processes, relying on previous experiences and/or feedback from students and faculty in the field. Therefore, we undertook an exploratory empirical study to investigate the types of comprehensive requirements in master’s and doctoral I-O programs, as well as the intended purpose of the comprehensive requirements, in a more systematic manner. We sought to identify the comprehensive requirements used in I-O graduate programs and report results to I-O psychology program faculty as well as current and prospective graduate students.

Method

In the spring of 2020, we sent email invitations to 143 I-O master’s and doctoral program directors with a request to participate in a 10-minute survey about the curriculum and educational requirements in their program. We gathered graduate program director contact information via the online SIOP Graduate Training Programs in I-O Psychology and Related Fields search engine. Next, we checked the information we obtained with information available on I-O graduate program websites. To account for the possibility that program director information was not up to date, faculty who received the invitation email were asked to forward the survey to the current program director (if applicable). We reported results specific to graduate training in ethics from this data collection in the winter 2021 issue of TIP (Brossoit et al., 2021).

In the survey, we asked I-O master’s and doctoral program directors to share information about whether their program has a comprehensive/preliminary/qualifying exam or project requirement for graduate students. Those who indicated that their graduate program has a comprehensive requirement then answered follow-up questions regarding the type of requirement (e.g., written exam, oral exam, applied experience) and related details about the requirement(s) (e.g., whether reading lists are provided to students, what happens if a student fails). Given that the authors did not presume to know all possible purposes of these requirements, we asked program directors to describe the intended purpose(s) of the comprehensive requirements via an open-ended question. Program directors also indicated the extent to which they believe the comprehensive requirement fulfills its intended purpose and the extent to which it could be improved.

Results

Half of the I-O program directors that were contacted completed the survey (i.e., 71 of 143 program directors). Program directors represented master’s only (63%), doctoral only (15.5%), and combined master’s/doctoral (32%) programs¹ that were conducted primarily in person (68%) compared to online (4%), or a hybrid of in person and online (20%),² prior to COVID-19, and were predominantly located in the United States (86%). When examined by degree type offered, 87% of I-O master’s program directors and 90% of I-O doctoral program directors (i.e., master’s/doctoral combined and doctoral-only programs) reported having a comprehensive or qualifying requirement.

Perceived Purpose and Quality of Comprehensive Requirements

I-O program directors listed several different purposes of the comprehensive requirements, including testing the breadth and depth of students’ mastery of content knowledge across core I-O competencies and evaluating students’ abilities to integrate and apply theory and content across domains of I-O. Other program directors reported that the purpose is for students to gain practical experience (e.g., in capstone projects) or to ensure students can demonstrate skills ranging from those needed for research and/or practice to more general skills (e.g., communication, networking, professionalism, problem solving) in preparation for different career paths. Further, some program directors identified that comprehensive requirements were intended to identify whether students are sufficiently prepared to complete doctoral work (e.g., the dissertation) as a form of “quality control.” See Figure 1 for a visual depiction of the variety of comprehensive requirement purposes that were identified by program directors.

Figure 1
Purposes of the Comprehensive Requirement in I-O Graduate Programs

Note. Figure includes words that were mentioned at least twice. The size of a word reflects the frequency of times the word was mentioned, with larger words being mentioned at a higher frequency. Some words (e.g., “students,” “although,” “other”) were filtered out for clarity.

Across master’s and doctoral programs, most (85%) program directors indicated that they believe their I-O program’s comprehensive requirement(s) accomplishes the intended purpose. However, slightly more than half of program directors indicated that they believe their I-O graduate program’s comprehensive requirement(s) could be improved (58% of master’s programs and 54% of doctoral programs).

Comprehensive Requirements in Master’s Programs

Of the 45 master’s programs represented in this study, 27% have a written exam requirement, 11% have an oral exam requirement, 31% have a research project requirement, 55.5% have an applied experience requirement, 15.5% have a personalized requirement for each student, and 4% have an “other” requirement³ (e.g., portfolio, analytic exam, see Figure 2). Of note is that some program directors indicated that their program has multiple requirements. The most common combination of requirements for master’s programs was to have an applied requirement and a research requirement. It was also somewhat common to have a combination of a written exam requirement and an applied requirement or an applied requirement with a personalized aspect.

Figure 2
Types of Comprehensive Requirements by I-O Graduate Program Degree

Written Exam Requirement

Of the 12 master’s programs with a written exam, most written exams are completed in person (83%) rather than take home (17%). Most in-person written exams occur within 2 days or less (90%), and the two programs with take-home exams give students 14 days to complete the exam. In most programs, the faculty schedule the exam date (92%). Most programs do not offer access to a reading list (67%), although two programs reported that they provide a reading list from faculty, and two programs allow students to compile the reading list. In most programs (67%), students are not allowed to use resources (e.g., notes, books, articles) when they take the written exam. In two programs, some questions on the written exam are tailored for individual students according to their minor and/or content area. If a student fails the written exam, programs either give students an opportunity to retake the full written exam the next time it is offered or give students an opportunity to rewrite the question(s) they failed. In one program, students are asked to leave if they fail. Some program directors shared additional details about what happens if a student fails the written exam (e.g., student retakes the section they failed with new questions, student writes an extended response to failed questions, the decision depends on a performance review).

Oral Exam Requirement

Among the five master’s programs that have an oral exam requirement, two programs have some questions on the oral exam that are tailored for individual students according to their minor and/or primary content area. Only one program provides students with a reading list. If a student fails the oral exam, programs either give students an opportunity to retake the full oral exam or give students the opportunity to provide written responses to the question(s) they failed. In one program, students are asked to leave if they fail.

Research Project Requirement

Of the 14 master’s programs with a research project requirement, no programs require a manuscript to be submitted for publication, though 36% of program directors reported that a manuscript submission is expected or highly encouraged.

Applied Experience Requirement

Of the 25 master’s programs with an applied experience requirement, such as an internship or practicum project, most (88%) require a minimum time commitment, ranging from 120–500 hours. Some programs require a supervised applied/consulting experience whereas others require a formal internship. Additionally, some programs require students to write a literature review as part of the applied experience requirement.

Comprehensive Requirements in Doctoral Programs

Of the 31 doctoral programs, 74% have a written exam requirement, 45% have an oral exam requirement, 16% have a research project requirement, 3% have an applied experience requirement, 13% have a personalized requirement for each student, and 10% have an “other” requirement (e.g., consulting simulation, portfolio, see Figure 2). Similar to master’s programs, doctoral program directors often indicated that their program has multiple requirements. The most common combination of requirements for doctoral programs is to have a written exam and an oral exam.

Written Exam Requirement

Of the 23 doctoral programs with a written exam, most written exams are completed in person (74%), rather than take home (22%) or a combination of in person and take home (4%). Most in-person written exams occur within 3 days or less (94%), take-home exams range from 2–21 days, and one program director indicated that students have “months” to complete the combined in-person and take-home written exam. In most programs, the faculty determine the exam date (65%). Approximately half of programs do not offer access to a reading list (48%), although 35% provide a reading list from faculty, and 17% allow students to compile the reading list. In most programs (69.5%), students are not allowed to use resources (e.g., notes, books, articles) when they take the written exam. Questions on the written exam are typically not tailored for each student (65%), though about one-third of doctoral programs (35%) have at least some questions on the written exam tailored for individual students according to their minor and/or content area. If a student fails the written exam, most programs give students an opportunity to retake the full written exam (74%). Otherwise, programs give students an opportunity to rewrite the question(s) they failed, give students the opportunity to orally defend the question(s) they failed, or determine retakes on a case-by-case basis. No programs ask students to leave the program if they fail.

Oral Exam Requirement

Of the 14 doctoral programs that have an oral exam requirement, most have some questions on the oral exam tailored for individual students according to their minor and/or content area (86%). Students are typically not provided with a reading list (71%). If a student fails the oral exam, most programs give students an opportunity to retake the full oral exam (86%) or give students the opportunity to provide written responses to the question(s) they failed. No programs ask students to leave if they fail.

Research Project Requirement

Of the five doctoral programs with a research project requirement, one program requires students to submit a manuscript for publication. However, the paper does not need to be accepted for publication to meet the research project requirement. Three doctoral programs with a research project requirement do not require a manuscript to be submitted for publication, though two program directors reported that a manuscript submission is expected or highly encouraged. One program director did not respond to the question about required manuscript submissions.

Applied Experience Requirement

Only one doctoral program has an applied experience requirement (e.g., internship or practicum project), which requires students to complete six credit hours.

Discussion

Comprehensive Requirements in I-O Graduate Programs

Overall, through our survey among I-O program directors in master’s and doctoral programs, we identified that comprehensive requirements (e.g., exams) are very common in I-O graduate programs. However, program directors within and between master’s and doctoral programs listed varied purposes for the comprehensive requirements. For example, the intended purposes ranged from demonstrating a mastery of I-O content and competencies to demonstrating skills needed for research or practice. Additionally, the formats for comprehensive requirements vary widely across programs and include written exams, oral exams, research requirements, applied experiences, personalized requirements, and other requirements like consulting simulations. Applied experiences are the most common comprehensive requirement in master’s programs, followed by research project and written exam requirements. In contrast, written exams are the most common comprehensive requirement in doctoral programs, followed by oral exams and research project requirements. We also found variability in the process for administering each of the requirements (e.g., tailoring of exam questions for individual students, procedure if a student fails). Taken together, the purposes, specific requirements, and formats of the comprehensive requirements differ substantially across I-O graduate programs.

Recommendations for the Field of I-O

Based on the findings from this study, we believe it is critical to first determine whether comprehensive requirements should be unique to each I-O graduate program (e.g., to meet specific program goals, for program evaluation) or if there should be some degree of standardization across programs. Answering this question could be achieved by surveying SIOP members, developing a subcommittee or task force within the SIOP Education and Training Committee, and/or creating panel discussions at future SIOP conferences. Regardless of whether the comprehensive requirements are developed by individual I-O graduate programs or standardized across the field, the purpose and goals of the requirement(s) should be thoughtfully and intentionally defined, aligned with the process, and communicated to students.

If the determination is that the comprehensive requirements in I-O graduate programs ought to be designed and implemented in a similar way across the field, then it may be worthwhile for the SIOP Education and Training Committee to systematically review the options for comprehensive requirements in I-O graduate programs (e.g., surveying program directors, students, and/or alumni, content analyzing formal program documents like manuals). From there, evidence-based recommendations could be developed and included in a revised version of the Guidelines for Education and Training in I-O. The establishment of common guidelines for the comprehensive requirements may foster more consistency across graduate programs regarding why and how comprehensive requirements are administered, assist program directors in determining how to effectively evaluate students, improve perceptions of procedural justice by prospective and current I-O graduate students, and ultimately improve the extent to which graduate students are prepared for success in their I-O careers.

Limitations and Future Research

Although this study advanced our knowledge and understanding about the comprehensive requirements in I-O graduate programs, it is not without some limitations. First, the survey questions were specific to I-O graduate program comprehensive requirements rather than general program requirements. Thus, some of the variability in responses may be due to the comprehensive requirement process being separate from broader program requirements. For example, a doctoral program may not require an internship as a comprehensive requirement to progress from master’s to doctoral training but may require students to gain applied experiences as an additional requirement of the program.

Second, the 50% response rate among program directors presents the possibility that not all programs were sufficiently represented. Program directors who chose to complete the survey may differ from those who did not in some systematic manner related to the questions or variables of interest. For instance, those who responded may be more involved in the comprehensive requirement process or more focused on or dedicated to improvements of I-O program curricula. Although having data from the other half of I-O graduate programs would be necessary to depict the full array of purposes, formats, and processes involved in the comprehensive requirement, our study sheds light on the range of requirements across programs, which may be even more varied if all programs were represented.

Third, we used single-source data collected from I-O program directors. It would be worthwhile to understand how comprehensive requirements are perceived by current students and alumni. Future work could examine what the experience of participating in the comprehensive requirement process is like for current students or the extent to which alumni believe that completing the comprehensive requirement prepared them for their doctoral and/or postgraduate work or relates to other criteria of interest (e.g., time to degree completion, employment or career outcomes).

Finally, we explored different types of comprehensive requirements in I-O graduate programs but did not investigate which methods are the most useful or effective for achieving the intended purpose(s). It is critical that future work examines which types of comprehensive requirements best prepare students for potential doctoral work and/or their future careers. Relatedly, more than half of the I-O program directors who participated in the study indicated that their program’s comprehensive requirement could be improved. Unfortunately, however, we do not have data to understand how they think the process could be improved. Therefore, future work should seek to identify which types of comprehensive requirements have the strongest predictive validity for outcomes of interest (e.g., graduation rates, student experience, alumni employment, career satisfaction) and how to improve I-O graduate programs’ current practices and processes.

Conclusion

We found that most I-O psychology graduate programs have comprehensive requirements (e.g., exams), but the intended purpose(s), formats, and process for administration varied widely. It is important for the field to determine whether comprehensive requirements should be distinct or consistent across I-O graduate programs. We recommend that the SIOP Education and Training Committee conduct a systematic assessment of the comprehensive requirement process with data from multiple sources (e.g., faculty, students, alumni) to offer specific recommendations to I-O graduate programs.

Notes

¹  Some participants selected more than one option for the type of graduate program. In all subsequent analyses, the denominator included all participants that selected a given response.

²  If the total percentage is greater than 100%, this reflects questions with a “select all that apply” response option and/or cases where the rounding of decimals equated to a value greater than 100. If the total percentage is less than 100%, this reflects questions that some participants chose not to answer.

³  When appropriate, authors recoded written “other” responses into the preexisting categories.

References

Brossoit, R. M., Wong, J. R., Robles-Saenz, F., Barber, L. K., Allen, T. D., & Britt, T. W. (2021). Is that ethical? The current state of industrial-organizational psychology graduate training in ethics. The Industrial-Organizational Psychologist, 58(3). https://0-www-siop-org.library.alliant.edu/Research-Publications/Items-of-Interest/ArtMID/19366/ArticleID/4888

Kostohryz, K. (2016). The doctoral comprehensive examination in counselor education: Faculty members’ perception of its purposes. Journal of Counselor Preparation and Supervision, 8(3), 6–30.

Society for Industrial and Organizational Psychology, Inc. (2016). Guidelines for Education and Training in Industrial-Organizational Psychology. Bowling Green, OH.