Areas in Need of More Science/Research: Results from the 2015 Practitioner Needs Survey

Ben Porr, Ted Axton, Meredith Ferro, and Soner Dumani

Introduction

In the July 2015 TIP, the SIOP Professional Practice Committee (PPC) presented the first of a series of articles reporting the results of the 2015 Practitioner Needs Survey that the PPC conducted between March and April 2015. The objective of the survey was to gather information about current needs of I-O practitioners to provide insights to SIOP leadership and committees (e.g., PPC, licensure, visibility) about developing future initiatives. In addition, the survey was designed to collect information that could be compared to the results of the 2008 Practitioner Needs Survey in order to examine progress on issues identified in 2008.

This article focuses on a question asked of I-O practitioners in the 2015 survey that requested their perceptions of areas of I-O psychology where additional research may be needed to support effective practice. This question was included in order to help inform both scientists and practitioners about possible priority areas where planning and conducting additional research may be beneficial. It is important to note that the nature of this question for the 2015 survey differed from that asked in the 2008 survey, which focused more broadly on participants’ perceptions about gaps between science and practice areas.  As such, although comparisons of results between the two surveys will be discussed in this article, the changed nature of the question may impact results of these comparisons.

Survey Respondents

A total of 469 valid responses were obtained from the 2015 survey, which reflects a response rate of 10% across the SIOP membership (the 2008 survey received 1,005 responses; which was a response rate of 36%). Detailed information on the characteristics of the respondent population is provided in the July 2015 TIP article. In order to compare the 2015 results with the 2008 results, we grouped respondents using the same “practitioner categories” used in analyzing and reporting the 2008 data. Each respondent was grouped into a practitioner category based on the amount of time the respondent indicated he or she devotes to being an internal or external practitioner (as opposed to an educator, scientist/researcher, or other):

• Full-time practitioners devote 70% or more of their time to practice
• Part-time practitioners devote 21-69% of their time to practice
• Occasional practitioners devote 1-20% of their time to practice
• Nonpractitioners do not devote any time (0%) to practice

Similar to 2008, most of the 2015 survey respondents were designated as full-time practitioners (see Table 1).

Table 1

Topics for More Science/Research

 In 2008, Cober, Silzer, and Erickson reported results when respondents were asked: In which areas do you find the biggest gap between the available science/research on a topic and actual organizational practice in your work? Respondents evaluated the gap between science and practice in 26 content areas identified during the survey development process to reflect both research and applied interest areas in our field. Respondents were asked to indicate whether they felt that a gap existed in the area by identifying whether (a) practice was ahead of science/research, (b) science was ahead of practice, or (c) little or no gap exists.

Results of the question from 2008 are presented in Table 2 below. Areas are listed in descending order, from the greatest percent of respondents indicating practice was ahead of science/research to the least. Respondents indicated that practice was ahead of science/research in the vast majority of areas (19 of 26). For the most part, “practice ahead” responses were provided for areas that tended to be:

• Hands on practice areas such as consulting and coaching
• On the organization side of I-O psychology, such as strategic planning and organizational development
• Core areas of human resource practice such as succession/workforce planning, talent management, employment branding, HR technology, labor and employee relations, and employee recruitment

Table 2

The 2015 survey included a modified version of the question that focused mainly on identifying where practice was ahead of science/research, in order to help provide scientists with potential broad areas of focus for future research. To identify the topics practitioners felt more research was needed, respondents were asked: In which topic areas do you find more science/research is needed to support effective organizational practice in your work? As a follow-up, respondents were also asked to identify their top three choices. Thus, the focus changed from whether practice was ahead of science/research to a more specific question of where the greatest needs were for more science/research. The topic areas to choose from remained the same on both the 2008 and 2015 surveys. 

Table 3 presents the percentage of respondents that endorsed the need for more science/research in each area, as well as the top three priorities rated by each respondent. As anticipated, the topics chosen and top three choices were very similar. Succession/workforce planning, talent management, and management/executive selection rose to 1, 2, and 3 from 6, 7, and 16, respectively, whereas consulting and advising, employment branding, and HR technology dropped from 1, 2, and 3 to 12, 19, and 10, respectively. It’s interesting to see that 2015’s top areas lend themselves to traditional I-O research areas (e.g., assessment, training and development) compared to the top 2008 areas, which are more fringe I-O research areas.

Table 3

Unfortunately, there is no clear answer to the reasons behind these changes, but we can assume it’s partly due to the following reasons: progress in practice/research, sample size, and/or change in question focus. First, practice has advanced in these strategic workforce and leadership areas over the past 7 years, and organizations have seen the value of a systematic approach to these human capital practices. Second, it may be due to the sample, which is half the size of the last survey. Last, these results could be due to the change in question. In the previous administration, the people were asked to think about how the two relationship between practice and research (e.g., is research ahead of practice or is practice ahead of research), which potentially put people in a different frame of mind than simply asking where more research is needed.

It’s also interesting to see the lowest percentage groups are either I-O areas that have an extensive research history (e.g., job and work analysis, groups/teams) or transactional HR processes (e.g., labor/employee relations, compensation, recruitment and staffing).

As shown in Figure 1, which presents the results for the areas identified as needing science/research broken out by practitioner group, there were no differences in the rank ordering of selections.  Outside of the top two choices (i.e., succession/workforce planning and talent management), part-time practitioners had a higher percentage of selections from every category. This might be due to the sample size, but as we saw in the last TIP article on use of SIOP resources, part-time practitioners tend to read more research than full-time practitioners and therefore might notice the gaps more frequently. There is a noticeable gap between research needs of full/part-time practitioners and occasional and nonpractitioners across all categories.

For Table 4, we continued investigating the differences between our four groups and rank ordered the percentage of people that identified an area (e.g., succession/workforce planning, talent management) as a top three science/research need. To conserve space, we only include the top 10 most selected (i.e., highest percentage selected) areas for each group. Succession/workforce planning, talent management, performance management and executive/management coaching were present in all three practitioner categories. Interestingly, litigation support was in the top 10 only for part-time practitioners. Three areas were in the top 10 for only full-time practitioners: leadership and management development; HR technology; and employee engagement, attitudes, and motivation. Finally, organizational development and competency modeling were in the top 10 for everyone except the full-time practitioners.

Conclusions and Recommendations

Based on the 2015 survey results related to practitioner development needs, practitioners indicated the key areas they would like to see more research performed in order to support effective practice. The results suggest a strong interest in more science/research in the following areas:

• Succession/workforce planning
• Talent management
• Management/executive selection
• Performance management
• Leadership and management development

There are many possible reasons that practice is ahead of research in these areas. First, most of these top priorities involve the convergence of multiple research areas (e.g., succession/workforce planning is a combination of employee development, future state assessment, competency gap analysis, and strategic planning). As we see in the data, each one of these areas needs additional research (e.g., training and development, competency modeling, strategic planning), so developing research that integrates these areas to inform their interaction multiplies the difficulty of advancing the field. The SIOP webinar on succession planning may offer some insights http://0-www-siop-org.library.alliant.edu/webinar.aspx. It includes areas of research that has informed succession planning.

Figure 1. Percentage of Respondents that Selected the Area as in Need of Science/Research

Table 4

Secondly, these topic areas often can’t be controlled in real-world settings, so it makes it difficult to test various approaches with the same sample. Some obstacles to measuring this would be criteria for making a successful transition, timeframe for how long it takes to determine success, environmental factors that could impact the transition, and so on. Most of the research that is done is post hoc, but future research should be done longitudinally to identify the drivers that lead to a successful transition.

Last, as is always the case, researchers cannot investigate these areas without partnerships with applied practitioners who have access to the data. Unfortunately, advancing the field is secondary to delivering client results. Most of the time, practitioners cannot even get access to the data. This is exacerbated when dealing with more sensitive data (e.g., effectiveness of leaders that drive the organization), which as we see are most of the top research needs (e.g., succession planning, manager/executive selection, leadership development).

Next Steps

Moving forward, we have multiple next steps. Immediately in response to this article, practitioners have told us the areas where we can provide additional research. This can be accomplished through basic research or simply compiling research that has already been done on the topic. For example, SIOP’s white paper series, which includes practical articles on research areas, could be a great place to start. As we showed in the last article, sometime practitioners do not even realize the resources that are available to them, so it is also important to communicate these resources (e.g., webinars, white papers) through multiple channels.

For the overall project, our next steps include finalizing the technical report on the 2015 survey results for the SIOP Executive Committee and writing one more TIP article to share summaries of the results more broadly with the SIOP membership. Our next and final article will provide survey results related to licensing issues. We welcome any feedback or questions you may have about the survey results, and we look forward to working with SIOP members and leaders as the PPC shapes its future agenda.

Reference

Cober, R., Silzer, R., & Erickson, A. (2009, July). Practice perspectives: Science-practice gaps in industrial-organizational psychology: Part I: Member data and perspectives. The Industrial-Organizational Psychologist (TIP), 47(1), 97–105.