Spotlight on Award Winners: 2018 Wiley Award for Excellence in Survey Research
Garett N. Howardson & Liberty J. Munson
The Wiley Award recognizes innovation and effectiveness in the design and implementation of customer or employee surveys. To merit award, nominated research must demonstrate innovation and effectiveness throughout multiple phases of the research. The 2018 Wiley Award winners of Catherine (Carrie) J. Ott-Holland (Google, Inc.), William (Will) J. Shepherd (Wendy’s Corporation), and Ann Marie Ryan (Michigan State University) exemplified innovation and effectiveness throughout the project’s entire lifecycle including conceptualization (e.g., examining actual change of employee outcomes), data collection (e.g., partnering with third party data management services to avoid data identifiability and privacy issues), data analysis (e.g., multilevel structural equation model with probit links), through reporting and publishing the project’s results in Journal of Occupational Health Psychology (see below for full paper reference). In this issue of the Awards Spotlight column, we interviewed the project’s two first authors (Carrie and Will, pictured below) to learn in more detail about these innovations. Our questions and the authors’ paraphrased responses can be found below.
Carrie Ott-Holland Will Shepherd
Describe the research/work that you did that resulted in this award. What led to your idea?
It is not uncommon to find research examining employee wellness programs using only single-source, self-reported data captured at a single point in time. It seems that much less research has examined employees’ actual participation in such wellness programs. More specifically, it was surprising how little research has examined (a) employee prior beliefs’ influences on wellness program participation, (b) how such employee beliefs change over time as a function of wellness program participation, and (c) how such participation choices actually influence individual-focused (e.g. perceived organizational support, job satisfaction) and organizational-focused (e.g., turnover, employee performance) outcomes. The idea for this particular project essentially arose from this lack of prior research in conjunction with access to rich, multisource, longitudinal data capable of answering such questions, which included objective wellness data (e.g., wellness program participation), self-reported employee opinion survey data, and supervisor-rated employee performance.
What do you see as the lasting/unique contribution of this work to our discipline?
Examining the relationship between actual health outcomes and work performance is likely a lasting contribution of this work. Perhaps more important, however, was the specific way in which we were able to accomplish this by partnering with both health program and survey vendors to secure the data necessary. Health program vendors might examine the actual effects on employee health outcomes (e.g., blood pressure) but have less interest in examining relationships with work performance. Similarly, survey vendors might examine trends in stress and health and related items without actually examining relationships with objective health outcomes. One of the primary reasons such relationships are not examined is simply lack of access to the relevant data.
To examine our focal research questions, however, we had to combine data from the survey vendor, the health program vendor, and the organization’s employee performance and turnover data, which was no small feat. The health data in particular is subject to the Health Insurance Portability and Accountability (HIPPA) Act concerns and related employee privacy rights. This required creating a HIPPA-compliant data aggregation confidentiality protocol. Once created and in place, the health vendor could share the objective health program data with the survey vendor in a HIPPA-compliant manner. The survey vendor was then able to create a de-identified, merged dataset with which we could merge the employee performance and turnover data also in a HIPPA-compliant manner. Developing the data sharing protocol and establishing relationships with both the survey and health program vendors allowed us to ask and answer a research question not previously possible using same-source data collected from only the organization. Perhaps more importantly, however, the data aggregation protocol allowed us to do this in a way that protected employees’ legal and ethical rights, which is most definitely an innovation in employee survey methods.
It is also worth noting that, in addition to the data management process, the actual data analytic method used is likely a lasting contribution. The final dataset was complex in that individuals were nested within organizational units, the objective wellness program participation variable was ordinal, the turnover variable was dichotomous, and the model and hypotheses involved multivariate path analysis. Needless to say, standard ordinary least squares regression methods were not going to work. Instead, we employed multilevel path analysis with multinomial regression and probit link functions. In other words, answering our research questions involved advanced and innovative methods for data collection, data management, and data analyses.
How can this research be used to drive changes in organizations, the employee experience, etc.?
Our findings could be used to market wellness programs more effectively to employees. In fact, with the help of an external marketing firm, our findings were actually used to create new such marketing materials for employees (e.g., posters, handouts, intranet pages) to raise awareness about the wellness program and its benefits. More indirectly, the very existence of a data management protocol could send signals to employees that their privacy is respected, which may itself drive positive organizational change.
What’s a fun fact about yourself (something that people may not know)?
Carrie: I am a classically trained opera singer (I was a music major in college in addition to psychology).
Will: I love seeing live music with friends and family and have been to over 200 rock and roll concerts.
Paper Reference:
Ott-Holland, C. J., Shepherd, W. J., & Ryan, A. M. (2019). Examining wellness programs over time: Predicting participation and workplace outcomes. Journal of Occupational Health Psychology, 24(1), 163-179. doi: 10.1037/ocp0000096
About the authors:
Garett Howardson is the founder and principal work scientist at Tuple Work Science, Limited and adjunct psychology professor at both Hofstra University and at The George Washington University. Most of his work focuses on quantitative, psychometric, and/or computational issues to better understand the psychology of modern, technical work writ-large (e.g., aerospace technicians, computer programmers).
Garett is also an avid computer geek. In fact, he has a degree in computer science, which he avidly applies to his research and work in pursuit of one deceivingly simple goal: better integrate I-O psychology and the data/computational sciences to understand work.
Liberty Munson is currently the principal psychometrician of Microsoft’s Technical Certification program in the Worldwide Learning organization. She is responsible for ensuring the validity and reliability of Microsoft’s certification and professional programs. Her passion is for finding innovative solutions to business challenges that balance the science of assessment design and development with the realities of budget, time, and schedule constraints. Most recently, she has been presenting on the future of testing and how technology can change the way we assess skills.
Liberty loves to bake, hike, backpack, and camp with her husband, Scott, and miniature schnauzer puppy, Apex. If she’s not at work, you’ll find her enjoying the great outdoors or she’s in her kitchen tweaking some recipe just to see what happens.