Brainwave Helmets and Algorithmic Copilots: A Black Mirror Episode or Just Modern Performance Assessment?

By: David Tomczak George Washington University

Advancements in performance assessment and management are emerging rapidly and have drastically changed in the age of big data. For instance,

Uber drivers are now managed by algorithms that instruct them on when to work and how to maximize their time on the road (Rosenblat, 2016).

Truck drivers receive automated feedback about safety behaviors that help them make safer turns (Bowman, 2014) and even instruct them to stop driving when they become drowsy (Haubursin, 2017).

Microsoft provides personalized dashboards detailing how employees spend their time, providing suggestions for how to engage with customers more frequently and spend less time in meetings (Krouse, 2019).

Other companies, like Life Time Fitness, analyze the content and intonation of voice and audio data from conference calls to understand who dominates conversation and how people respond to emotional discussions (Krouse, 2019).

And just when you thought real-time speech analysis was invasive, a Chinese company has created helmets that monitor employee brainwaves to understand worker emotions and productivity (Chan, 2018).

Historically, monitoring employee behavior was deliberate in execution and limited by the type of data collected, and feedback was only delivered on an annual or biannual basis. Now practices are starting to resemble Black Mirror episodes where human–technology interaction can feel eerily personal. (Black Mirror is the Netflix sci-fi series that closed a recent season with an interactive episode.)

Compared to the well-established annual performance review, the modern world of performance assessment and management is data driven, dynamic, and real time. The era of real-time assessment is just beginning. Recent patents hint at a future of highly surveilled workspaces where audio sensors are placed all around Walmart cashiers (Etienne, 2018) and Amazon warehouse workers wear bracelets that emit haptic feedback (light vibrations) when workers make mistakes (Ong, 2018).

Companies are looking to gather vast amounts of personal information to better understand employee behavior, and the advances have both positive and negative consequences

The Good: Advantages for Performance Assessment

Objectivity. With traces of actual employee behavior, organizations no longer need to lean heavily on subjective performance ratings from supervisors and peers. This is beneficial because rating error has been consistently cited as a major limitation of the performance appraisal process. Now, organizations can see exactly how employees spend their time and generate metrics of employee behavior to compare to performance standards, which ultimately can be used to supplement performance ratings (Tomczak & Behrend, 2019). Employees also have the opportunity to “prove” their performance using such metrics, and research suggests some employees even prefer this type of appraisal (Bhave, 2014).

Real-time performance data. New monitoring technologies capture employee behavior as it occurs, which allows for faster performance interventions. Rather than going a year without meaningful performance feedback, monitoring devices can identify problematic behaviors and offer development recommendations on the spot. This is possible because employees leave behind “digital exhaust”, when they interact with computers and online applications. That is a trail of data that characterizes their behaviors, such as temporary files, cookies, and website visits, a vast data source. Analyzing an employee’s digital exhaust can reveal meaningful patterns regarding performance on the job, allowing for swift organizational intervention if necessary.

Contextual performance information. New performance data provide more information about the context in which employees are most (or least) productive. This is the difference between saying “Bill is an underperformer who missed his target for this month” vs. “Bill appears to be setting too many internal meetings on Thursday afternoons, which causes him to miss sales opportunities at the end of the week. Some time management and scheduling adjustments should help him get back on track.” In other words, data-driven methods can lead to more accurate, targeted performance improvement recommendations.

The Bad: Potential Unintended Consequences

Privacy invasion and fairness perceptions. The past few decades of monitoring research have shown that, in general, employees perceive electronic performance monitoring as both unfair and an invasion of privacy. These perceptions intensify when employees believe that non-work-related information is being collected. This is an easy trap for organizations to fall into because popular work tools, such as laptops, smartphones and even company cars, are equipped with applications that automatically collect personal information about employees, such as their personal conversations, Internet search habits, and whereabouts (e.g., Stanton, 2000; Zweig & Webster, 2002).

Negative employee reactions. Being monitored is associated with greater levels of stress, and in the current environment of flexible/remote work, employees can feel tethered to their workspaces, hesitant to take breaks. Some employees are downright angered by monitoring, leading them to lash out against the organization in whatever way possible, whether it is purposefully showing up late to work or even stealing (e.g., Yost et al., 2019).

Behavior change. Observer effects can change employee behavior, which can ultimately affect employee effort and performance. Monitoring is a signal to employees that basically says, “If it’s monitored, it matters,” so employees allocate more time and effort toward monitored tasks. Although this may sound like a boon to productivity, results can be disastrous if the wrong tasks are being monitored. For example, when quantity is monitored, quality tends to suffer. Moreover, the stress from being watched can cause low-skilled workers to perform even worse when being monitored (e.g., Aiello & Kolb, 1995; Becker & Marique, 2014; Stanton & Julian, 2002).

The Future: What’s Next?

Important insights have been found regarding the context and sources of error in performance assessment, but how should we view performance in the context of big data? Do organizations need new models of performance? Do existing models of performance sufficiently accommodate new data inputs.

There are no brief, easy answers to these questions, but as we explore this possibility, careful consideration of what we deem performance and data quality remain key considerations. We must ask ourselves:

• Just because we have big data, should we use it?
• Do these digital traces of employee behavior truly represent job performance?
• Can we make legally defensible assertions that this behavioral data is job related?
• Can employees game the system?
• What are the pitfalls and blind spots of objective performance data?
• What ethical and privacy concerns must we consider when monitoring performance?
• Are we putting employees’ personal information at risk by taking a more data-driven performance assessment approach?
• How do employees react to modern methods of performance monitoring and assessment?
• Who is most resistant to behavioral data collection and why?

In a time when people are growing more aware of the repercussions of sharing personal data, modern performance assessments raise just as many privacy and data quality questions as theoretical and practical questions.

Nonetheless, the advantages of big data suggest a promising future where we can further hone performance assessment and management using rich sources of behavioral data. In the meantime, we can lean upon a wealth of best practices derived from industrial-organizational psychology, such as job analysis, as a foundation. What we now must consider is how big data can inform these processes. So let’s put on our brainwave helmets and get to work.

Have questions or comments? Connect with the author, David Tomczak on LinkedIn or email comms@siop.org.

We hope to see you at the Leading Edge Consortium on October 25-26 where we will continue exploring topics related to the future of assessment. Seats are limited so please be sure to register here before the 2019 LEC is sold out!

References and Suggested Resources:

Aiello, J. R., & Kolb, K. J. (1995). Electronic performance monitoring and social context: Impact on productivity and stress. Journal of Applied Psychology, 80(3), 339–353. http://doi.org/10.1037/0021-9010.80.3.339

Becker, T. E., & Marique, G. (2014). Observer effects without demand characteristics: An inductive investigation of video monitoring and performance. Journal of Business and Psychology, 29(4), 541-553.

Bhave, D. P. (2014). The invisible eye? Electronic performance monitoring and employee job performance. Personnel Psychology, 67, 605-635.

Bowman, R. (2014, Feb 11). Is new truck-monitoring technology for safety—or spying on drivers? Forbes. Retrieved from http://www.forbes.com/sites/robertbowman/2014/02/11/is-new-truck-monitoring-technology-for-safety-or-spying-on-drivers

Chan, T. F. (2018, May 1). China is monitoring employees' brain waves and emotions —and the technology boosted one company's profits by $315 million. Business Insider. Retrieved from

https://www.businessinsider.com/china-emotional-surveillance-technology-2018-4

Etienne, S. (2018, Dec 21). Walmart secured a patent to eavesdrop on shoppers and employees. The Verge. Retrieved from https://www.theverge.com/2018/12/21/18151738/walmart-eavesdrop-patent-customer-employee-privacy

Haubursin, C. (2017). Automation is coming for truckers. But first, they’re being watched. Vox. Retrieved from https://www.vox.com/videos/2017/11/20/16670266/trucking-eld-surveillance

Krouse, S. (2019, July 19). The new ways your boss is spying on you. Wall Street Journal. Retrieved from https://www.wsj.com/articles/the-new-ways-your-boss-is-spying-on-you-11563528604

Ong, T. (2018, Feb 1). Amazon patents wristbands that track warehouse employees’ hands in real time. The Verge. Retrieved from https://www.theverge.com/2018/2/1/16958918/amazon-patents-trackable-wristband-warehouse-employees

Ravid, D., Tomczak, D. L., White, J. C., & Behrend, T. S. (in press). EPM 20/20: A review, framework, and research agenda for electronic performance monitoring. Journal of Management.

Rosenblat, A. (2016). The truth about how Uber’s app manages drivers. Harvard Business Review, 1-7.

Stanton, J. M. (2000). Reactions to employee performance monitoring: Framework, review, and research directions. Human Performance, 13(1), 85-113.

Stanton, J. M., & Julian, A. L. (2002). The impact of electronic monitoring on quality and quantity of performance. Computers in Human Behavior, 18(1), 85–101.

Tomczak, D. L. & Behrend, T. S. (2019). Electronic surveillance and privacy. In R. N. Landers (Ed.), Cambridge handbook of technology and employee behavior (pp. 708 - 742). Cambridge, UK: Cambridge University Press.

Yost, A. B., Behrend, T. S., Howardson, G., Darrow, J. B., & Jensen, J. M. (2019). Reactance to electronic surveillance: A test of antecedents and outcomes. Journal of Business and Psychology, 34(1), 71-86.

Zweig, D., & Webster, J. (2002). Where is the line between benign and invasive? An examination of psychological barriers to the acceptance of awareness monitoring systems. Journal of Organizational Behavior, 23(5), 605-633.