From self-driving cars to cancer screenings, artificial intelligence (AI) looks to revolutionize almost every aspect of our lives.
This week, we hosted a webinar featuring HireVue’s Dr. Nathan Mondragon. He sat down with our data scientist, Lindsey Zuloaga, to discuss how AI is transforming the traditional pre-hire assessment.
What is a Pre-Hire Assessment & Why Does it Need Changing?
Dr. Mondragon began with an overview of the current pre-hire assessment landscape. Simply put, a pre-hire assessment is any test designed to identify the performance potential of candidates. These include (but are not limited to) hard skills tests, interviews, personality tests, and work sample tests.
Most pre-hire assessments are given as a series of multiple choice questions, the interview being a notable exception. Multiple choice tests have been used and refined since the 1970s, when they were distributed in “paper and pencil” assessment centers. As the internet gained widespread use in the late 90s, the assessment test followed. You can read more about the history of assessments here.
Pre-hire assessments are still used because they work. Unfortunately, for all the value pre-employment tests bring, they are not a good experience for the candidate.
Mondragon identified three key factors that make up an AI-driven assessment, starting with the candidate's experience.
A Candidate Friendly Intake
No one looks forward to a 100 or 200 question test, and this is a problem for employers looking to strike a balance between building a candidate-friendly employer brand and finding the best talent.
“Around 35% of candidates take their video interview on a mobile platform, and if you look internationally, it’s sometimes double that. If I asked a candidate to go through and take a 75 or 100 question test on their mobile phone, that mobile phone would go against the wall - they’d throw it against the wall at around question 25.” - Dr. Nathan Mondragon
Artificial intelligence allows assessments to be distributed with a candidate friendly intake: in HireVue’s case, a video interview.
AI on the Interview
Even the best recruiters and hiring managers can “phase out” during critical portions of an interview (both live or recorded), potentially missing the best parts of a candidate’s response. There are lots of subtle cues we subconsciously make sense of - think facial expressions or intonation - but these are missed when we zone out.
“Artificial intelligence, on that video data, lets us capture and evaluate all those things we go through in our brain when we’re evaluating somebody. We can capture that to look at all the words a person is saying, what their voice sounds like while giving their responses, and all the expressions and emotions in their face while they’re talking and giving their answers.” - Dr. Nathan Mondragon
A video interview provides around 25,000 usable data points from which to assess each candidate. In other words, a 15 minute video interview gains the ability to deliver a 25,000 question test.
Those 25,000 data points are turned into an assessment by linking relevant business metrics with video data.
Select from the Best
Where previously the best recorded responses would be distributed randomly throughout the applicant pool, the AI ranks them according to their score on a scale of 0-100 (indicating the percent match with a response expected from the best performers). The best candidates are consistently in the top third of scorers.
Once this data is available there are many different ways to use it. Some HireVue customers automatically trigger an action that pushes top scorers to the next step in the hiring process. Others will watch the videos of the “average” scorers to locate more talent.
An AI-driven assessment facilitates and augments a human screener’s decision making.
Existing Results & ROIs
Mondragon concluded with some current case studies, results, and analyzed the potential for bias in artificial intelligence. Generally speaking, since AI-powered assessments are bespoke, you get more good:
And less bad:
While also drastically cutting time to hire - as you can see in this example from Unilever, who decreased time to fill from 4-6 months to 2 weeks:
In the 150+ models HireVue has built, only 2 have found a significant group difference. In both cases, the discrepancy came about as the result of biased job performance data. For HireVue, this meant removing the data points that led to the bias.
"Since each video interview gives us 25,000 data points to work with, we were able to remove the 500 or so data points without compromising the assessment’s predictive validity. Of course, this also meant we uncovered existing bias in those organizations, which they could then deal with on their terms." - Dr. Nathan Mondragon
We had lots of great engagement during the webinar, and a ton of relevant questions were asked. Dr. Mondragon answered some of the most common questions, which are listed below:
- You mentioned high volume jobs like call centers. What about applying algorithms to higher level jobs?
- How does this get implemented? What does the process look like in practice? What do you need from us?
- What about biases against groups that are not necessarily legally protected? Such as accents?
- Is it legal?
- Can it be gamed?
- In Europe it is common to use assessment centers. How do these types of assessments fit with that step in the hiring process?