Creating AI-driven pre-hire assessments

June 7th, 2021

Dr. Nathan Mondragon, Chief IO Psychologist

Artificial Intelligence

Artificial intelligence (AI) is everywhere. It powers Google Maps' ability to bypass traffic jams. It lets your smartphone’s digital assistant adapt to how you navigate the world. It’s also used in HireVue Assessments — AI-based pre-hiring assessments. HireVue Assessments help companies quickly, accurately, and fairly assess candidates for open roles.

What are AI-based assessments?

HireVue AI-based Assessments are pre-hiring assessments designed by people for people. The purpose of Assessments is to give recruiters and TA teams a standard, structured, and fair way to quickly screen more candidates.

  • Assessments don’t replace recruiters. They simply help recruiters and talent acquisition teams assess more candidates more quickly and more accurately.
  • Assessments are like putting more reviewers to work for you without adding more people to your team.
  • HireVue democratizes the hiring process by minimizing bias in our algorithms.

What AI-based assessments measure

HireVue employs a team of industrial-organizational (IO) psychologists and data scientists. This team develops assessments that target the skills needed for a variety of roles. That gives TA teams assurance that candidates have the requisite competencies needed to perform the job, are likely to be higher performers, and are potentially more likely to stay with the company.

In a broad stroke, HireVue develops AI-based assessments with models that measure the competencies of potential employees':

  1. Ability to work with others
  2. Ability to do the job — both the ability to work with information as well as needed technical skills
  3. Work style and personality, for example, the willingness to work hard and persevere

Assessing for ability to work with others, ability to do the job, and work style and personality gives recruiters and TA teams a 360-degree view of a candidate.

The human side of developing AI-based pre-hire assessments

But how does AI measure these competencies? When HireVue develops algorithms that score our assessments, we work with real people. We ensure the algorithms reflect how expert human raters assess job-related competencies.

We ensure that there's a strong correlation between the human raters' assessments and the algorithm.

We don't stop with assuring the algorithm matches the assessment of human raters, though. We go on to ensure those assessments actually predict job success. We also follow the Equal Employment Opportunity Commission (EEOC) Uniform Guidelines when developing, testing, and monitoring all HireVue AI-driven assessments. The Guidelines require fair treatment of applicants in the hiring process regardless of race, gender, and age (40 and over). These protections include the 4/5ths Rule, also known as the Red-Flag Rule, and other statistical tests for group differences; all of which HireVue uses to monitor and mitigate our assessments.

If we see that one group passes the assessment at a significantly different rate compared to another group, known as adverse impact or algorithmic bias, we drill in and determine why. We then remove or minimize any data points that lead to that bias to ensure proportional outcomes for all groups and protected classes.

For instance, if we see that a disproportionate number of men score higher than women, we determine what's causing the bias and mitigate for it. We repeat the mitigation process until men and women perform similarly enough to confirm that the bias is minimized.

In addition to EEOC guidelines, at HireVue, we test for a wider array of group differences, beyond those required by law. That’s because there are biases beyond age, gender, and ethnicity, for example, attractiveness, country of residence, non-native accent, etc.

We release an algorithm for customer use, only after we’re confident that race, gender, and age group differences (or adverse impact) have been minimized. And even after it's released, we continue to monitor the algorithm for any bias creep.

The goal of AI-based assessments

The goal of AI-based assessments is to:

  • Help companies reduce time to hire and increase retention.
  • Help TA professions in selecting the best person for the needs of the job by screening more candidates in a fast, fair, consistent fashion.
  • Minimize any algorithmic or human bias from the hiring process.

For example, is the candidate who applied for a customer service role good at working with people? Or is a candidate who applied for an accounting role good at working with numbers?

Candidates aren’t assessed for a single competency. They’re assessed for multiple competencies, cognitive abilities, and personality traits, all of which are shown to contribute to success on the job. Their scores are shared with the recruiter and the candidate. That feedback can be used in hiring decisions. It can also be used post-hire as a development opportunity.

Seeing the whole candidate

HireVue offers a broad suite of assessments, including video interviewing, game-based assessments, and coding challenges. These can be combined in a modular fashion for specific roles. Using more than one type of assessment offers divergent validity, meaning each measures for different things. For example, you can combine an assessment that assesses for soft skills with one that assesses for hard skills.

HireVue's library of competencies can be mixed and matched to assess the general competencies needed for the role you’re filling.

For example, a HireVue assessment for call-center representatives will, at a minimum, assess for communication skills, ability to learn, and work ethics which are the key competencies needed to be successful in a call center role. We might also include questions to measure dependability, problem-solving, and negotiation/persuasion given the specific requirements of the call center role for a given customer (for example, a Sales-focused representative). The assessment would include 3 to 4 AI-based interview questions and 2 to 3 game-based questions for a total of 5 to 7 questions in the assessment (15 to 25 minutes completion time).

A report of results shows recruiters the candidates' overall assessment score and their specific competency scores. Also, recruiters see all candidates’ scores stack-ranked against each other. Candidates also have the ability to get a report of their assessment results.

AI-driven assessments work and can be trusted

AI actually powers a more accurate, less biased, more engaging screening process for both recruiters and candidates.

The HireVue team strives to eliminate as much bias as possible. It strives to deliver on the goal of reducing time to hire and increasing performance and retention.

About the Author

Dr. Nathan Mondragon, Chief IO Psychologist, HireVueDr. Nathan Mondragon is Chief IO Psychologist at HireVue where he builds, researches, and maintains the AI-driven HireVue Assessment product. Nathan has more than 20 years of experience in the human capital management (HCM) space and is a recognized expert at blending IO psychology tools and technology to deliver seamless, integrated recruitment, hiring, and development solutions. In 1996, Nathan helped lead the creation and delivery of the first-ever online selection assessment. In 2004, he built, from the ground up, the first integrated assessment solution within an enterprisewide applicant tracking system (ATS) (Taleo). And in 2015, as part of HireVue, he delivered the first-ever AI-driven pre-hire assessment solution. Nathan received his Ph.D. in industrial-organizational psychology from Colorado State University. Find Nathan on LinkedIn.