When fast means flawed: The case against chasing short assessments

In the high-stakes world of hiring, it’s understandable to crave a modicum of simplicity. We long for a silver bullet for the complex problems plaguing a company’s most important problem: finding the right people for the job.

But sometimes in the desire for efficiency, vendors can lose sight of the actual goal of valid measurement and promise assessments with lengths that aren’t sufficient for the task. Fixating on an arbitrary time limit may hold a talent team back from embracing an assessment that has enough time to measure the skills and qualities a role really needs. 

We've seen some great results with 4-minute assessments, and if you are just looking to measure turnover, a 3-minute assessment can be great. However, just because a 5-minute assessment can be very successful in some contexts does not mean you should make the blanket statement that a good assessment can be that length. As always, it depends. 

If you are trying to measure something beyond a narrow construct like turnover or one specific competency, or you are concerned about the candidate experience, then there are multiple downsides to defaulting to the shortest possible assessment. Let’s talk about why reducing a sophisticated selection tool down to a stopwatch reading can be risky.

Brevity doesn’t increase completion

Asking whether an assessment is under five minutes is like asking your doctor if they prescribe antibiotics without telling them what’s wrong with you. Sure, they can, but should they?

This threshold has become a noisy benchmark, mostly driven by a vocal minority of vendors who have convinced companies that brevity is always a sign of high completion rates. But there’s zero empirical data showing that a specific amount of minutes results in some magical sweet spot for engagement or completion.

In fact, our published research in the Journal of Applied Psychology shows no statistically significant relationship between assessment length and candidate drop-off rates. 

Too short can be just as bad as too long

Candidate experience matters, and speed can certainly be a part of that, but often what candidates are most concerned with is the time between application and follow-up about the status of their candidacy. Shortening an assessment often doesn’t solve the problem that talent teams are concerned about. 

And frankly, a part of assessment design that’s not named in a lot of conversations is its role as a filter for “good drop off.” Candidates taking assessments are often still gathering information about the role and company, and they may realize after a realistic job preview and a few assessment questions that the role isn’t a good fit for them–we’d label that case of drop-off a net positive. 

There are edge cases where a short assessment might make sense, such as high-volume roles with sky-high turnover, where the only thing you care about is whether someone will stick around for 30 days. But for roles requiring judgment, skill, or leadership? A five-minute test could be perceived by top candidates as insulting to their expertise and the value of the role.

We’ve seen short tests work as highly calibrated, niche interventions, not out-of-the-box solutions. To assume you can plug-and-play a five-minute test for every role, from call center to management, is just untrue.

There is no such thing as “one assessment length.”

Here’s a fun fact: most modern assessments are self-paced. That means candidates can take as little or as long as they need. So, when a vendor says “Our test is 20 minutes,” what do they really mean?

In one real-world dataset of over 30,000 completions, the median test time was 37 minutes. That means half of the candidates took longer. Ten percent took over an hour. A handful finished in under twenty minutes. Does that make it a twenty-minute assessment?

Of course not. But it does highlight how vendors can cherry-pick numbers to look more competitive. Without knowing how test length is measured and reported, you’re comparing apples to estimates.

Length as a byproduct of measurement

The length of your assessment should be based on what you are trying to measure. So, for example, if you just want to measure turnover probability for an entry-level job. Great, that's when a 5-minute assessment might make sense. We have tools that can help you do this, they are actually under 5 minutes long. However, if your goals for an assessment are broader than this (and they likely are), a shorter assessment just isn't going to get you what you want.

If your RFP asks, “Is your assessment X minutes or less?”, consider asking instead:

• What factors determine how long you make an assessment?
• How do you measure test length, and how is it reported?
• What is your test design philosophy when balancing a candidate’s time investment with predictive power?

These questions open the door for nuance and can help differentiate between a tool that measures what's needed for the job and a tool meant to hit a time target.

Complexity isn’t the enemy. Ignorance is.

Hiring is hard. Predicting human behavior is even harder. While a 5-minute assessment sounds nice, great hiring is built on insight, evidence, and asking meaningful questions to collect valuable data. 

Ready to experience assessments that are scientifically validated to predict real-world success, not just who can click fastest? Request a demo today.