Do longer assessments mean higher applicant drop out?
Recent Shaker research shines a light on long-held assumptions that shorter assessments mean higher completion rates. According to the data, assessment length is not a driver of candidate drop out, so shortening your assessment may not be buying you what you think it is.
Shaker associates Carter Gibson and Alison Carr and intern Matt Sloan conducted research and co-authored the article "Are Applicants More Likely to Quit Longer Assessments? Examining the Effect of Assessment Length on Applicant Attrition Behavior” recently published by the Journal of Applied Psychology.
The research also identified what actually motivates early candidate drop out and the variables that improve completion rates.
The study used data from nearly a quarter of a million job-seekers from within 69 selection systems to challenge popular notions talent acquisition professionals have about how assessment length is related to the likelihood of candidates to leave an assessment incomplete. The study concluded that setting expectations about the length of the assessment is the biggest influencer on drop-out behavior.
“One of the first things clients say when they see opt-out numbers is: should we be worried about completion rates?” Carr says. “Attrition is in the forefront of a lot of clients’ minds."
“We know people care about this,” Gibson agrees, explaining that recruiters are often concerned about candidates who drop out, believing they are losing high-potential applicants to attrition. Clients often assume shorter assessments mean better completion rates and a stronger candidate pool.
But, as Gibson explains, candidates who drop out actually represent “good attrition."
In fact, instead of dropping out, a candidate may be said to be opting out of future consideration after determining the job isn’t a good fit for them. Gibson explains that recruiters may be tempted to focus on possible negative candidate reactions to the assessment instead of how the candidate may have recognized they were underperforming or that the role or company were not a good fit. A well-designed assessment experience invites the candidate to be a decision maker, too. It should offer candidates an opportunity to think about what they are learning about the job and decide if it’s right for them.
“Some clients worry that those who don’t finish the assessment are the potential rock stars,” Gibson says, “but the research strongly suggests that those candidates would have ended up having lower scores anyway.”
Gibson and Carr caution that shortening your assessment to a certain threshold just to retain more candidates can compromise its psychometric reliability and negatively impact your selection process.
While many clients think cutting their assessment from 35 to 25 minutes will dramatically improve their evaluation process, Gibson says, demands for short assessment can compromise the integrity of its performance measurement and stability. “Eliminating large portions of your assessment content in the interest of trying to retain more candidates will reduce the value of the assessment and have minimal impact on completion rates.”
Carr agrees. “The shorter the assessment, the less robust its measurement,” she says, explaining that some assessments are so short they can’t even really be determined to be reliable.
Shaker’s completion rate research is already convincing clients to change their assumptions about candidate attrition, Carr explains, excited that Shaker is on the forefront of a dialogue about candidate behavior and assessment design.
“A lot of what we do at Shaker is educating clients–helping them see what they should do to solve their hiring challenges and why it works,” Gibson says, adding that the research shows that creating more engaging and simulation-based assessments contributes to higher completion rates.
“We’ve done the research, so now we know.”