Shaker Consulting Group Logo Virtual Job Tryout Logo

Posts Tagged ‘Quality of Hire’

October 14, 2014

High Fidelity Simulations for Employee Selection: Are They Right for My Company?

When designing a Virtual Job Tryout, a question our clients often ask is, “How do we decide what type of content to include?” Traditionally, online pre-employment assessment has been limited to what amounted to an electronic version of a paper and pencil test. Candidates would answer multiple choice questions on measures such as personality and work history. Online testing offered convenience, but little else beyond traditional assessment.

However, with growing technology capabilities, SHAKER has been introducing increasingly realistic and interactive job simulations to our Virtual Job Tryouts for over a decade. These simulations are customized based on the nature of the job and run the gamut from entry-level roles to senior management roles. Some recent examples include simulations of multitasking for call center roles, pallet building for warehouse distribution roles, and running a business for store manager roles. In each of these simulations, candidates complete tasks very similar to those they would do on the job within a virtual environment.

While an interesting and exciting option, what are the benefits of using a simulation beyond traditional assessment measures? SHAKER evaluates these from three perspectives:

Candidate Experience
Candidate reactions are an important measuring stick of the effectiveness of any selection assessment. Beyond an opportunity to gather information from candidates, assessments offer a chance to acknowledge the two-way decision-making process inherent in hiring. Candidate experience reactions to our game-like simulations have been positive, with candidates almost always reporting that they finish the experience feeling that they better understand the job and what it entails.

Candidates also comment that they find simulations fun and engaging, especially compared to less interactive hiring processes they’ve experienced. Candidates can finish the experience with a more positive impression of the company than when they began. Taking the time to develop a simulation not only conveys to candidates that a company cares about its candidates and finding the right fit, but it also helps the company appear more progressive in the eyes of the candidates, enhancing brand image.

Clients Improve Quality of Hire
Companies also appreciate that they can see a candidate “in action” and have a window into how the candidate might perform without hiring him or her first. Creating a virtual work environment is an efficient and cost-effective alternative to in-person approaches such as assessment centers. Simulations can also be created for a variety of environments as long as the environment can be approximated in an online experience. Brief online simulations, such as those used in the Virtual Job Tryout, work best when the job task being simulated is not exceedingly complex, as this would require lengthy instructions and training beforehand or significantly disadvantaging external candidates.  A brief online simulation may be less appropriate for a highly complex job such as an astronaut, but can quickly and accurately tap important components of performance in a wide range of jobs.

By trying out the job first, candidates also gain a better understanding of the job and their fit with the requirements of the job. Prior research has shown that gaining a realistic understanding of the job before starting can significantly reduce turnover, leading to cost and time savings for the company. New employees are also more likely to view challenges encountered in the job as fair if they were informed of those challenges before accepting the job. Thus, simulations can help companies improve quality of hire and create a more effective and engaged work force.

As Industrial Organizational Psychologists, we also want to know how simulations contribute to our ability to predict job performance. Does adding a simulation improve our ability to select the right candidates beyond just using traditional measures such as personality and work history? Adding a simulation often adds time to the length of an assessment. Given that companies are facing increasing pressure to reduce the length of their application process, is it worth it?

For each Virtual Job Tryout, we analyze what each measurement experience is adding in terms of predicting on-the-job performance. Specifically, for simulations we analyze how much better we are able to predict the competency measured by the simulation than if we only assessed that competency with traditional measures. The chart below shows examples of four different simulations for which we were able to predict performance on the relevant competency significantly better (i.e., we had a statistically significant increase in the correlation between our test and job performance) when adding a simulation.

This effect is strongest for simulations that measure abilities such as business acumen and multitasking that require more than just experience to develop. For these competencies, just knowing that a candidate has experience does not give the full picture of the quality of their work as two people with the same amount of experience may differ substantially in their ability. These jumps in predictive power add up to a substantial return on investment (ROI) impact by enabling companies to select stronger performers.

A well designed and executed simulation can add a lot of value to the hiring process, for both candidates and the company. Simulations are not a replacement for traditional measurement types, but can enhance prediction of on-the-job performance. They offer a unique glimpse into performance that cannot be captured through low fidelity measurement alone. SHAKER uses simulations and traditional content simultaneously to create an experience that is both engaging and thorough, and both informative and predictive

August 8, 2014

Hsien-Yao Swee, Ph.D., Appointed to SHAKER’s Insights Team

To keep up with continued demand for its innovative Virtual Job Tryout®, SHAKER today announced that Hsien-Yao Swee, Ph.D., joined the firm’s Insights Team.

Hsien-Yao Swee, Ph.D Joins SHAKER Insights Team

Using sophisticated methods to analyze performance metrics, turnover data and hiring data, SHAKER’s rapidly growing Insights Team is focused on using business intelligence and selection science from the data to enhance companies’ existing SHAKER simulations and better predict the likelihood of employee success on the job. Swee’s experience teaching college-level psychology courses will serve him well, as the Insights Team also is responsible for providing ongoing training and education to SHAKER clients to ensure they are getting the most out of their evidence-based hiring system investments.

In addition to teaching skills, Swee brings talent analytics, performance evaluation and business intelligence expertise to his new role with SHAKER. Prior to joining the company, he served as the manager of Organizational Research, Analysis and Planning at Eaton Corporation, a global power management company based in Cleveland. Swee also was a consultant at Atrain, an HR consulting company based in Germany, where he designed assessments to determine employee and candidate competence in a variety of skill sets.

“Yao brings invaluable experience from his past role as an internal analyst. He understands the challenges and opportunities of synthesizing big data analysis into insightful stories that convey business impact,” said SHAKER executive vice president, Joseph Murphy.

“I’m passionate about applying advanced analytics to maximize our clients’ ROI,” commented Swee. “This new opportunity at SHAKER is exciting, and I’m looking forward to helping our clients hire the best candidates.”

Swee earned a doctorate in industrial/organizational psychology from the University of Akron in Ohio. In addition, he holds both a Bachelor of Arts degree in psychology and sociology and a Master of Arts degree in applied psychology from the National University of Singapore.

April 15, 2014

Your Next Great Employee Is Calling – Improving Quality of Hire for Your Call Center

Last year, Deloitte published the results of a survey of global contact centers, which identified several noteworthy trends, including:

  1. Contact centers continue to grow in size and strategic importance.
  2. Customer experience is very much a competitive differentiator.
  3. Contact centers suffer from very high annual turnover rates among their representatives.
  4. Contact center staffing is more important than ever and many contact centers face the challenge of hiring people who just aren’t a good fit.

Call centers are providing differentiated customer service as a means to increase loyalty and satisfaction. They are seeking to ensure that their staff emulates the service delivery models of their clients – demonstrating the same kind of customer focus, product knowledge, and service delivery etiquette as a full-time employee of that client.

This enhanced customer service effort ultimately comes down to people. Hire the wrong call center agent and suffer the consequences of low customer satisfaction, customer loyalty, productivity (including sales), team morale, profitability and brand delivery, to name a few.

Does your evaluation tool fit your performance need?

Finding the Right People – The Resume Fallacy

So, how do you ensure you have methods to manage quality of hire for your call center?

Unfortunately, when it comes to talent acquisition, there is no magic bullet.  Not all call centers are created to deliver the same experience.  As such, qualities that make for success in one call center are often not the secret sauce in another.  A call center agent’s long-term success depends more on job-fit factors than on past experiences and product knowledge. Some key characteristics of a successful call center candidate are their ability to:

  • Respond to callers in a service oriented, brand positive way
  • Convey confidence and rapport while providing proof understanding
  • Split their attention between two tasks simultaneously
  • Quickly and accurately enter data
  • Navigate complex and multiple data sources for information and answers
  • Understand and draw conclusions from information pertaining to the caller
  • Accurately and quickly document call outcomes and actions needed

Often recruiters and hiring managers hone in on resumes as a means to ascertain the best fit for their environment. This is a logical, but limited approach. Research on the usefulness of resumes for identifying future successful employees shows that resumes are at best marginally useful to ascertain job fit. They add almost no predictive value and are not consistent from one applicant to the next.

A Better Hiring Approach

Work-sample, or simulation based assessment of candidates’ capabilities can help you hire better call center agents who will stay longer. Work samples provide objective and consistent evidence to support hiring decisions that improve quality of hire.

Simulation-based pre-employment assessments systematically evaluate candidates’ capabilities and help you make smarter, more informed hiring decisions that drive business results. These multi-method assessments are used to measure a variety of attributes ranging from personality traits, motivations, specific skills and abilities like multi-taking, computer navigation, problem solving, and overall job fit. Simulations often provide candidates with realistic job previews of the call center work environment and a sampling of agent job tasks. By rolling up their sleeves and trying out aspects of the role during the screening process, albeit virtually, you can ensure that the candidate experience delivers a better understanding of the reality of the job.

Here are a few quotes from call center candidates after completing a simulation.

“Very user friendly, easy to understand and I would highly recommend it to any of my close friends looking for work.”

“What an amazing application! I’ve never experienced a Virtual Job Tryout, and what an amazing tryout it was!”

“Really well presented and easy to understand. A good introduction to the CSR position, as it gives examples of what the CSR may encounter when real people are calling the office.”

“Loved the Tryout. It really showed how fast paced things are in a CSR environment.”

“It’s a great way to see if you are really interested in this job. Very creative!”

“It was well thought out and provided me with a realistic model for the job expected of me.”

Effectively identifying high-potential candidates requires a more evidence-based hiring approach than resume reviews. Words on a page do not convey a meaningful story of job-fit. Companies that go beyond resumes and experience to assess candidates’ competencies achieve double-digit percentage improvement in sales, productivity, retention, and other key call center metrics.

So what are some of the best practices for improving quality of hire for your call center? And, what type of ROI can you expect from pre-employment assessments? Stay tuned for my next post, where I’ll delve deeper into both of those topics and more.

April 10, 2014

Call Center Staffing – Best Practices and ROI

When you operate a call center, you want to know which candidates can perform the job, will stay in the job long enough to be productive, will provide a return on the initial recruiting and training investments, and will have the greatest impact on key call center metrics.

To ensure you’re using the right approach to assessing candidates for your call center, I’ve identified a few best practices to consider for improving quality of hire:

Multi-Method Measurement: A combination of assessment types during the hiring process provides a more comprehensive assessment of candidates, compared to using a single test.

Realistic Job Previews: Present candidates with a balanced and candid over view of the company, the call center environment, the work hours, management style, and performance expectations so they can make an informed decision.

Data-Driven Decisions: In the case of high-population jobs, such as call centers, in particular, using data as the foundation for creating a business case and documenting return on investment for your staffing processes is critical.

Use of Job Tryouts or Simulations: The best way to identify whether an individual has what it takes to thrive in a call center environment is to allow them to attempt tasks that mirror the complexity of the work. This gives insight into both their behaviors (how they do it) and outcomes (what they achieve).

Job Tryouts – A Closer Look

A true job tryout or work sample – where a person is hired and then demonstrates whether they can handle the job demands – can be very effective, but it is not a very practical approach to staffing. A better option is to obtain a work sample from the candidate prior to extending an offer of employment. This is the premise of the Virtual Job Tryout® or customized simulation. This type of whole-person, day-in-the-life assessment is like test driving a car before buying it.

For applicants, a Virtual Job Tryout is a fun and engaging experience that provides a realistic preview of the job and the company. At the same time, you get to learn about the candidates’ capabilities, not by searching for key words or experiences in a resume or application, but by immersing the candidate in the role and actually having them perform key duties. What better way to ascertain candidates’ capabilities than to have them perform activities that mimic key job responsibilities?

Return On Investment

When it comes to investing in a staffing process such as job simulations, there are three avenues where you can expect a return:

  1. Administrative efficiency: The use of technology delivers efficiency improvements with regard to managing candidate flow. Cost-per-hire, time-to-fill, and interview-to-hire ratio are a few metrics that can be tracked to demonstrate administrative efficiency.
  2. Reducing waste, rework and defects: New hires that are separated within 90 days of starting can be viewed as staffing process defects or waste. The total investment in sourcing, recruiting, on-boarding, training and wages paid can be calculated to determine the cost of a staffing defect. Rework is then required to replace or repair defects and waste. Replacement hires must back-fill the new vacancies, doubling the cost and time to achieve the proficient performer stage. Significant savings can result from reducing staffing waste.
  3. Reducing performance variation: When it comes to staffing, reducing hiring decisions that allow poor performers into the organization can have significant economic impact. It is important to reduce low end performance variation by screening out anyone who would emerge as an under performer. This can lift the average level of performance in a work group. And hiring above average candidates is important.  Quantifying the impact of performance variation with talent analytics reveals vital business insights.

Improving Quality of Hire

In high population jobs such as call center agents, candidate evaluation methods can be used to objectively and cost-effectively differentiate performance potential. In our consulting work, we have documented the difference between candidates’ performance on simulation-based assessments (i.e., work samples) and on-the-job performance. Agents who score higher on the assessment perform at higher levels when it comes to key indicators of successful call center agents.

Quality of hire is documented by on-the-job performance. The chart below documents the Average Handle Time for two groups of hires: Those with Low Job-Fit Scores (bottom 30% of candidates) and those with High Job-Fit Scores (top 70%of candidates).  High scoring individuals entered the job producing more efficient work, at a more consistent level.  The low scoring individuals were immediately less effective and it took a year of experience (plus additional coaching) to approach the call efficiency levels of other new hires.  The impact of that performance gap can never be closed.

Through proper talent analytics, you can be armed with measurable assessment results and robust data to compare and contrast candidates, your recruiters and hiring managers can objectively and reliably make choices about which call center candidates should be considered for an interview. With this information in hand, you can invest your time with better candidates, reduce the interview-to-hire ratio, and increase the quality of hire.

Interested in discussing further? Come visit us at ERE in San Diego (Booth 820).

March 28, 2014

Kudos for Kevin – Wheeler Sees Trends, Writes Great ERE Post

Kevin Wheeler wrote a great trend piece for ERE.

Kevin, thanks for finding examples due to your ability to see the positive movement of emerging practices, pushed forward by leaders seeking continuous improvement.  You offer all an opportunity to step along a continuum.

Kevin Wheeler See Talent Trends

Engagement/Experience 2.0 or 3… Yes experience matters, and many organizations are actively building practices which enhance skills, superior outcomes. See the growth and participation in the Candidate Experience Awards as additional one element of evidence.

Social Media… an explosion of tools, simplification of formerly sophisticated tools, and the power of a personal message via multi-media.  People are connecting and sharing at an unprecedented magnitude.  Talent acquisition can watch, or jump in and be present, be visible and connect.  Heck, even I have been making short videos for years.

Talent Analytics –big data begins at the local level.  The Bureau of Labor Stats list Industrial/Organizational Psychology (I/O) as the fastest growing occupation into the near future.  This profession leads the way in developing analytical practices that link talent to organizational outcomes.  I have been hiring I/Os for over 20 years.  Being able to push the advancement and acceptance of their skills has been rewarding. We have done amazing work on raising quality of hire for clients around the globe.  Evidence-based hiring has been around for about a century now.

Org Structure – nobody has it right, but those willing to dip their toe in some Bauhaus philosophy may see how form can follow function.  30 year ago I read an article that stated the winners will be those organizations that can un-learn, invent and re-learn.  The research on personal development indicates unfamiliar and demanding experiences cause the most effective improvements in capabilities.  Broad roles create that.  The new workforce expects that, and will seek out career opportunities where that is the norm.

I will be at ERE.  Any reader is welcome to swing by our booth to continue this conversation.  Ask to see our metaphor for evidence-based hiring.

March 6, 2014

Transformative Metrics – Quality of Hire

It is not enough to measure recruiting outcomes, it is an expectation of business.

Your measurements must support the on going transformation of your contribution and alignment of recruiting to strategic objectives. Evidence of quality of hire with talent analytics is a mandate for talent acquisition teams.

“Quality begins with the intent, which is fixed by management.” W. Edward Demming

Here are a few examples of recruiting functions that did the work to document their candidate evaluation methods contributed to operational outcomes.

Retail Banking – The Cash Drawer Dilemma
The organization wanted to deliver a more engaging and consultative client experience supported with technology.
A candidate evaluation experience was designed that mimicked the client facing technology and presented a range of simulated interactions.

In collecting evidence on performance factors, it was documented that those front-line employees who delivered the most engaging client experience were more prone to cash drawer errors. Those more likely to manage cash drawer accuracy delivered a less engaging client experience. And, there was a policy in place to terminate an individual after a certain number of cash errors. Thus, the organization was firing its employee with the most effective client engagement capabilities.

The use of a multi-method assessment – the Virtual Job Tryout – allowed this organization to balance the results of the candidate evaluation. The scoring algorithm places importance on both desired traits, thus advancing candidates with the capability to deliver exceptional service and manage the details of a transaction.

As a result, the talent acquisition team was able to use transformative metrics and align hiring decisions with operational outcomes. New front line associates delivered greater client service and early turnover was reduced.

Capital Equipment Sales – The One Trick Pony
The organization with one flagship product wanted to increase market penetration with its full product line. The existing sales team relied heavily on sales from one product.

A candidate evaluation experience was designed that mirrored a range of territory management and client engagement scenarios.

In collecting evidence on performance factors, it was documented that those account executives achieving broader market penetration took a more consultative and holistic approach to their client’s business. As a result, they discovered more needs and opportunities to recommend additional products and services.

The use of a multi-method assessment – the Virtual Job Tryout – allowed this organization to identify candidates with a consultative approach to client engagement and holistic sales approach. The result was hiring new account executives that delivered a 10% higher level of sales from other products and services. The talent acquisition team was able to use transformative metrics and align hiring decisions with operational outcomes.

Hiring decisions are only as good as the data that supports them. Creating multi-method evaluation experiences that mimic the complexity of the job can provide an evidence-based approach to differentiating among your candidates. And the data you collect from robust candidate evaluation can deliver transformative metrics, documenting how your recruiting process contributes to the company’s strategic objectives.

Get a free copy of our e-book on evidence-based hiring. Fill out the Contact Me form and write Book in the comment section. Learn how to use talent analytics to document quality of hire.

December 17, 2013

Gerry Crispin on the 2013 Candidate Experience Award Survey

I first interviewed Gerry Crispin about the candidate experience at ERE in 2010. In that video he asserted the candidate experience should be defined, and then measured.  The 2013 Candidate Experience Award Survey has taken great strides in that direction.  Click PLAY below to hear what Gerry has to say about what we learned from conducting this year’s survey, then scroll down to read more.

I had the privilege to lead a creative group of seasoned recruiting and talent management professionals in the refinement of the Candidate Experience Award Survey for 2013.  Our team included:

Andrew Gadomski of AspenAdvisors

Ben Gotkin of Recruiting Toolbox

Bryan Wempen of DrivethruHR

We reflected on the 2011 and 2012 experience and looked to the future.  Thanks to the committed collaboration of the Survey Team, a strong, focused, candidate experience evaluation tool was created.

With expanded awareness and stronger market presence, the Talent Board was able to attract a larger number of participating organizations and a significantly larger number of candidates.  Survey data was abundant, as was the opportunity to consider, compare, learn, and discern.  The Content Team is working hard and furiously to summarize key observations and outcomes from the survey.  Register here to obtain a copy when it is released.

The Candidate Experience Survey, at its core, is about bringing evidence-based practices to staffing process improvement.  The intention is to help organizations raise their quality of hire and improve the quality of their hiring process.

As an example, one thing we now know is that a poor candidate experience can drive away a customer.  A meaningful percentage of candidates clearly stated that their experience turned them away from the brand and that they would take their money elsewhere.  That’s unfortunate, but a powerful wake-up call to examine the candidate experience.

We are compiling and organizing the final report.  But I wanted to share a few tidbits along the way.

Applicant to Hire Ratio

Half of the surveyed companies report applicant to hire ratios of 200 :1  for entry-level hourly and professional jobs.  And, about 40% of companies also report a similar level of applicants for experienced professional, mid-management and senior management jobs.   The magnitude and professional/ethical demands of making decisions to disqualify 199 candidates to make one hiring decision are daunting.  This reinforces that recruiting is the business of rejecting candidates. And the process by which that rejection occurs can have dramatic impact on a company’s brand image, consumer perception and applicant behaviors.  And a great deal of hope and high expectations rest on the job-fit and performance impact of the one individual who gets hired.

Applicant to Hire Ratio of 200+

Stay tuned.  Come back here to see more details. And register to receive the final report.

November 19, 2013

Michael Reeves, Ph.D. Joins Quality of Hire Team at Shaker Consulting Group

CLEVELAND – November 19, 2013 — Michael Reeves, Ph.D. brings industry and big data analytical experience to help Shaker Consulting Group improve their client’s hiring processes.

The Shaker Consulting Group has bolstered its ability to help clients hire exceptional employees with the recent addition of Michael Reeves.

Reeves’ blend of academic achievement and real-life experience in the assessment field make him an ideal fit for Shaker, a Cleveland-based firm that is widely recognized as market leader in custom simulations for pre-employment testing.  Shaker’s core product offering, Virtual Job Tryout, guides candidates through a series of customized exercises that mirror key job demands. The approach provides candidates with a deeper understanding of the role, while enhancing the employer’s ability make evidence-based hiring decisions derived from work-sample results that predict likelihood of success on the job.

Michael Reeves, Ph.D. Joins Quality of Hire Team at Shaker Consulting Group

Reeves, a San Diego native, recently earned his doctorate in Industrial/Organizational Psychology from the University of Central Florida, where much of his research focused on assuring that online job assessments are administered in a fair and unbiased manner.  Previously, Reeves worked for Behavioral Neuroscience Systems, where he created and delivered assessments through a range of smart technologies.

At Shaker, Reeves will focus in a large part on providing ongoing quality of hire analysis to support the most effective use of the Virtual Job Tryout  as clients change and grow in a dynamic economy.

“We are committed to growing our client management function, which is a real differentiator for us,” says Shaker president Brian Stern, Ph.D.  “We are able to leverage talent analytics and big data insights we derive from client hiring systems on an ongoing basis. This creates smarter predictive modeling to ensure they hire the best people for their organization.”

The opportunity to leverage the full range of his experience, research, and skills with clients – many of which are market leaders in their respective industries — led Reeves to choose Shaker over other potential employment opportunities.

“I’m really impressed with their strong client focus at Shaker,” Reeves says. “From day one, they emphasized they use a big data approach to solve complex hiring problems. They demand creativity and innovation at every level of the organization to satisfy customer needs.  With a firm like Shaker, I have the opportunity to impact the hiring outcomes for an incredible client portfolio and contribute to the growth and evolution of the company.”

Shaker executive vice president Joseph Murphy says “Reeves has the ability to analyze big data in ways that will help clients maintain a strong quality of hire process as their business needs evolve.”

“Our business has grown to the point that we are dedicating more resources to ongoing talent analytics for existing client relationships,” Murphy says. “Mike’s experience working in the field combined with his analytical capabilities will bring real value to our clients and assure they continue to get a solid return on investment when it comes to staffing their organization with the best candidates.”


Joseph P. Murphy

About Shaker Consulting Group, Inc.
Shaker Consulting Group is the market leader in developing custom simulations for pre-employment testing. Their proprietary and award winning Virtual Job Tryout improves quality of hire. It presents candidates with a company branded, day-in-the-life experience that educates and engages the individual, while obtaining a work sample that predicts job-fit and competencies critical for success. Virtual Job Tryout has been deployed across most industries for positions from entry-level to mid-management. Clients achieve a documented return on investment from evidence-based staffing process improvement.

November 7, 2013

Data Miners Guide To Quality of Hire – Crowd Sourcing Content

Debbie McGrath at has been intrigued with our work in helping organization improve their quality of hire. Debbie invited me to submit a presentation for the Talent Acquisition Conference – Quality of Hire Track. My submission is Data Miners Guide to Quality of Hire.  I am asking for your support for this content to become part of the conference.   Please read on.

I recently presented a version of this session for the Cleveland Staffing Management Association. Here is what Frank Zupan, SMA Chair had to say about the session.

Joe Murphy absolutely nailed it with his recent presentation “Miners Guide to Quality of Hire in the Era of Big Data”. He drilled in to all layers of HR and recruiting data, leading our audience into the “Big Dig” of current and future requirements of critical organizational information. This track was extremely well received by the Cleveland SHRM SMA audience, and we’ve had great ongoing discussions prompted by Joe’s mining expedition.

Data Mining In Your Backyard

Getting the presentation on the slate is up to you. I need your vote. The conference content is being determined by crowd sourcing. It is an interesting approach. Each person voting on content has to make a decision. It is said that decision quality is directly related to the data that supports that decision. So, I have provided a review and some details of my presentation so you can make an informed decision. The link to the voting site is at the end of this post.

Reference Point
The word quality implies some standard, a reference point to move toward or exceed. As such, quality of hire is concerned with defining the reference point and measuring the relationship to that point with an outcome. In the case of a staffing process, the reference point is performance requirements or quality standards of the job. The outcome is the performance of the individual. Therefore, quality of hire is determined by the relationship of performance to standards.  This impacts the time horizon over which data is collected.  When can you  measure performance? The critical element to consider is time to proficiency.

Measurement Categories
Kirkpatrick’s Evaluation Model is widely known for distinguishing the methods and relative value of various forms of measuring learning outcomes. Recently Josh Bersin took that same framework and applied it to talent analytics maturity. My session explores recruiting metrics for all four levels of data gathering and talent analytics suggested by these two models.

At the core are the Three Os of Quality of Hire:

  1. Opinions – data commodities
  2. Observed Behaviors – data rare minerals
  3. Objective Metrics – data gems

In addition, the session places the recruiting data into a leading-lagging indicator framework. This demonstrates that quality of hire is the lagging indicator of a measurement and analysis process.  How many sources of data are in your quality of hire metrics?  There may be data in your backyard ready for mining.

Big Data
This catchy buzz word overshadows the rigorous work that selection scientists have been doing for about 100 years. Quality of hire can be documented and improved over time with evidence-supported hiring decisions. The Data Miners Guide to Quality of Hire will help you audit your current practices and invite you to add more rigor to your staffing process improvement.

The session examines three stages of evolution in the quality of hire journey:

  1. Prospector
  2. Developer
  3. Operator

And three difference specialist roles/skill-sets  that are required along the way are defined.

Thanks for your consideration and the time you took to read this.
I would appreciate your vote of support. Follow the link below and find my session in the Quality of Hire track.

Talent Acquisition Conference Content Voting

Thank you.

I look forward to seeing you in San Francisco in January.

November 2, 2013

Linda Brenner on Improving Quality of Hire

Linda Brenner, founder of Designs on Talent and Skillsify and I had a chance to talk about staffing process improvement at HRTech13 in Las Vegas.  She offered a few thoughts on what can be done to improve quality of hire.  Click PLAY to hear what she has to say.  Then scroll down to read my reflections and additions to her suggestions.

Linda offers excellent advice for staffing process improvement.  Her key points are:

  • Define the performance criteria or demands of the job.
  • Isolate and describe ‘home run’ or exceptional degrees of performance
  • Link candidate evaluation methods to the job criteria

Define Performance Criteria

Job analysis is a structured approach to observing, capturing and defining the performance requirements of a job.  It is an evidence-based approach to documenting job content through examination of what knowledge, skills, abilities and other characteristics (KSAO) are required in a job.

A common error in selection process design is looking first at people in the job versus looking at the job.  There is a difference between defining the job content and looking at people doing the job.   It is important to have exceptional performers describe what they do, but it is important to have a job-first methodology.

To truly explore the range of performance demands in a job, a multi-method approach to job analysis is essential.  In our work with clients we deploy the various types of data gathering methods in the diagram below to ensure a robust and rigorous examination and documentation of the performance drivers in a job.

Multi-method Job Analysis Used by Shaker Consulting Group

Define Exceptional Performance

Competencies have become the language of performance.  Proper job analysis captures and documents the examples and descriptions of exceptional or superior performance and the results are often summarized as a competency model.  The models are a collection of behaviors that together deliver superior on-the-job results.  Competency models are used to create behaviorally anchored rating scales (BARS).

BARS can be organized various ways to describe differences in performance.  One method is simple task statements such as: Engages others in decision making.  This can be used to rate an individual by adding an effectiveness rating scale offering a range from high to low.  Another format is a proficiency scale that documents observable differences ranging from novice, to proficient, to expert such as: contributes to team decision making process, facilitates team decision making process, designs and manages team decisions making processes.

Comprehensive rating methods can then be used to evaluate and document new hire performance at various stages along their learning curve.  This periodic evaluation documents time to proficiency.

Link Candidate Evaluation Methods to the Job Criteria

Employment law states candidate evaluation methods should be job relevant and consistent with business necessity.  These standards hold true for criteria used to sort resumes, content of telephone screening interviews, content of interviews and the content of any form of test or assessment.

With a well developed and rigorous competency model clear criteria can be established for each level or step of candidate evaluation.  Scorable applications can be created that place weight on various KSAOs.  Phone and in-person interviews can be organized by competency and scored with BARS.  And objective and job relevant assessment and pre-employment simulation methods can be designed or selected and administered.  Each of these methods adds a level of job relevant evidence to determine the degree of candidate-job fit.


Every leader knows the quality of a decision is dependent upon the quality of information used to make that decision.  Improving quality of hire – or the outcome of a hiring decision – requires a great quality of hiring process.  Staffing process improvement can be driven by increasing the quality of evidence obtained on candidate-job fit.  Linda offered a good spring board to raising the quality of the hiring process.

Free Book on Evidence-Based Hiring

To learn more about evidence-based hiring ask for a free copy of our e-book: Practical Rigor.  Go to our Contact form and write Rigor into the text box. We will send you a copy.  Read it, share it with your colleagues, call us to talk about your reactions to our suggestions.  Together, we can help raise the skills and methods needed to improve quality of hire across the profession.

RSS Feed LinkedIn