Shaker Consulting Group Logo Virtual Job Tryout Logo
Blog home page

Posts Tagged ‘employee selection’

« Older Entries
November 7, 2011

Blinded By Star Gazing and the A Player Myth

Writing in the recruiting space has generated a lot of attention on strategies for hiring A Players, Top Talent and Star Performers. While that sounds great, I think all that star gazing has blinded a few recruiters. In part, having a poorly calibrated candidate evaluation process in place is to fault.

There is no team roster filled with the likes of Lebron James, Michael Jordan or Barry Bonds, (Although Miami tried). There is no company executive committee completely staffed with the likes of a Warren Buffet. The reason can be largely explained by population statistics. A Players or bright stars only make up a small percent of the available population.

As such, it is more of a myth to make all A Player hires. The size of the candidate population might have to be enlarged exponentially to create a finalist pool of only A Players to choose from. That could be a monumental task. The organization might not have the appetite for the time requirements nor the budget to complete such an undertaking. There is another approach to contributing to organization performance with each hiring decision.

Dim Stars Get Hired Too
Making the right hiring decision requires complex reasoning. To put this into perspective requires that you wrestle with another concept within population statistics known as variation. This can be best understood by looking at your hiring track record. You hired your best, and you hired your worst. When you examine the performance differences between those hired into one job, the variation in decision quality is revealed. Using a process improvement tool called Pareto Analysis (80-20) the impact of low end variation can be revealed.

Obtain a data set of performance variables from a group. Sales performance is easy place to look. Obtain territory revenue per sales rep in a spread sheet.  Calculate the average sales per territory. Next calculate the average for the top 80% and the bottom 20%.   Look at the gap.  After you stop shaking your head, you have to admit, “Yes our process hired those bottom 20% folks too.”  You can explore the impact of this with our ROI Calculators.  You can perform this same form of HR Analytics on any dimension of performance. It is pretty revealing.

This analysis reflects the current nature of the population from which you draw, and the decision quality variation that allows in your poorest performers. In your Shining Star hiring program, Dim Stars get hired too.

Scale of Magnitude
Hipparchus , the ancient Greek astronomer created a the six point magnitude scale to calibrate the relative brightness of stars. Since then the scale has been expanded, revised and refined to better describe the difference observed in the brightness of heavenly bodies.  Hipparchus uses analytical models to refine his conclusions. Your process hired the dim stars because of the calibration of your brightness scale.   Shining stars and dim stars looked more alike than different. The evaluation process was unable to see the difference.  Using HR Analytics your candidate evaluation can be refined and your hiring decisions improved. Better candidate data can improve the yield of your staffing process.  Maybe your recruiter was blinded by star gazing.

With a well calibrated candidate evaluation process, you get better data, which can support more effective hiring decisions.

Here are a series of examples of on-the-job performance differences that were identified by score ranges during validation analysis of a Virtual Job Tryout. Each chart depicts the performance gap between individuals who scored in the top 80% versus those who scored in the bottom 20% of a Virtual Job Tryout created specifically for the job they hold.

Top 80% Achieve 47% Higher Territory Revenue

Individuals who scored in the bottom 20% on the Virtual Job Tryout, on average produced $1.6 million LESS in sales revenue. In other words, about $1.6 million of performance variation was at risk with every hiring decisions. Recruiters letting soft glowing heavenly bodies into the sales organization.

Top 80% Achieve 8% Higher Closing Ratio

Individuals who scored in the bottom 20% on the Virtual Job Tryout, on average closed the deal on 8% FEWER opportunities.   In this case, millions of dollars of revenue are at stake with a lower closing ratio.  Candidates with less effective skills and attributes for bringing in new business were entering the sales organization.

Top 80% Scores Achieve 70% Higher Level of Products Booked

Individuals who scored in the bottom 20% on the Virtual Job Tryout, on average produced 70% FEWER transaction per month.  The previous candidate evaluation process confidently advanced less capable  individuals  into the sales organization.

Top 80% Scores Achieved 21% Higher Commission Levels

Individuals who scored in the bottom 20% on the Virtual Job Tryout, on average earned$21% LESS  in sales commissions. The Virtual Job Tryout is well equipped to discern underlying traits and characteristics that drive performance differences.

When you see the order of magnitude and the insight into performance provided by candidate results, ask yourself; “What would my workforce look like if I could hire from the top 80 %, or even the top 50% of the candidate pool?”

When you calibrate your candidate assessment process to on-the-job performance, you can better distinguish the difference between stars and black holes.

Call us for more information on calibrating your candidate assessment process to reduce low-end performance variation.  And, remember, the sun is a star. If you stare at it, you can go blind.

Posted by: Joseph P. Murphy | November 7th, 2011
Permalink, Comments Off
, Tags: | Posted in General, HR Analytics

September 27, 2011

Is Interviewing a Waste of Time? Ask Kevin Wheeler

Candidate Evaluation Time Well Spent?

In his recent post on ERE, Kevin Wheeler suggests interviews may be a waste of time. He poked the nest and stirred up some good interaction. He also suggests, that the use of more objective methods, such as simulations for pre-employment testing can improve the candidate evaluation process.

At the end of the day, the hiring decision is an act of personal judgment. The interview plays a big role in supporting that decision. And suggesting that companies abandon the interview will fall on deaf ears. However, the bigger question is what methods and criteria are used to determine who to interview. With high applicant to hire ratios, getting to the best candidates, the ones worth investing the high cost of interviewing, is a place to put some considerable focus.

A note about the gap between interviewing concepts and practices.
I conducted a survey with SHRM and in 2006 published a white paper The Use of Objective Candidate Evaluation Methods. (Non SHRM members may write me for a copy).

Here are some stats.
55% of respondents stated they use behavioral interviews that are based upon questions prepared in advance. 40% of respondents stated they do not conduct interviews with prepared questions. Kevin most certainly has his editorial comments pointed at this group.

When I explored further, and asked who uses behaviorally anchored rating scales and numeric summary of interview outcomes. Only 24% of respondents stated they use of this known best practice. This is the group that Tom Janz (see comment to Wheeler’s post) may be using to support his retort.

Another way to look at it is this:
Every company interviews (bold assertion) in their candidate evaluation process.
40% of companies begin with no preparation for what they want to learn.
76% of companies have no structured way to evaluate what they hear.

I think Kevin’s assertion has more generalized weight about the current state of affairs than Tom Janz’s point (see comment to Wheeler’s post) about results that can be achieved when interviewing is done exceptionally well. The process that Lou Adler teaches is a great example. Lou demands adherence to the process to get the results he purports.

Kevin’s bigger point is that there are better ways to learn more objective information about a candidate. And thanks to Tom Janz for the reference to the Virtual Job Tryout as one example (see comment to Wheeler’s post). At issue here is the professional/technical capabilities of practitioners in talent attraction and selection roles and their ability and desire to build rigor, selection science, and discipline into their practice. Kevin is inviting a shift to more thoughtfully developed and objective methods for candidate evaluation. When your process obtains better candidate data, you may be able to make better decisions.

Thanks Kevin.

Posted by: Joseph P. Murphy | September 27th, 2011
Permalink, Comments Off
, Tags: | Posted in General, Surveys

May 5, 2011

Candidate Experience – Make It Engaging and Interactive, Part 6 of 6

This is part of a series connected to the Candidate Experience Monograph

We asked job seekers to clarify their outlook for an interactive application experience two ways.  In general, we wanted to know if there was a strong preference for engaging activities and if there was an expectation for interactive activities over text only experiences.  The large number of neutral responses to a preference for interactive experience might indicate that candidates do not have enough exposure to job applications with this feature to have a strong opinion one way or another.   However, the majority of job seekers have expectations that they will find a more engaging candidate experience than just reading about the job.

Click Here to Play

Most careers pages have not kept pace with delivering the type of robust interaction that can be found so easily just about anywhere else online.  Why?

Maybe you have noticed the web has become a fascinating place, combining qualities of a playground, a maze, an entertainment center and a shopping mall,  just to name a few.  Contrast that diversity of experiences with your job application.  How ya feeling about that?

John Sullivan suggested our careers web sites are boring our candidates.  Go and apply for one of your jobs and rate the experience from dreary (1) to dynamic (10).  A little bit of interaction can take your candidate experience up a notch.  And 37.6% of candidates want it.  53.4 % of candidates are Neutral about it.  But only 9% are sure they don’t want it.  My recommendation would be to satisfy the preference in the group that want it and convert the majority who are on the fence, but could be fans.

Candidates Prefer Engaging Experiences

Inspect What They Expect

When 46.9% of your candidates expect to find engaging activities in your online job application, what should you do?  Deliver.

Candidates Expect Interactive Experiences

Similar to the response pattern on a preference for engaging activities, a large group, 32.5% of job seekers, state a neutral position on their expectations for interaction.  My position remains the same: candidates have a poor frame of reference.  I do submit though that time spent on other web sites is highly correlated to the degree of interaction found on the page.  This might be reinforced by the small number who disagree and have little or no expectation to find interactive tasks and activities when applying.

Candidate Reactions to Interaction

We ask every candidate completing the Virtual Job Tryout for feedback about their experience.  Some candidates continue to sell themselves, yet others share a comparative view of how this form of application is different.  Here are a few poignant examples.

“Not your ‘typical’ online job application. I actually really enjoyed the process and I feel as though I have a better understanding of the job and its requirements and that [Company] will have a better understanding of me as an individual and not just what is on my resume. “

“This was the most interactive experience I have ever had while applying for a job!  I enjoyed how this puts the applicant in real life situations and tests how well you can meet the challenges.  The team of people who designed this program are very creative and intuitive to the needs of [Company] as an employer.

“I am very impressed with how well this was set up.  It’s the first ‘pre-screening’ application I’ve encountered that really challenged the way I think. It’s creative, easy to use, and works exceptionally well at describing the job role.  Thank you for the perspective. “

“The Virtual Job Tryout is exactly what potential employees need to do, so that they can see what they may be faced with. I never knew until doing the tryout, that there was so much that a Representative has to accomplish during a phone call. “

In 2006 about 6% of companies stated they were using simulations in their candidate evaluation process.  In 2010 that had risen to 12%.  That trend can be expected to continue.  So, watch it, or become part of it.  Take some decisive action to improve the interactive nature of your candidate experience.

Part One,  Part Two,  Part Three, Part Four, Part Five

Posted by: Joseph P. Murphy | May 5th, 2011
Permalink, Comments Off
, Tags: | Posted in Candidate Experience, Surveys

March 22, 2011

Candidate Experience – Communication Preferences, Part 3 of 6

This is Part Three of a series connected to the Candidate Experience Monograph

We asked job seekers about their preference for how they wanted to communicate with recruiters and hiring managers during the application process.  We wanted to see if there were strong preference one way or another, and in fact there are.

Candidates have preferences for communication modes

In order of preference:

  1. Telephone
  2. Email
  3. Paper mail
  4. Texting
  5. Chat

You Know My Name, Look Up My Number

Job search is a personal endeavor through a largely impersonal maze or obstacle course.  Candidates want to make a connection on a personal level with another human being.   At the end of the day, each candidate knows the hiring decision will ultimately be made by another human being. As such, candidates have the highest preference for talking with recruiters on the phone.   A very common retort from unsuccessful candidates is the classic:“if only I could speak with someone, I could sell myself.”  The telephone was the communication vehicle of choice for 88% of respondents

Everybody uses the phone somewhere in their recruiting process.  However, the personal connection of a phone call is reserved for the most highly qualified candidates.  And with the applicant-to-hire ratios common to high volume sourcing, this means that 50% to 98% of candidates will never hear the ring, never experience the type of interaction they prefer the most.  It creates a clear and pronounced expectation gap that will not be closed.  Even making the ‘No thanks’ call to the unsuccessful is impractical, given the number in the rejection pool.

Digital Hand Shake

There is hope.  Email was rated at an almost identical level of preference with the telephone at 87.4% and 88% respectively.  And the number of respondent who are neutral (10%) or do not prefer email (1.6%) are also fundamentally the same as with telephone.  The simple answer here is that an email can fill the communication gap and, if done well, deliver a candidate experience that meets certain expectations for communication.

The similarity here raises my curiosity.  The relative value of each medium is on par.  My assumption is that candidates may sense that a reasonable degree of thoughtfulness might go into each form of exchange – a dialogue or a digital handshake via email.

For candidates receiving an invitation to continue in the process, it is easy to see how an email will have a similar impact as a phone call.  The ‘you are wanted’ message will ring true and provide the needed details for the next step.

In spite of mail-merge mass messaging, that powerful phrase – “you’ve got mail” must meet a certain need and override (at least initially) any negative perceptions of getting rejection spam.  As was detailed in Part II, candidates want to know where they stand.  Email can be expedient and personal to a degree, in conveying the message and closing an expectation gap.

TXT? Not!

Given the heavy balance of twenty-somethings in the survey, it is somewhat surprising to see the strong negative attitude (52.9%) toward the use of texting as a form of communication in the job search process.  Without any qualitative data behind these responses, a few assumptions come to mind.  The foremost being jobs are far too important a topic to be left to abbreviations and truncated phrases.

While not completely discounted, 22.7% of respondents would find texting acceptable.  I only have one personal trend line as a point of reference here.  My college-age son’s use of texting peaked during junior year at about 2,500 per month.  I understand this was a pretty typical volume in 2010, but the 75% fall-off in his use of this form of communication caught my attention.  Certainly not a vehicle going the way of the telegraph, but maybe a shift in the how and why it is being used now. (Excuse me while I tweet this post.)

This also points to the viability of text based job posting such as TweetaJob.  It may make it easy to distribute – the push into a community.  But does it get the attention?

Chat me Up? Not so much.

Chat service has immediacy and it comes with some sense of intimacy.  After all, someone is reading and responding, often pretty quickly.  Job seekers might feel the personal connection with the individual at the keyboard on the other side.  However, having done job observations in contact centers, I know an individual agent may be carrying on six or more conversations at once. It’s easy to see that candidates might feel the person on the other end of the chat box has little or nothing to do with evaluating job-fit or making a hiring decision.  That may be why44.2% of candidates prefer not to use chat for the recruiting process and only 16.9% indicate a preference for this mode.

Give a Letter to the Postman

How can you go wrong with a letter to the candidate?  Think of the reaction you would get from candidates.  Sending snail mail almost seems arcane, but how classy.  And 50% of job seekers still rate this form of communication as highly preferred.  The most consistent use of snail mail in today’s staffing process might be the offer letter.  Again, this form of communication only touches the small percent of candidates who make it to the finish line.

However, with a combined 85% of candidates prefering or being open/neutral about hard copy, there may be a place for physical components in your candidate communications.

Final Thoughts

The good news is that job seekers are OK with email.  Email is extremely efficient and low cost.  Automated methods make it easy to use this tool to communicate with your candidates.  All your candidates, the successful and the yet to be.

When considering the need for process information described in Part II, and the preferred method of communication described here, you should be able to add value to your candidate experience and achieve a measurable degree of staffing process improvement.

In the next part of this series, we will explore candidate reactions and behaviors to technology problems with your on-line application.  Patience may not be their virtue.

Part One,   Part Two,  Part Four, Part Five,  Part Six

Posted by: Joseph P. Murphy | March 22nd, 2011
Permalink, Comments Off
, Tags: | Posted in Candidate Experience, Surveys

February 10, 2011

Do You Have A Talent-matician?

Kevin Wheeler wrote a great article on ERE asking about selection science and measurement.  His is suggesting staffing professionals adopt better methods for candidate evaluation or assessment and make more effective use of HR analytics to link candidate evaluation data to business outcomes.

Here are a few questions around measurement discipline, the answers to which may be revealing.

  1. Ask your CFO – “How much has been invested in the data capture and analysis system you use to report EBITA?”
  2. Ask your EVP of Sales – “How much has been invested in the data capture and analysis system you use to report daily sales performance?”
  3. Ask your EVP of Manufacturing ; “How much has been invested in the data capture and analysis system you use to calculate process yield?”
  4. Then ask your EVP of HR (self) – “How much has been invested in the data capture and analysis system you use to create a differentiated work force?”

In every case, for Fortune 1000 companies, the answer to the first three will be hundreds of thousands and in some cases millions of dollars.  Unfortunately the answer to #4 typically pales by comparison.  Why? 

I have never sat with an executive who stated their organization was just like their competition.  In fact, great pride is expressed in how their people, their products, their services are different than others.  The work that true talent-maticians (I just invented that) do is using HR analyitics in quantifying, to the degree possible, the human variables that contribute to those differences.  That requires, rigor, discipline, experiment design, and time.

Michael Porter of Harvard suggests competitive advantage comes from business processes which are difficult to copy.  Authors Becker, Beatty, Huselid, in The Differentiated Workforce present a similar framework for evaluating HR practices that put forth a ‘Me Too’ or a Differentiated outcome.  An example of this is the use of off-the-shelf assessments without local validation.  By default the user states, we are willing to use a measurement tool developed for and by someone else and calibrated by another organization to provide data on our talent decisions.  Sounds like a Me Too tactic.  One path to a differentiated workforce is at least conducting a validation analysis on how the measurement tool (pre-employment test) is adding value to your decision process.  The underlying premise is that a good assessment provides a degree of better data and therefore, better decisions.   With in-house validation, you document the relationship between assessment results and business outcomes. 

Without an in-house validation, the test is not calibrated to performance in your organization and outcomes are anecdotal.  The practice that gives assessment a poor reputation is poor implementation.

In an earlier work by the three authors above The Workforce Scorecard, they document those organization hiring a higher percentage of employees with validated evaluation methods achieve higher levels of financial performance.  Aon and SHRM conducted a significant piece of research in the mid 1990s that included a glimpse at staffing process outcome (out of print but avaiable from the research dept).  Survey participants stated the most lacking qualities in new hires were defined as work style, and basic reasoning.  Those traits or attributes can be objectively evaluated with a variety of pre-employment tests.  Companies stating they were most satisfied with staffing process outcomes were using the most comprehensive candidate evaluation methods.

  • Companies hire engineers to solve complex measurement problems.
  • Companies hire actuaries to solve complex measurement problems.
  • Companies hire statisticians to solve complex measurement problems.
  • Companies that know their competitive advantage comes from their people hire industrial organizational psychologist to solve complex measurement problems in staffing.  These folks are the talent-maticians.

Even if you do not measure variables that provide insight to performance potential, performance variation exists.  In fact you hired your best performer and your worst performer with the same evaluation process.  In manufacturing terms that is known as performance variation and is marked by upper and lower limits.  You see, staffing is a business process with a yield to measure and manage.  To do that requires data capture and analysis.

However, enter another piece of data.  It has been known for some time that a structured interview extracts better candidate evaluation data than an unstructured interview.  In a survey on Use of Objective Candidate Evaluation Methods I conducted with SHRM (write for a copy), very fascinating evidence of interview practices emerged.  Only 55% of respondents stated they use behavioral interviews with questions written in advance (an intentional discovery process).  When asked if the interviews were supported with behaviorally anchored rating scales (a method to discern an effective response from an ineffective response), only 24% of respondents stated this practice was used.  Staffing practitioners are largely ignoring known practices which at the simplest level produce better outcomes.  Implementing assessments requires the same rigor the CFO expects from data capture and analysis in financial matters.

In some jobs, learning more about what factors contribute to retention can add signnficant value.  However,most companies do not even  measure and track the cost of early turnover.  In a survey on Staffing Waste I conducted with SHRM (write for a summary), only 8% of 636 respondents stated they track and report the costs of what I call False Starts – new hire turnover that occurs in less than 120 days.  The analogy would be a head of manufacturing that does not measure defects and scrap rates.  Manufacturing is held accountable for managing the yield of that process.  In my paper Staffing Waste: Identify it, Measure it, Reduce it, a range of examples for applying measuremen- based process improvement to staffing is offered. You can read it here

Yes Kevin, the future of staffing practices will include more measurement, more science, more accountability for understanding and managing process yield.  There are exceptional methods to evaluate candidate-job fit.  It can be measured, it can be analyzed and it can contribute to the bottom line.  However, the practice leaders are already out there, doing the work right now. 

For one example kook at the 2010 ERE Award winner KeyBank.  They reduced staffing waste in one position by over $1.7 million in one year by bringing science and measurement rigor into their staffing process.  They were able to add objective candidate evaluation in a manner that measured candidate-job fit.  The retention and gains in a range of job performance metrics are impressive.

We have many more examples of how talent-maticians drive economic impact from staffing process improvement.  To explore the scope of opportunity you might have, see our ROI calculators.Call me.  We can discuss your opportunity.

Posted by: Joseph P. Murphy | February 10th, 2011
Permalink, Comments Off
, Tags: | Posted in General, HR Analytics, I/O Psychology, Surveys

January 7, 2011

SHRM HR Magazine Features Virtual Job Tryout – Effective Assessments

When doing research on thought leadership in assessments, writer Dave Zielinski was directed to us by several resources.  He wrote a thoughtful article for SHRM’s HR Magazine from his findings.

Dave found out that market leaders, high performing organizations and highly brand conscious companies are using custom simulations for pre-employment testing and employee selection.

In the article, Dr. Nina Brody of Take Care Health Systems (a division of Walgreens) stated: “the assessment gives candidates a highly realistic job preview – causing some to self-select out early and others to solidify their commitment – and creates an impression that Take Care is operating at technology’s cutting edge..”

Also in the article, Beth Yates of KeyBank states, “the simulations create an interactive, highly immersive, multi-media experience; they mimic key job tasks and test for competencies such as providing client service, adpting to change, supporting team members, following procedures, cross selling, and working efficiently…”

Simulations deliver a highly engaging multi-method assessment inside a company branded candidate experience.  Candidates walk away feeling like they learned a great deal about the job and the company.  The company also learns a great deal about the candidate from the work sample they provide by completing the Virtual Job Tryout.

If you are a member of SHRM, check out the January issue of  HR Magazine article Effective Assessments on page 61.

If you are not a SHRM member, drop me an e-mail at joe(dot)murphy at shakercg.com (you can figure that out)  I can forward the article to you.

Posted by: Joseph P. Murphy | January 7th, 2011
Permalink, Comments Off
, Tags: | Posted in Candidate Experience, Client Case Studies

Virtual Job Tryout Overview – Improving the Candidate Experience

The nature of our experience economy has caused the population in general to expect more from every form of interaction.

  • More engaging – provide me with an experience that draws me in.
  • More informative – feed me information in a manner that is easy to digest.
  • More interactive – give me a role to play.
  • More rewarding and satisfying – help me walk away thinking I am glad I took the time to do this.

Candidates expect the same qualities from the employment experience too.  Ideally, they want to look inside the job, try their hand on some of the tasks, experience the decision making and problem solving they will face on the job.   And, the great thing is that you can do that now.

The Virtual Job Tryout is a custom simulation for pre-employment testing.  Key elements of your job are recreated in an interactive on-line environment.  Candidtes get to take the job for a test drive.  In doing so, they learn a lot about the job.  And, the work sample they provide allows you to learn  a great deal about them at the same time.

Send me an e-mail  joe(dot)murphy@shakercg.com to recieve a link to a Virtual Job Tryout demo.  You can learn more about how the Virtual Job Tryout is working to improve the candidate experience, extend the company’s brand message into the market, and provide recruiters with evidence-based decision support.  

You can deliver an exceptional return on investment while delivering a candidate experience that create fans.

Posted by: Joseph P. Murphy | January 7th, 2011
Permalink, Comments Off
, Tags: | Posted in Candidate Experience, Recruiter Experience

September 29, 2010

Intuition or Intelligence: How Do You Hire?

Talent Intelligence was a big theme at TaleoWorld 2010.  Taleo CEO Michael Gregoire, in his opening remarks stated 47% of new placements into management positions fail.  I am not sure where that statistic came from, but it does not speak well about how companies are making decisions to hire or promote individuals.  It makes me ask:  Is the hiring decision based upon intuition or intelligence?

Where else in business would a 47% failure rate be tolerated?  What Mr. Gregoire is referring to here is one form of staffing waste.  This abysmal success rate seems to indicate a strong need for talent intelligence.  Better candidate data for making more accurate hiring decisions.

Getting useful, meaningful data is the central challenge.  Hiring managers and recruiters, while well intended, often place disproportionally high value on candidate data that is either not related to job performance or even worse, negatively related to job performance.  And, one of the more common areas where we see this is the value placed on specific job experiences that while intuitively seemed to make sense, the evidence from HR analytics proved othewise.  Here are a few examples. Previous cash handling experience negatively related to cash drawer accuracy, prior food service and hospitality experience negatively related to success in a food and beverage management position, previous sales experience with a competitor negatively related to sales success.

Thoughtful people in successful companies establish these screening criteria.  However, in the majority of cases these criteria are assumptions.  Assumptions that are never tested or proven.  By not conducting the appropriate HR analytics, decisions get made based upon ego, not evidence.  This is allowed because of the common and accepted assertion from recruiters and hiring managers: “I am a good judge of talent.”  With a 47% failure rate, it would seem prudent to do some analysis.  Just better than a coin toss does not seem like good odds for a critical and expensive business decision.

Employee selection is a process.  The yield of the process can be measured and improved. Candidate evaluation with pre-employment assessments can be conducted in a manner that produces evidence in the form of data. This data can support HR analytics which in turn provides guidance to improve the objectivity and effectiveness of the hiring decision.  If you are a Taleo user and want to make your ACE work better, we can help.

Check out a few of our case studies to see how HR analytics and pre-employment testing validation analysis have made a measurable difference in the yield of a business process called staffing.  We can help you make the transition from ego to evidence, from talent intuition to talent intelligence.

Posted by: Joseph P. Murphy | September 29th, 2010
Permalink, Comments Off
, Tags: | Posted in Client Case Studies, Conferences, HR Analytics

September 23, 2010

Dissertating and Assessement Development – Similar Scientific Methods

I am Lei Qin, (sounds like: ‘chin’) the newest member of the Shaker Consulting Group team. 

I have recently completed the defense of my dissertation. It was a mix of feelings: relief and anxiety. Why was it a relief? I don’t need to make more endless revisions. This is really a relief.  But as the defense date approached, anxiety accumulated, slowly but robustly.  How did I deal with it? I took a simple strategy: not thinking about it too much until it comes.

I am not blogging to just expose my feelings about dissertating. My experience as an intern with Shaker provided a window of time in which building employee selection tools and dissertating overlaped in my life. This leads me to an interesting observation. A similar scientific process is shared by building a pre-employment assessment and dissertating.

 The first step of building an assessment tool for hiring is job analysis. The primary purpose of job analysis is to understand the targeted position and find out what knowledge, skills and abilities and other variables are essential for superior performance in the job. A competency model of the position is developed based on the job analysis. The first step of dissertating is a literature review and proposal. This step helps to define and understand the research progress of a specific area. After thoughtful consideration you can make a hypothesis about what is missing, unanswered and critical to advance this research area. A research proposal is developed based on the findings of literature review.  In my case, I chose Trust in Leadership as the focus for my research.

The second step of building a hiring assessment tool is content development. A story board, measurement hypothesis, item content, and computerized deployment system are developed based on the competency model and the job analysis results. The developers need to draft or choose appropriate testing components to assemble the pre-employment test. Similarly, the second step of dissertating is research design and material development. A researcher needs to choose appropriate designs for the study and develop study materials such as instructions and survey items. For my research I created three experimental conditions to activate different mindsets in anticipation of influencing down-stream thinking and responses.

The third step of building a assessment tools for hiring is validation analysis. Typically hundreds of participants are invited to take the selection tool. Predictor (test results) and criterion (job performance) data are collected. HR analytical tools are used to explore the relationship among the response patterns on the test and on-the-job performance. This type of HR analytics determines the power of the test to predict on-the-job performance. The third step of dissertating is data collection. Subjects will participate in the study. Data of independent variables and dependent variables are collected. Statistical analysis is conducted to test the hypotheses. My research explored the differences in how concrete versus generalized knowledge about an individual impacts how we establish trust with them.

The final stage of building an employee selection tool is rollout.  After a thorough review of the results, the selection system needs approval from the selection practice leaders prior to rollout  The pre-employment assessment system will be tested in the real recruiting environment. Similarly, the final step of dissertating is defense. The success of a dissertation comes from the approval from a dissertation committee, prior to rolling out a new Ph.D.  Well, I did it! My research was determined to contribute to the body of knowledge on leadership. And now I have been rolled out with a Ph.D. and Shaker has hired me to continue my work and research in the real recruiting environment.

The comparison of building a selection tool and dissertating demonstrated they follow the same rigorous scientific process. At Shaker, the development of Virtual Job Tryout® has one more component; that is, managing a feedback loop which makes the Virtual Job Tryout® adaptive to the real world, getting more accurate.  Data will be collected over time and periodic validation analysis will be conducted to improve the predictive power and document the return on investment (ROI) of the pre-employment assessment 

Earning my Ph.D. has been just one step along my path of continuous learning from experience.  And just like an assessment can get smarter over time, my skills and ability to contribute to a body of knowledge will be tested time and time again.

Posted by: Lei Qin | September 23rd, 2010
Permalink, Comments Off
, Tags: | Posted in General, HR Analytics

September 17, 2010

Pre-employment Testing in the Experience Economy

Charles Handler wrote about the movement from test to experience in his ERE article.  It was a great invitation to consider the candidate experience.  John Sullivan wrote a few years ago about how career web sites are boring candidates. It may actually be worse.  Applying may have total disregard or abuse in the candidate experience. While some corporate careers pages have added a touch of pizzazz with videos and testimonials, the actual application and pre-employment assessment components continue to be ignored by many, but not all.

I spoke on Pre-employment Testing in the Experience Economy at the SHRM Staffing Management Conference in Orlando this year.  The premise was that candidates expect more.  More information, more engagement, more use of multi-media, more insight into the job and culture.  More support to their decision making process.  Simulations offer the candidate a lot more of what they seek in learning about and applying for a job, or even better yet, a career.

 At Taleo World this week, I had a conversation with an individual who’s firm just implemented a long standing, yet very traditional assessment.  She recently completed the assessment herself and without all the emotional embellishments, this is how she described it: “The questions were stupid!”  “There did not seem to be any relevance to the questions.”    I asked her:  “How do you think your candidates will feel about completing the assessment?  It made her eyes pop out.  It was as if this was the first time anyone had invited her to consider the candidate experience.

Pine and Gilmore have been writing about the experience economy since 1999.   There is a lot business in general and recruiting in particular can learn from their research and point of view.  They suggest we consider and evaluate an experience with two continuum variables: interface and immersion.

Interface is the type of interaction the candidate is offered.  At one end of the continuum is read and watch, the other end comprise choices and interactions.  Examples would be reading a job description to typing, clicking radio buttons, dropping, dragging among options.

Immersion addresses degrees of cognitive, emotional, and physical engagement.

At one end is attending to, studying, absorbing information, at the other end is active processing, raised emotional and physical participation.   Examples would be examining a puzzle to racing to complete a timed exercise.

Creating a matrix that overlays these to continuum provides an evaluation framework to determine the nature of an experience.  The diagram below sets out four types of experiences: Educated, Entertained, Enthused and Engaged.

Candidate Experience Evaluation Matrix

Educated – Traditional media, Web 1.0

Entertained – Video games and cut and paste, drop and drag

Enthused – Movies and Videos with a compelling message, realistic job previews (RJP)

Engaged – Challenging mental and physical tasks, Wii and other dynamic games

Simulations as pre-employment assessment draw the candidate immediately into a high interface, high immersion experience, thus delivering a cognitive, physical and most importantly, an emotionally charged experience.  

  • Selection assessment exercises that can be deployed in simulations include activities such as:
  • Situational judgment – listen and choose what to say next in a challenging conversation
  • Problem solving – information look-up task from an interactive information source
  • Diagnosis – use rule-based logic to determine fault or errors
  • Business acumen – reasoning with financial statements under time pressure
  • Keyboarding – data entry and accuracy, under time pressure
  • Visual estimation – quick calculation of quantities from pictures or illustrations
  • Productive thinking – idea generation capacity in finite time frames
  • Prioritization – compare and differentiate among competing resources

And the list goes on.  Web 2.0 and emerging interactive technologies affords companies the opportunity to deliver a candidate evaluation experience that engages, informs and satisfies their desire for more from the application process. Simulations make it easy to deliver a multi-measure evaluation. Therefore the power and accuracy of selection science available from simulation based pre-employment testing cannot be achieved with conventional assessments. The return-in-investment (ROI) from implementing a simuation can be huge.  Some approaches to project the impact can be explored with these ROI calculators.

Very few organizations evaluate the candidate experience. Candidates are not given a chance to describe their reaction to the application process.  However, they do think about it, they do have opinions about it and it does impact how they think and feel about your company.  Candidates who experience a simulation as part of the application process have a lot to say.  Read some of their feedback here.

Virtual Job Tryout is a simulation for pre-employment assessment.  Each is custom built and validated for a specific job. Candidates find this type of experience highly rewarding, very job relevant and are willing to talk about it in a positive manner.

If you would like to deliver a more engaging candidate experience and deliver a more results oriented recruiter experience, give me a call  216.292.0202

Posted by: Joseph P. Murphy | September 17th, 2010
Permalink, 2 Comments »
, Tags: | Posted in Candidate Experience, Conferences

« Older Entries
RSS Feed LinkedIn