Shaker Consulting Group Logo Virtual Job Tryout Logo

Archive for the ‘Client Case Studies’ Category

October 14, 2011

Assessments Are Now As Much About the Brand

Top Employers Deliver Better Candidate Experience

According to Moses Bar-Yoseph, the national director, talent attraction, for KPMG in Canada, “The line is now blurring between assessment and branding.”

KPMG Canada recently launched a pre-employment assessment for managerial candidates.  Positioned as a Day-in-the-Life experience, it provides a candidate experience as unique as the KPMG brand, and as challenging as the role of a manager in tax, audit or consultancy.  KPMG sees candidate engagement as a two-way process of both education and evaluation.

The KPMG Canada virtual job tryout was developed with high levels of involvement from across the firm.  Hundreds of existing managers completed the initial version of the assessment to support the in-house validation analysis.  Their responses to various day-to-day work issues, job relevant questionnaires, and individual business results were analyzed to create a scoring algorithm.  The go-live version reports on Overall Fit and provides ratings on KPMG Canada specific competencies.  Recruiters identify best-fit candidates quickly and objectively.  The results provide better candidate data for comparing and contrasting among candidates and exceptional decision support for hiring managers.

Bar-Yoseph was recently interviewed by Todd Raphael of ERE Media about the process.  The article is here.

July 21, 2011

Gaining Management Acceptance for Assessment Tools

This spring over Easter dinner, I was told by a highly successful, MBA grad from an Ivy League university that he could tell, without a doubt, whether a candidate was right for a job in 5 minutes flat. Perhaps he could. After all, Babe Ruth could hit 60 home runs while breaking every training rule in the book.

AssessmentUnfortunately, few of us are the recruiting equivalents of Babe Ruth. We need all the help we can get to accurately assess and select the best candidates for our jobs. But, how can we convince the would be Babe Ruth’s that there is a better (read: more valid, reliable, legally defensible and fairer) way to assess candidates?

My experience, working as both an external and internal consultant in candidate assessment, points to 3 important factors in gaining assessment tool acceptance:

  1. Assessment Fidelity, how close the assessment looks like the job, is assessment‘s secret weapon. It not only has a stronger track record in prediction of success for most jobs, but more easily gains acceptance from hiring managers and candidates alike. They “get it” because they can readily see and experience the job through the assessment process. Fidelity enhances the candidate experience.
  2. Have a Champion, Mid-way through a recent assessment project, we lost our executive champion. At that point we needed to re-sell the project and find a new champion. We didn’t; leading to acceptance challenges. Lesson learned. Visible leadership endorsement is a must.
  3. Ease of Use impacts  2 audiences: candidates and hiring managers. Hard to find candidates may not always be willing to take your best thought out assessment tools (though fidelity helps a lot here too). This in turn can lead to hiring managers blaming the tool for sourcing problems. Recruiters need to have a clear candidate message to convert those sourced into applicants. Know the organizational culture you’re working with. Are hiring managers and candidates patient, analytic and detail oriented or, fast paced, go with the gut types? Tailor your tool to meet your audiences’ appetite and realistically balance assessment style and time against tool acceptance.

A valid tool with strong psychometric properties and eye popping utility numbers is still only as good as management’s willing acceptance of it. The key to assessment success goes beyond validity. It is achieved by gaining the support of hiring managers through fidelity, a visible champion and making it easy to use.

Administrator’s Note:

John Miraglia is our first external contributor to the blog, a former client, and professional colleague. He has worked on the implementation of the Virtual Job Tryout for professional positions in the financial services industry. His insight and experience on implementing assessment is highly valued.

May 26, 2011

Music Video Meets Realistic Job Preview

Amerigroup, a leading provider of Medicaid insurance wanted to extend a compelling invitation to become part of the real solutions they provide.  They had already developed a collection of videos that demonstrate the nature of their work and profiled the clients they serve.  Commissioning The Verve Pipe to write, record, and film a song that pulls together the Amerigroup service brand created a realistic job preview with an emotional appeal that is palpable.  The best part is that this message works just as well for current associates as it does of candidates.  So Rise Up and click Play to step into a very unique candidate experience.

Here are three reasons this form of communication works well in an employment brand message.

Compelling Story

The lyrics invite the candidate to step into and step up to the demands of the service role Amerigroup delivers to the community.

Balanced Images

The examples of the workplace and the variety of clients served are candid and frank.  This is not a gloss-over message.  These are real people, with real issues, getting real solutions.  No bait and switch or one-sided message.

Multi-media

In this “experience economy”, candidates expect more from the web, more interaction, more truthful information, more interaction.  Candidates walk away with a sense of the contemporary spirit of the organization.

Kudos to Amerigroup for the vision to combine music video and realistic job preview.

February 15, 2011

New Employee Testing Case Study: Fortune 500 Discount Retailer

As developers of the innovative Virtual Job Tryout®, Shaker Consulting Group’s latest case study looks at one of the fastest-growing discount retail chains in the United States.

To help predict order filling, accuracy, teamwork and core work traits, the retailer implemented the Virtual Job Tryout with amazing results.

To see the results of how Shaker improved quality of hire as measured by increased retention rate, performance and productivity, view our latest case study.

February 4, 2011

Improving Employee Selection Case Study

For one of our clients, the difference between a good hire and a bad hire is not just a projected figure, but a very literal cost of millions of dollars. Due to low turnover and low volume of hiring, it was difficult for them to find a customized employee selection process to fit their needs.

Enter the Virtual Job Tryout.

To hear some of the results, and learn why this client said of the Virtual Job Tryout “You didn’t charge us enough for this” click here.

January 25, 2011

Get Your Game On! – Three Questions about Your Candidate Experience.

Game designer and TED presenter Jane McGonigal had an article in the January 22, 2011 WSJ “Be a Gamer, Save the World.” She states we (the royal We) spend three billion hours a week gaming.  The average 18 year old spend 80 minutes a day gaming and extreme gamers spend up to 45 hours per week connected to a digitally delivered challenge. The quick lesson here is that engaging experiences hold our attention.  The first question is: What kind of engagement does your candidate experience deliver? 

Charles Handler, in his Candidate Bill of Rights suggests a candidate assessment should last no more than 30 minutes.  Is that to free up time for gaming?  Or is that because the assessments he is thinking about are a form of torture.  Maybe he is seeking to limit the pain of a putting a candidate through poorly designed on-line assessments. He wants to limit (but not ban)– Applitorture.  Or, the degree to which the applicant is subjected to mental or physical anguish from the act of applying for a job.

McGonigal goes on to suggest gaming can create a “blissfully productive” feeling.  And that “When we play, we also have a sense of urgent optimism. We believe we are up to any challenge…gamers spend on average 80% of their time failing in game worlds, but instead of giving up, they stick with the difficult challenge…”

Again, a quick lesson here is that an engaging challenge, even if it is difficult and demanding, will hold attention, and maybe even attract attention.  McGonigal supports this with asserting “..we like and trust someone better after we play a game with them – even if they beat us.“  The second short question is: What demanding challenge does your candidate experience deliver? 

When done correctly the candidate evaluation experience can be demanding, challenging, and highly engaging.  The candidate experience can be Applitainment.  Or, the degree to which the applicant obtains amusement or pleasure from the act of applying for a job.

McGonigal has used games to solve world problems with EVOKE.  And by her estimation believes if we spent about 21 billion hours per week gaming we can solve issues such as starvation, global warming and the like.  The task of creating a game challenge for recruiting is far less demanding, but can be equally productive at solving a quality of hire problem.

By bringing a dose of gaming mindset it is possible to create an assessment that combines the intellectual stimulus of solving a variety of tough day-in-the-life problems, the social context of competition for the job, and a rich graphical experience that presents your corporate culture and work environment.   And to quote McGonigal again,…”so many gamers feel they become the best version of themselves in games.” 

Now, the third question, In what way does your candidate experience bring out the best of an applicant?

A universal question we get asked is “How long is a Virtual Job Tryout?”  Followed by “We are worried about keeping it short.”  Do you think game designers ever ask how to keep their user experience short?  I think game designers ask, What challenges can we create?, What problems do we want them to solve?, What thinking do we want to push?,  What environment do we want them to feel and sense? 

We ask similar questions – What do you want to know about the way your candidate thinks, solves problems, and works with others?  What do you want your candidates to know about your job, your unique company?  A universal response is, “We want to know if they can be successful in the job and fit our culture.”  Success is complex, so a simple candidate evaluation experience just will not do.  To confidently assess job-fit and culture-fit requires robust candidate evaluation, the type of evaluation a simulation for pre-employment testing can deliver.

On average, Virtual Job Tryouts we have deployed take less than an hour to complete.  One of the longer Virtual Job Tryouts we have deployed takes 75 minutes to complete.  About 20,000 candidates a year complete it, vying for 2500 open positions as a medical technician.  That translates into collecting more than 25,000 hours, or twelve and half (12.5) person-years of candidate engagement time to provide decision support for the hiring demands of one position.  This company has rich evidence of which candidates can best solve their patient service challenges.

When asked about their candidate experience, here is what the applicants have to say:

  • 98% Agree or Strongly Agree – The Virtual Job Tryout helped me gain a better understanding of the position.
  • 95% Agree or Strongly Agree – Having experienced the Virtual Job Tryout, I am in a better position to determine if the job is right for me.
  • 99% Agree or Strongly Agree – Based upon my experience, I will gladly tell my friends about employment opportunities at this company.

And after 75 minutes of applitainment, a survey asks for feedback and reactions to the engaging and demanding nature of the candidate experience.  Here are some verbatim responses:

  •  It was an awesome experience, it felt like I was actually working for COMPANY. Thank you for this experience I now have a better understanding of what is expected of me.
  • What a wonderful way to begin an interview process! I thought your virtual tryout was well thought out and based on the various areas you are testing, gives you a good picture of not only the ability to draw blood with appropriate equipment, but gives you a look into the candidates thinking process. All employers should do this to screen their candidates.
  • The Virtual Job Tryout was wonderful, I enjoyed the experience.  It was very interactive and not nearly as dull as a paper test might have been.  It kept my attention.
  • I was impressed with the Virtual Job Tryout. I felt like it helped me understand the job and made me very excited to have a possible opportunity to work for COMPANY!
  •  Was a very good way to find out how qualified an applicant may be.

Simulations for pre-employment testing may not be on the same interactive plane as games, but they are a significant stride in the right direction.  If you are not satisfied with your answers to the three questions about your candidate experience, there are options.  Give us a call.

January 7, 2011

SHRM HR Magazine Features Virtual Job Tryout – Effective Assessments

When doing research on thought leadership in assessments, writer Dave Zielinski was directed to us by several resources.  He wrote a thoughtful article for SHRM’s HR Magazine from his findings.

Dave found out that market leaders, high performing organizations and highly brand conscious companies are using custom simulations for pre-employment testing and employee selection.

In the article, Dr. Nina Brody of Take Care Health Systems (a division of Walgreens) stated: “the assessment gives candidates a highly realistic job preview – causing some to self-select out early and others to solidify their commitment – and creates an impression that Take Care is operating at technology’s cutting edge..”

Also in the article, Beth Yates of KeyBank states, “the simulations create an interactive, highly immersive, multi-media experience; they mimic key job tasks and test for competencies such as providing client service, adpting to change, supporting team members, following procedures, cross selling, and working efficiently…”

Simulations deliver a highly engaging multi-method assessment inside a company branded candidate experience.  Candidates walk away feeling like they learned a great deal about the job and the company.  The company also learns a great deal about the candidate from the work sample they provide by completing the Virtual Job Tryout.

If you are a member of SHRM, check out the January issue of  HR Magazine article Effective Assessments on page 61.

If you are not a SHRM member, drop me an e-mail at joe(dot)murphy at shakercg.com (you can figure that out)  I can forward the article to you.

October 5, 2010

General Powell on Promote from Within: Assess for Potential

General Colin Powell  spoke about growing talent in the Army at TaleoWorld 2010.  He asserts that the Army may be doing more than most companies when it comes to grooming and advancing leaders.  Listen to what he has to say, then click here to review a client case study and learn about CVS Caremark and their approach to growing leaders from within for their pharmacy operations.  Great minds think alike.

Assessing potential is essential for both internal promotions and external hires.  General Powell is quite clear in his observations that success in a current position is not a predictor of success in another position.  CVS Caremark set out to achieve staffing process improvement for promotions with the use of realistic job preview, assessment and assessment-based development.

Internal candidates for the role of pharmacy supervisor are able to access a four part overview of the job, its demands, rewards and required competencies.  Through a career interest prioritizing exercise, candidates receive immediate feedback about how their aspirations and interests align with the supervisor role. 

Initiative to come forward after the feedback is left to the individual.  A tailored development plan is created and the individual is invited to complete the Pharmacy Supervisor Virtual Job Tryout to further educate the candidate and evaluate job-fit.  This multi-method assessment lets the candidate take the job for a test drive, completing a series of day-in-the-life exercises.  While in-house validation of the assessment was a critical performance requirement, it was considered table stakes.  The differentiator CVS Caremark was looking for was an assessment that delivered a candidate experience as unique as their brand and as challenging as the job.  They wanted to make better decisions using better data.

Just like General Powell and the Army, the Leadership Development Team at CVS Caremark acknowledges the challenge of making well considered promotion decisions. The combined use of interactive realistic job preview and Virtual Job Tryout has elevated the quality of hire into leadership roles.

September 30, 2010

Social Media and Quality of Candidate | Candidate Competencies Vary by Source (Part 2)

A few months ago, I posted a blog on social media and quality of candidate. In the post, I suggested that we need to use HR analytics to evaluate this source of candidates not only by the volume of candidates generated but also by the quality of candidates produced.  We conducted some preliminary analysis using assessment scores from the client’s Virtual Job Tryout and candidate conversion rate (what percentage of candidates that actually hired from a source) as quality of candidate measures.  Results were somewhat mixed, but suggested that social media was generating a quality of candidate that was less than other sources used by the organization (e.g., referrals, job boards, etc.).

Candidate hiring rate varies by social media source

Well, we dug a little deeper into this data and a very interesting picture emerged.  When we looked at the data by the various social media sites used by recruiters, two surfaced as being particularly effective:  LinkedIn and Facebook.   Candidates sourced via LinkedIn performed much better on the pre-employment assessment than candidates sourced through other channels.  In addition, these candidates were hired at a higher rate than the typical candidate.  This pattern held true for Facebook as well, but the results were not as impressive.

Candidate quality varies by social media source

We also compared pre-employment assessment results for candidates surfaced from LinkedIn versus Facebook and found some differences that at first glance seem to make sense.  Candidates sourced through LinkedIn performed better on professionally oriented competencies such as Leads Courageously, Develops Others, and Achieves Results.  Conversely, candidates sourced via Facebook performed better on more socially oriented competencies such as Customer Focus and Works Well with Others.  Source can impact quality of hire.

While we have only scratched the surface here, these results from detailed HR analytics show that there is great promise and potential value to evaluating social media, as well as other recruiting sources, on the quality of its yield.  Further, the data suggests that different social media channels generate different types of candidates with unique competencies and characteristics.  Recruiters can use this kind of information to drive more strategic sourcing efforts by placing their bets on the channels that are best aligned with the type of candidate they’re looking to source.

Part 1

September 29, 2010

Intuition or Intelligence: How Do You Hire?

Talent Intelligence was a big theme at TaleoWorld 2010.  Taleo CEO Michael Gregoire, in his opening remarks stated 47% of new placements into management positions fail.  I am not sure where that statistic came from, but it does not speak well about how companies are making decisions to hire or promote individuals.  It makes me ask:  Is the hiring decision based upon intuition or intelligence?

Where else in business would a 47% failure rate be tolerated?  What Mr. Gregoire is referring to here is one form of staffing waste.  This abysmal success rate seems to indicate a strong need for talent intelligence.  Better candidate data for making more accurate hiring decisions.

Getting useful, meaningful data is the central challenge.  Hiring managers and recruiters, while well intended, often place disproportionally high value on candidate data that is either not related to job performance or even worse, negatively related to job performance.  And, one of the more common areas where we see this is the value placed on specific job experiences that while intuitively seemed to make sense, the evidence from HR analytics proved othewise.  Here are a few examples. Previous cash handling experience negatively related to cash drawer accuracy, prior food service and hospitality experience negatively related to success in a food and beverage management position, previous sales experience with a competitor negatively related to sales success.

Thoughtful people in successful companies establish these screening criteria.  However, in the majority of cases these criteria are assumptions.  Assumptions that are never tested or proven.  By not conducting the appropriate HR analytics, decisions get made based upon ego, not evidence.  This is allowed because of the common and accepted assertion from recruiters and hiring managers: “I am a good judge of talent.”  With a 47% failure rate, it would seem prudent to do some analysis.  Just better than a coin toss does not seem like good odds for a critical and expensive business decision.

Employee selection is a process.  The yield of the process can be measured and improved. Candidate evaluation with pre-employment assessments can be conducted in a manner that produces evidence in the form of data. This data can support HR analytics which in turn provides guidance to improve the objectivity and effectiveness of the hiring decision.  If you are a Taleo user and want to make your ACE work better, we can help.

Check out a few of our case studies to see how HR analytics and pre-employment testing validation analysis have made a measurable difference in the yield of a business process called staffing.  We can help you make the transition from ego to evidence, from talent intuition to talent intelligence.

RSS Feed LinkedIn