Shaker Consulting Group Logo Virtual Job Tryout Logo

Posts Tagged ‘SIOP’

May 11, 2012

CSI: Recruiting – Candidate Science Investigation

The popular TV show CSI has created a fascination with the science behind crime scene investigation.  It has raised interest and awareness to the science of forensics, or as Webster defines it: the use of science and technology to investigate and establish facts.  In law, decisions should be supported with evidence.

Forensics for Recruiting

The result of this TV series has been an explosion in enrollment in criminal justice and law enforcement related programs at colleges and technical schools across the country.  The outcome will be a flood of graduates imbued with knowledge and skills, hopeful about being hired by leading edge crime fighting police departments who want more science in their prosecution.  In recruiting, hiring decisions should be based upon evidence supported with sound data collection and analysis too.

A similar science-based approach for establishing decision support evidence is available for recruiters: Industrial/Organizational Psychology (I/O Psych).  These professionals are the CSI: Recruiting specialists – Candidate Science Investigators

This professional discipline was established in the early 20th century, to apply psychological principles and techniques to business and industrial problems, as in the selection of personnel.  Before crime investigator forensics hit prime time TV, forensic specialists worked quietly behind the scenes improving the quality of data collected and used to build a case.  Industrial/Organizational Psychologists (I/O Psych) are quietly at work around the world, building data collection methods that improve the quality of  hire  with selection science.  These professionals design the data collection methods called assessment.

Every law department has someone trained in forensics because they know better evidence improves the decision in our justice process.  Will it take a fast paced TV series before companies sit up and take notice of the work being done by CSI Recruiting?

Doctorate degrees have been offered in I/O Psych for over 80 years.  These graduates have been snapped up by leading edge companies who understand the competitive advantage of more science in their recruiting process.  The evidence of I/O Psych’s contribution is compelling.

Here are a few simple examples from our work with selection science and HR Analytics that shed a bit of light on to the potential of better candidate data.

A retail operation with thousands of stores had been using a candidate screening criteria of “years of experience in a similar industry.”  Intuitively, every one thought it had made sense.  However, during a recent study by a team of I/O Psychologists, evidence determined that the longer a candidate had been in the similar industry, the less likely they were to be an above average contributor at this firm.

Similarly, in a capital equipment field sales representative position, the company had established screening criteria of hiring people who had worked for the major competitors.  After an investigation by a team of I/O Psychologists, the evidence demonstrated the longer a candidate had worked for the competition, the less likely they were to be an above average performer.

Ironically, “related experience” is often a candidate screening criteria.  In both cases, using that factor places positive weight on a evaluation criteria with negative value.  The CSI Recuiting teams had been chasing bad leads, intuitive assumptions, proved wrong by evidence.

In both cases, the evidence collection process was dramatically improved across a broad range of factors that contributed to a better quality of hire through CSI.

More examples can be found on our case study page.

Crime scene investigation is typically more like making one high stakes hiring decision.  The consequence of the decision can be significant.  The approach used to leverage I/O psychology varies depending on the scale of the staffing process.  A once every few years hiring decision requires a different solution than staffing processes which make hundreds or even thousands of hiring decisions every year.

If your company has a job or job family with over 100 employees in it, I/O psychology can begin to add measurable impact on performance with each hiring decision.  If you have a job with thousands of employees engaged in fundamentally the same work, it could be the basis for a charge of Recruiter Negligence for not engaging an I/O psychologist.  In high volume hiring processes, the size of the data set, the frequency of decision making, and the potential for significant performance variation almost mandated Candidate Science Investigation.  Click here to review some criteria to consider to determine if CSI is appropriate for your recruiting situation.

The decision to hire will always be an act of personal judgment.  However, every executive knows a decision is only as good at the data behind it.  There is a great opportunity for staffing process owners to do real CSI: Candidate Science Investigation.  Without the use of selection science, data collection and analysis, recruiting could be activity without insight.

Watch our movie to learn more.

April 26, 2012

Simulations Plus Other Assessment Modalities: Is the Whole Greater Than the Sum of the Parts?

SIOP 2012 San Diego CA

Three Shaker Consulting Group psychologists discuss the power of combining simulations with other assessment data for more predictive power.

Throughout the history of psychometric assessment, researchers have focused on discrete measurement modalities such as multiple choice biographical data, scale-based personality, and various types of mental ability measures. Researchers and practitioners have generally studied each modality individually, and attempted to show that individual constructs have a degree of reliability and correlate significantly with some outcome measure. As sample sizes and, correspondingly, statistical computing power have grown, researchers have also been able to examine complex effects such as interactions between constructs, though even these analyses are most often done within a particular assessment modality (e.g., examining the interaction between two personality scales).

Society for Industrial Organizational Psychology

Now, as computing and networking technologies continue to evolve at an exponential rate, nearly any type of assessment modality imaginable is possible to study and implement (Greene, 2011; Trull, 2007). As a result, novel assessment techniques are being developed faster than ever before. Today, simulations are at the forefront of modern assessment design and, as work samples, are known to be quite predictive of job performance (Schmidt & Hunter, 1998). Yet, as a category they are quite heterogeneous.

This presentation will contribute to the growing literature in this area by examining two aspects of simulation design:

1. Whether measuring respondent behaviors during the simulation rather than just responses to embedded questions might confer a psychometric advantage.
2. How simulation measures may interact with other types of assessment measures, to better predict various outcomes.

Ultimately, as computerized simulation of the real world increases in complexity and fidelity, our science can move from creating measures of theoretical constructs to measuring actual behavior. In this manner, in time we may begin to transcend the need to measure proxy variables and theoretical constructs, and instead, use more direct measures of behavioral outcomes.

Simulations offer us a venue to ask many new questions that were not feasible and/or informative before. What can we learn, for example, by observing how a person interacts with an environment, be it virtual or real? For example:

• What do errant mouse clicks tell us?
• What do we learn when a respondent changes her answer?
• Can we learn something about a candidate’s personality by whether she skips over questions she doesn’t know, or spends extra time trying to get each question correct?
• Does repeating an example item during simulation instructions tell us anything about an individual’s personality?

In short, by simulating environments, psychologists can expand the measurement space from the traditional and simple question and answer, to a broad range of non-question-based behavioral measures. As simulations grow in complexity and fidelity, we may be able to move away from actual questions, and more towards measuring how an individual actually interacts with a particular environment; in other words, measure that individual’s choices and behaviors (Hornke & Kersting, 2006). At that point, our measurement focus will be on anything and everything that can be measured, from how a person moves through a space to what they actually do after making a decision.

While one simulation-based measurement advance is thus a gradual moving away from construct measurement to direct behavior measurement, another advance occurs as practitioners combine simulations with other assessment types. This allows researchers to measure interaction and other complex effects across measurement modalities. For example, we may find that a person with a high score on a multitasking simulation is better at a certain job, but only if that person also has a high score on attention to detail. Or, perhaps we will find that multitasking scores are only predictive for people who have a background doing work that involves multitasking. This type of cross-modality research is in its’ infancy, but it should increase as researchers integrate more measures in to online assessment experiences.

Psychological science has a hundred-year history of theory and research, but we feel that it has barely begun to tap its’ potential to explain and predict human behavior. Traditionally, researchers in our field focus on finding that elusive, “holy grail” measurement construct that can predict uncharted amounts of variance in job performance. We believe that this pursuit is ill-fated. Instead, researchers should explore ways to measure human characteristics with increasing fidelity, and in particular, move beyond paper-and-pencil measures of abstract constructs to actual direct behavioral measurements, especially in virtual environments. Furthermore, and especially with the vast amounts of data increasingly being captured by modern organizations, our science needs to focus less on the direct effects of measured variables, and more on complex and cross-modality effects. The latter topic represents a tremendous area of untapped opportunity for exploration.

References
Greene, R. L. (2011). Some considerations for enhancing psychological assessment. Journal of Personality Assessment, 93(3), 198-203. doi:10.1080/00223891.2011.558879
Hornke, L. F., & Kersting, M. (2006). Optimizing Quality in the Use of Web-Based and Computer-Based Testing for Personnel Selection. In D. Bartram, & R. K. Hambleton (Eds.), Computer-based testing and the Internet: Issues and advances (pp. 149-162). New York, NY: John Wiley & Sons Ltd.
Schmidt, F. L., & Hunter, J. E. (1998). The validity and utility of selection methods in personnel psychology: Practical and theoretical implications of 85 years of research findings. Psychological Bulletin, 124(2), 262-274. doi:10.1037/0033-2909.124.2.262
Trull, T. J. (2007). Expanding the aperture of psychological assessment: Introduction to the special section on innovative clinical assessment technologies and methods. Psychological Assessment, 19(1), 1-3. doi:10.1037/1040-3590.19.1.1

April 25, 2012

Pre-Employment Testing Company Opens Fourth Office

Shaker Consulting Group, an industry leader in pre-employment assessments, has expanded their footprint with a new office in Chevy Chase, MD. Shaker is the creator of Virtual Job Tryout, a custom simulation for pre-employment testing, and added its fourth office in response to robust growth of the business,

Now Serving East Coast

The growth of our east-coast client base, and the expanding demand of our global Virtual Job Tryout implementations requires a dispersed team,” said Joseph Murphy, vice president of Shaker Consulting Group.

Dr. Christie Cox, a Maryland resident, recently joined the firm and opened the new office in February. Shaker Consulting Group’s other offices are located in Atlanta, and Pittsburgh, with headquarters in Cleveland.

“The growth of Shaker Consulting Group’s international client work was very interesting to me. I will be able to immediately leverage my global pre-employment testing experience,” said Dr. Cox, an industrial organizational psychologist.

June 11, 2010

Janz on Improving the Candidate Experience

Tom Janz, Ph.D. of PeopleAssessments and I had a chance to catch up at SIOP.  We spoke about the candidate experience.  Given the amount of time people are spending on YouTube, Tom thinks the on-line application process should be fun.  Long questionnaires with hundreds of radio buttons to click are just not engaging.

Tom also suggests the experience be job relevant.  Click Play to hear what Tom had to say:

Go ahead and apply for a job at your company. Then ask for candidate testimonials: “Did I have fun?”

April 6, 2010

Shaker Consulting Group to Present and Exhibit at SIOP

The Society for Industrial and Organizational Psychology will convene for its 25th annual conference this week in Atlanta, GA.  Industrial-organizational (I-O) psychology is the scientific study of the workplace. Rigor and methods of psychology are applied to issues of critical relevance to business, including talent management, coaching, assessment, selection, training, organizational development, performance, and work-life balance.

This event brings together a diverse group of professionals with deep interest in measuring a wide range of factors from the world of work.  We at Shaker are particularly pleased to be participating in five sessions this year.  Topics covered will range from improving the candidate experience to the additional science power of multi-media in pre-employment testing to assessment design to reduce faking.

In addition to conference sessions, we are also a conference sponsor and exhibitor.  Visit us in Booth 808.

Stop by and ask for a demo of a Virtual Job Tryout.  You will see why brand conscious companies have come to expect more from assessments.

Please review the topics and times  listed below.  We would enjoy having you present to engage in the dialogue.

Cool Assessment Tools Symposium –
Marriott Vacation Club presenting on Virtual Job Tryout®
THURSDAY AFTERNOON, 4/8
12:30 – 1:50pm
Salon B

Interactive Multimedia Simulations: Criterion-Related and Incremental Validity
THURSDAY EVENING, 4/8
6:00-6:50pm
Grand Ballroom A
Top Rated Posters

Faking it well: Effects of surface acting on task performance
SATURDAY AFTERNOON, 4/10
1:30 – 2:20pm
Galleria

Leveraging Technology to Engage Candidates and Deepen Assessments
SATURDAY AFTERNOON, 4/10
2:00-2:50pm
Room 201

There’s More to Selection than Correlation Coefficients: r You Serious?
SATURDAY AFTERNOON, 4/10
3:30-4:20pm
Room 202

RSS Feed LinkedIn