Quality of Hire Blog

arrowBack to Quality of Hire Blog

Dissertating and Assessement Development - Similar Scientific Methods

by Lei Qin

I am Lei Qin, (sounds like: ‘chin’) the newest member of the Shaker Consulting Group team. 

I have recently completed the defense of my dissertation. It was a mix of feelings: relief and anxiety. Why was it a relief? I don’t need to make more endless revisions. This is really a relief.  But as the defense date approached, anxiety accumulated, slowly but robustly.  How did I deal with it? I took a simple strategy: not thinking about it too much until it comes.

I am not blogging to just expose my feelings about dissertating. My experience as an intern with Shaker provided a window of time in which building employee selection tools and dissertating overlaped in my life. This leads me to an interesting observation. A similar scientific process is shared by building a pre-employment assessment and dissertating.

 The first step of building an assessment tool for hiring is job analysis. The primary purpose of job analysis is to understand the targeted position and find out what knowledge, skills and abilities and other variables are essential for superior performance in the job. A competency model of the position is developed based on the job analysis. The first step of dissertating is a literature review and proposal. This step helps to define and understand the research progress of a specific area. After thoughtful consideration you can make a hypothesis about what is missing, unanswered and critical to advance this research area. A research proposal is developed based on the findings of literature review.  In my case, I chose Trust in Leadership as the focus for my research.

The second step of building a hiring assessment tool is content development. A story board, measurement hypothesis, item content, and computerized deployment system are developed based on the competency model and the job analysis results. The developers need to draft or choose appropriate testing components to assemble the pre-employment test. Similarly, the second step of dissertating is research design and material development. A researcher needs to choose appropriate designs for the study and develop study materials such as instructions and survey items. For my research I created three experimental conditions to activate different mindsets in anticipation of influencing down-stream thinking and responses.

The third step of building a assessment tools for hiring is validation analysis. Typically hundreds of participants are invited to take the selection tool. Predictor (test results) and criterion (job performance) data are collected. HR analytical tools are used to explore the relationship among the response patterns on the test and on-the-job performance. This type of HR analytics determines the power of the test to predict on-the-job performance. The third step of dissertating is data collection. Subjects will participate in the study. Data of independent variables and dependent variables are collected. Statistical analysis is conducted to test the hypotheses. My research explored the differences in how concrete versus generalized knowledge about an individual impacts how we establish trust with them.

The final stage of building an employee selection tool is rollout.  After a thorough review of the results, the selection system needs approval from the selection practice leaders prior to rollout  The pre-employment assessment system will be tested in the real recruiting environment. Similarly, the final step of dissertating is defense. The success of a dissertation comes from the approval from a dissertation committee, prior to rolling out a new Ph.D.  Well, I did it! My research was determined to contribute to the body of knowledge on leadership. And now I have been rolled out with a Ph.D. and Shaker has hired me to continue my work and research in the real recruiting environment.

The comparison of building a selection tool and dissertating demonstrated they follow the same rigorous scientific process. At Shaker, the development of Virtual Job Tryout® has one more component; that is, managing a feedback loop which makes the Virtual Job Tryout® adaptive to the real world, getting more accurate.  Data will be collected over time and periodic validation analysis will be conducted to improve the predictive power and document the return on investment (ROI) of the pre-employment assessment 

Earning my Ph.D. has been just one step along my path of continuous learning from experience.  And just like an assessment can get smarter over time, my skills and ability to contribute to a body of knowledge will be tested time and time again.