Hiring and Pre-Employment Testing.

 

Suppose your organization creates a new position, and you are tasked with choosing the right pre-employment test to identify the best new talent. You start your search by googling “pre-employment test” which yields about 195,000 results (or at least it did for me).

The good news is that you’ll have lots to choose from. The bad news is that pinpointing the right pre-employment tools for your specific hiring needs can be baffling.

There are personality tests, skills tests, cognitive ability tests, biodata tests, workstyle tests, and integrity tests. There are graphology (handwriting) tests, how you behave at a party tests, and there’s even a new smartphone test of how quickly you can inflate party balloons without bursting them.

With so many options, how do you decide which test is best?

If you follow a few key guidelines, it’s actually easier than it sounds. And I’ll walk you through each step.

HOW TO EVALUATE PRE-EMPLOYMENT TESTS

Step 1:  Define What You Want to Measure

As we previewed in my last blog, the first step in finding the right pre-employment test is to define what your organization wants to measure and the outcomes you want to achieve. The U.S. Department of Labor’s Testing and Assessment: An Employer’s Guide to Good Practices advises you to start by determining the following: 

Purpose: What do you need to measure? Examples include skills, abilities, and values that individuals must possess in order to perform well in the target job.

Prediction: What do you need to predict? Pre-employment tests are designed to predict a number of critical success factors, including job performance, productivity, training needs, turnover risk, leadership potential, etc.

Format: What is the best way to measure the desired skills and attributes? Types of testing formats include paper-and-pencil, online survey, work sample, and simulation, among others.

Step 2:  Evaluate Tests Based on The Three Essentials 

This step is the most critical. All pre-employment tests are subject to professional and legal standards. Fortunately, the U.S. Department of Labor offers guidelines for understanding what’s most important. To be specific, all good pre-employment tests must have three essential elements.

  • Validity (the test is an accurate predictor)
  • Reliability (test results are consistent and dependable)
  • Compliance (the test is fair and meets legal guidelines)

To ensure that a pre-employment test meets the three essentials for legal defensibility, ask these questions:

  • Is the test relevant to the job?
  • Are the results valid and reliable?
  • Is the test fair? Does it meet or exceed legal compliance guidelines?
  • Does the test minimize your risk of adverse impact against protected classes?

A pre-employment test must deliver on each of the three essentials:  validity, reliability, and compliance. If it doesn’t, cross it off your list. It’s that simple.

Step 3. Compare Tests Based on Practical Considerations 

Once you’re confident that a test predicts the key success factors you want to measure, plus the test offers strong evidence of validity, reliability, and compliance, then it’s time to look at practical considerations.

Cost and Return on Investment (ROI): Test costs vary widely based on factors such as test type, complexity of the job, and hiring volume. The time and training required to administer the test must also be considered. Look for tests that provide strong ROI in both cost savings and performance gains.

Efficiency: With internal resource costs high, and application abandonment rates as high as 60 percent or more, be sure your screening process is convenient and user-friendly for both administrators and applicants. 

Applicant Experience: Applicants prefer pre-employment tests that are engaging, clear, and job relevant. “Face validity” is key. For instance, a keyboarding test for a keyboarding job has high face validity. Conversely, (unless you’re applying for a job that involves planning or attending parties) the balloon inflation test, or tests that query party behavior will likely fall short on the face validity scale. 

We’ll look at practical considerations in more detail in Part 3 of this series. 

TYPES OF PRE-EMPLOYMENT TESTS 

While the number and types of pre-employment tests can seem overwhelming, the U.S. Office of Personnel Management (OPM) offers easy-to-follow guidelines for understanding and evaluating the most common test types.

Based on the OPM’s independent and well-researched review, the following list summarizes some of the most commonly used test types, along with the advantages and disadvantages of each.

Job Knowledge Tests measure how much an applicant knows about specific job information or tasks. These tests are usually in pencil-and-paper or computerized formats. Typically consisting of multiple choice questions or essay items, job knowledge tests are designed to evaluate the technical or professional expertise and knowledge required for specific jobs or professions.  One example is a police promotional exam that tests knowledge of laws and statutes.

Advantages

High validity, face validity

Relevant to particular aspects of job

Easy and inexpensive to administer

Best for jobs requiring immediate specific job knowledge

Good ROI for applicants who possess technical expertise in specific areas

Disadvantages

Risk of adverse impact

Expensive and time-consuming to develop and update

Poorer ROI for some jobs in which other tests are better predictors

 

Cognitive Ability Tests measure abilities involved in the mental processes necessary to solve work-related problems or to acquire new job knowledge. These tests consist of questions designed to estimate an applicant’s potential to use logic, reasoning, reading comprehension, problem-solving, and other mental abilities that are fundamental to success on the job.

Advantages

High validity

Easy and inexpensive to administer

Best for complex jobs

Inexpensive when purchased off-the-shelf

Good ROI for applicants who possess technical expertise in specific job knowledge areas

Disadvantages

Risk of adverse impact

Poorer ROI for some jobs in which other tests are a better predictor

 

Biographical Data Tests are based on the principle that past behavior predicts future behavior. These standardized questionnaires measure education, training, work experience, and interests in order to predict success on the job. Some biographical data instruments also ask about an applicant’s attitudes, preferences, and assess personality.

Advantages

Moderate validity

Easy and inexpensive to administer

Relevant to particular aspects of a job

Inexpensive when purchased off-the-shelf

Good ROI for top performers

Disadvantages

Risk of adverse impact

Low to moderate face validity

Costly to develop

Privacy concerns among test-takers

Subject to candidate embellishment and faking

 

Personality Tests measure motivation, teamwork, preferences, interests, emotional make-up, and style of interacting with people and performing in situations. They can be especially useful where job performance requires a great deal of interpersonal interaction or working in teams. The personality traits most frequently assessed in these tests include extroversion, emotional stability, agreeableness, conscientiousness, and openness to experience. These five traits are often referred to as the Five-Factor Model.

Advantages

High validity

Easy and inexpensive to administer

Low risk of adverse impact

Inexpensive when purchased off-the-shelf

Good ROI for critical competencies upon entry into the job

 

Disadvantages

Low face validity

Should be used with other assessments

Applicants have concerns about privacy and invasiveness

Subject to faking

Caution must be taken to avoid assessing medical (mental health) conditions

 

Simulations and Work Sample Tests place applicants in simulated work situations that are the same or similar to those performed on the job in order to assess their level of skill or competence. For example, work simulations might replicate interactions with customers, decision-making, or using data to solve problems. Simulations and work samples by design are highly job relevant and offer the highest predictive accuracy.

Advantages

High validity

High face validity

Low risk of adverse impact

Provides realistic preview of the job and the organization

Reduces turnover

Good ROI for critical competencies upon entry into the job

Disadvantages

If not automated, work sample administration can be time-consuming and may require training

Can be difficult, time-consuming, and expensive to develop in-house

 

In addition to its summary of test types, the U. S. Office of Personnel Management ranks how each type compares in terms of validity, face validity, and risk of adverse impact.

Check out this chart based on data from the OPM, documenting the most accurate test type – and revealing the tests that place you at the greatest risk.

EMERGING TECHNOLOGIES

As pre-employment testing continues to evolve, expect to see upgrades in delivery methods, user-interfaces, and predictive analytics. AI (artificial intelligence) applications are all the buzz, offering employers new ways to collect and interpret applicant data. While the potential for faster and sleeker ways to gather job-relevant data is positive, some AI applications are opening up new questions related to privacy, EEO compliance, and ADA compliance. (More about AI testing in a future blog.)

That’s why it’s more important than ever to follow our three steps for evaluating pre-employment tests, with a focus on The Three Essentials of validity, reliability, and compliance. Whether you’re evaluating a traditional test, or one that just hit the market, these guidelines will help you cut through the uncertainty and marketing hype so you can choose the right pre-employment tools with confidence.

In an upcoming post, we’ll conclude our 3-part series on pre-employment testing with specific tips and strategies for matching the right pre-employment tools to your specific hiring needs. You’ll also get an easy-to-use checklist to guide you!

Maurie Jones is an Industrial/Organizational Psychology consultant from Orlando, Florida. She has spent more than 30 years in the industry designing, developing, and administering employment and promotion tests and systems. Her clients have included Fortune 500 organizations, financial institutions, law enforcement and fire agencies, and many others.

 

RESOURCES

A Helpful Guide to Pre-Employment Testing – Part I:  The Three Essentials

U.S. Department of Labor, Testing and Assessment: An Employer’s Guide to Good Practices

 U.S. Office of Personnel Management, Assessment Decision Guide

 Uniform Guidelines on Employee Selection Procedures