Validity of a Psychometric Test

Anamika Singh
07 May 2018
|    6 min read    |
Content Marketing

Psychometric tests are a standard and statistical way of evaluating your mental abilities and behaviour. Term Psychometry means measure of mind. Psychometric tests are also called as personality tests or Career Aptitude tests though there is difference between all of the three. If you are interested to know more about the actual difference and where to use each of these three Psychometric, Personality and Career Aptitude Tests you can find it here. Unlike technical assessments or aptitude tests that measure competencies or cognitive ability, Personality tests are more inclined to measure a person’s behavioural dimensions. The shift from Human Resources (HR) to Talent Management (TM), introduced new needs in Talent Acquisition like below

  • How suitable a candidate is for this role?
  • Will the candidate’s behaviour be aligned to organisation values?
  • How suitable is he/she for grooming to a leadership role in future.
  • Can we describe an employee or potential employee?

To address the above, Psychometric tests are adapted widely in modern Talent Management & Talent Acquisition phases. It is almost impossible to ascertain the traits/behaviour in face to face Interviews. Various online assessments measure Skill and Knowledge. In recent past, on demand video interviews gained a lot of popularity and many companies use it to replace time consuming telephonic interviews. It is also important to know why you should start using on-demand video interviews for your hiring. Still these tests don’t act as a career aptitude test. Psychometric tests are dealing with one of the most sensitive areas of a human which brings a lot of legal concerns regarding test’s soundness for usage in these areas. Claims are made that these tests are conducted for the audience under same circumstances, like same questions and same time which is not sufficient to prove the legitimacy of a test. This is where critical components Validity & Reliability of a test come into picture.

In this article we want to explain about the measures taken to prove the test validity. The things mentioned below are not carved in stone, every assessment provider uses its own method to demonstrate validity.

What is Validity in Psychometric tests ?

Validity is the extent to which a concept, conclusion or measurement is well-founded and corresponds accurately to the real world.
Test validity is a measure of three things

  • Construct Validity
  • Criterion Validity
  • Content Validity

1. ​​Construct Validity

Construct validity defines how well a test or experiment measures up to its claims. It refers to whether the operational definition of a variable actually reflects the true theoretical meaning of a concept.

Ex: A doctor testing the effectiveness of painkillers on chronic back sufferers. Every day, he asks the test subjects to rate their pain level on a scale of one to ten - pain exists, we all know that, but it has to be measured subjectively. In this case, construct validity would test whether the doctor actually was measuring pain and not numbness, discomfort, anxiety or any other factor.

Therefore, with the definition of a construct properly defined, we can look at construct ability, a measure of how well the test measures the construct. It is a tool that allows researchers to perform a systematic analysis of how well designed their research is!

Statistical Test Accepted Standard
Exploratory Factor Analysis (EFA) Eigenvalues > 1.0
Confirmatory Factor Analysis (CFA) Factor Loadings > 0.40

2. Criterion Validity

Criterion or concrete validity is the extent to which a measure is related to an outcome. Criterion validity is often divided into concurrent and predictive validity.

Concurrent Validity:

If the outcome of a test when compared to other subjective results or a proven measurement at present is highly correlated at a statistically significant level,, then concurrent validity exists.

Ex: You prepare a test for assessing student’s attitude. This is test is for 100 point scale. Now you go to the student’s lecturer and ask him to rate this student’s attitude from 1 to 10 point scale. Then we compare the results. If the results are in an acceptable range, we state the test passed concurrent validity.

Statistical Test Accepted Standard
High Pearson Product Moment Correlation Substantial & High r>=0.45

Predictive Validity:

Main aim of any test especially used in hiring is to predict a candidate’s behavior or suitability to the role he/she will be hired for. So predictive validity is an important element and must be satisfied at all times.

Ex: You ask the candidate to take a test for his “Learning Ability”. The test is on a 100 point scale. After the test is completed over a period of time, you continue to take the feedback of this candidate’s learning ability from his/her mentor and correlate the results to the test results. If the results are in an acceptable range, we state the test is admissible.

Statistical Test Accepted Standard
High Pearson Product Moment Correlation Substantial & high r >= .45

3. Content Validity

In this a test’s content is validated to check if it is actually questioning on the attributes related to what the test is intended to measure.

Ex: Online Assessment for Sales Manager role with a question like “You are a great runner, you run for miles a day without getting tired”

a)Completely Disagree b) Disagree c) Neutral d) Agree e)Completely Agree makes no sense rather questions like “I make friends easily”, “I am optimistic” are more apt.

Statistical Test Accepted Standard
Content Validity Ratio/Index (CVR/I) Depends on number of expert reviewers

Face Validity:

Face validity is all about look and feel of the test, how comfortable the test audience are while taking the test and a lot of other things that don’t add up to a candidate not doing the test well.

Ex: When you are giving an equation of three lines, you don’t want to put two lines in one page and the other one in next page which is confusing for the test taker.

However Face Validity and Content Validity are two less prominent concepts of validity when compared to others. The reason might be because these are the basic things that come with tests these days and with most of the assessments being online Face validity is an issue that is inherently addressed.

No Statistical Tests are used to measure Face validity Since we are discussing about construction of psychometric tests and how validity plays an important role, another sub-factor which also plays an important role is Preventing, Detecting & Correcting faking exams

Preventing Faking in Psychometric Tests

  • By explicitly asking them not to falsify in the tests.
  • Making the tests empirical criterion keyed
  • It is hard to administer the empirical criterion keying questions through a pen and paper tests. The best mode to conduct a personality tests is via Online Assessments.

Detecting Faking in Personality Tests:

Social desirability - These are detected by making a note of overly favourable answers.
Detecting Consistency - Sometimes the individual will just go on a spree of selecting random answers without even paying attention to the questions, a few questions imbibed in the test will check for consistency and gives a seriousness score for the candidate. This will be the first level of check even before results of the test are considered valid.

Correction of Faking in Behavioural Tests:

  • Once its identified that the candidate’s responses are not genuine, the candidate should be offered a chance to retake the test or notify the test administrator to adapt a different source of assessment for this candidate.
  • A few questions where over favourable answers are given, adjusted scores can be given.

A test complying to the above can be deemed satisfactory from a Validity standpoint. Another important criteria in soundness of a personality test is Reliability. How is Reliability Measured, and What is Reliability in a Personality Test can be read here.

One closing suggestion for all the audience who are creating and administering the personality test is to be cautioned about Social desirability factor. There are limitations for personality test creators on the usage of wording, straightness of the question and minimising the connotation for possibility of diverse answers that makes the psychometric tests slightly vulnerable to social desirability. Ignoring this minor snag, still Psychometric/Personality tests are considered the industry standard in identifying the behavioural dimensions and traits in a person.

We work closely with you and carry out research to understand your needs and wishes.

Start to Bring a change in

your Hiring Process with eyardstick