Monday, 8 October 2012

IQ, Personality, and understanding test design


We need to talk statistics. No, seriously, you'll thank me. You'll get a handle on Item Response Theory (IRT), something pretty crucial to occupational assessment, and be able to appreciate the important study we'll go on to discuss. It'll be fine...

If you've taken a modern occupational test, IRT was probably sitting beneath the bonnet making sense of the responses. Traditional tests count correct responses to give an estimate of your true ability: 30/40 means you ought to be better than if you'd scored 23/40. In contrast, IRT moves the unit of meaning from the test to the test item. Getting one question right or wrong gives us some predictor of ability right off the cuff; a coarse one, admittedly, but increasingly accurate as further responses are given.

Let's say Item E is easy. Someone at or above average should get it, and those below average have a fighting chance. Item H is hard: the chances of a correct answer are low for most, but the chances rocket up for the sharpest. Each item has a different relationship between test-taker's ability and the likelihood of them getting it right: these are the test parameters. (Because I love my readers, I've bodged up a visual example). You don't need to understand the maths to appreciate that armed with these parameters, it quickly becomes possible to home in on the true performance behind the item responses. Potent stuff.

As well as powering the tests, IRT offers an investigative methodology for the following problem: if two populations differ in test performance, does this reflect genuine difference or simply artefacts of how those populations approach the test? Well, if parameters are similar for both groups - the verbally sharp and weak Montagues have the same pattern on items E and H as do their Capulet counterparts - then the items are functioning in the same way, making us more confident the differences are real. If not, we should start to wonder if the test is being contaminated by something else - perhaps Capulets get stressed and guess blindly to items that look tricky, even ones they ought to have gotten right on account of their raw ability.

Put Verona aside. The real issue investigated by Chakadee Waiyavutti, Wendy Johnson, and Ian Deary is whether individuals with low IQ respond to personality tests differently. Personality? Yep, IRT is used for these assessments too, in a slightly fiddlier way - item 'difficulty' and right/wrong binaries need to be translated - with the concepts remaining solid. Higher and lower IQ groups do show slight personality differences in aggregate. If these differences were because personality items were understood differently by these different groups, it would call into question the validity of making judgements about personality when testing across ranges of IQ, which would impact occupational testing in a profound way.

Waiyavutti's team drew on a large data set of 683 individuals born in 1936, categorised into two groups with a mean IQ difference of 21 points. Participants completed two personality tests, the NEO-FFI and IPIP (both based on the Big 5 personality factors) and the researchers produced parameters for each item in each group, and analysed whether averaged parameters across the groups were significantly different. They found that while the two groups did differ on average - in expected areas such as Intellect and Openness to Experience and Emotional Stability - the personality test items operated similarly. This gives reassurance that these are meaningful differences.

So: we can be more confident that personality tests (at least these) are operating in the same way in people of differing IQ, making it reasonable to use them to draw their intended insights. And along the way we've figured out something about how modern tests operate. If you want a fuller exploration of IRT, you may be interested in this open-access article in the Psychologist online.


ResearchBlogging.orgWaiyavutti C, Johnson W, & Deary IJ (2012). Do personality scale items function differently in people with high and low IQ? Psychological assessment, 24 (3), 545-55 PMID: 22082036

15 comments:

  1. We do get to share in the celebrations and the progress of your child. More importantly, we get to know, appreciate and love your child’s personality, resilience and joie de vivre. personality tests

    ReplyDelete
  2. Thanks for Informative blog, it is pretty good and impressed me a lot.If you buy online medicine to like Buy Ambien 5 mg Online than Visit us.

    ReplyDelete
  3. Great Information...
    Thanks For sharing with us..

    If want to buy Tamiflu online or Tamiflu Cash on Deliery, you can visit myanxietypills, here we deliver many types of online medicines at your Doorstep with full satisfaction.

    Buy Propecia Online
    Buy Modalert Online
    buy tapentadol 100mg online
    Buy Tramadol 100mg Online

    ReplyDelete
  4. Amazing Blog!
    Thanks for sharing such Information with us..

    If you want mobile app development services in India? you can visit mobileappcoder, we are the top rated Mobile App Development Company across India with full satisfaction services. Various services provided by us are:

    Android App Development Services in India
    iPhone App Development Services in India
    React Native App Development Services in India
    Hire Android Developers in India
    Hire iOS app developers in India

    ReplyDelete
  5. Great Information, Thanks for sharing with us...

    if you want to buy online medicines then visit superpharmacyusa.com, here we deliver best online medicines in USA at your doorstep with full satisfaction..

    Buy Artvigil Online
    Buy Modafinil Online
    Buy Modalert online
    Buy Propecia Online
    Buy Tamiflu Online
    Buy Tapentadol Online
    Buy Tramadol Online

    ReplyDelete
  6. Thanks for sharing wonderful information with us...
    Good Worrk! Keep Going.

    If you are looking for web development services? Drudge Media, is top leading web development company across the India. We provide various services:

    WordPress Development Services in India
    Shopify Development Services in India
    drupal development services in india
    mobile app development Services in India

    ReplyDelete
  7. Wondering how to buy clonazepam online? Get Clonazepam online cash on delivery without any prescription from the best online pharma store in USA. To know more detail visit our website.

    ReplyDelete
  8. Great Article...
    Thanks For Sharing With Us!!

    My reputation Vault is top leading ORM Company in India offering best Online Reputation Management Services in India for your Online business Growth.

    remove ripoff report from google
    pissed off consumers
    cheaterland removal

    ReplyDelete
  9. Very Nice Article!!
    Keep it up..

    If you want to buy online Meds then visit us, here you can buy tramadol 100mg online or cash on delivery. We deliver best quality of online medicines at your Doorstep.

    Buy Tapentadol 100mg Online

    ReplyDelete
  10. Great Information, Thanks for sharing with us!

    If you want to remove ripoff report from google then visit myreputationvault, here we provide thw ORM Services to your Business at an affordable price. . Report removal solutions are part of online reputation management, which itself is part of digital marketing services.

    Image Removal services on Google

    ReplyDelete
  11. Very Unique Article!
    Thank You for Sharing This...

    Looking to buy Zopiclone 7.5mg Online or cash on delivery then visit us. We deliver best quality of online Medicines across the USA at your doorstep with full satisfaction.

    Order Zopiclone Online
    Buy Zopiclone Online
    Buy Zopiclone 7.5mg Online
    Zopiclone Online
    Zopiclone
    Buy Zopiclone
    Zopiclone 7.5mg
    Order Zopiclone Online

    ReplyDelete