How do occupational psychology lab experiments hold up in the real world?

A new paper combs meta-analytic data from across psychology to better understand how laboratory research translates to the real world. Here at the Occupational Digest, we report on evidence from a range of sources, often field work from within specific organisations (such as managers ratings of performance), or more general surveys of working individuals - but also from laboratory studies, typically involving students rather than participants with a working history. Are findings from such set-ups likely to be reliable?

The answer from Gregory Mitchell's analysis is a qualified yes. He looked at 217 lab-field comparisons drawn from 82 previous meta-analysis that covered the breadth of psychological research areas, looking at when a finding in the lab was corroborated in in the field. This follows an earlier analysis using a smaller data-set by Anderson, Lindsay and Bushman in 1999. At the highest level, Mitchell's analysis dovetailed with the Anderson et al. finding: lab and field reports correlate at an r of around .7, which means a strong relationship between the two. Although this is reassuring, the analysis also revealed that 14% of laboratory results actually changed signs when tested in field; that is, the effect ran the other way to the original finding.

Things get really interesting for the analysis by sub-field. Mitchell drew out the two fields with the most lab-field comparisons available, which were social psychology and, fortunately for us, occupational psychology. Social psychology showed a lower correlation between lab and field of .53. The occupational results, meanwhile, showed a very strong correspondence, with an r of .89. (1 would be perfect correspondence.) Only two of 71 lab effects changed signs for occupational research, with the likelihood of this happening nearly tenfold greater in social psychology. The data suggests that one reason for this is a much higher proportion of small effect sizes reported in the social psych laboratory literature; a smaller effect can much more feasibly 'flip' than a large one, all else being equal.

All told, the use of laboratory studies to inform workplace psychology appears to be in fairly good health. That said, we can't jump from this general finding to the specific confidence that a given laboratory effect will translate into the field with the same strength or even the same direction. We should continue to look for real-world investigations to follow on the heels of laboratory proof-in-concepts. What we can be comfortable with is that this strategy seems to be working pretty well in our field.

Mitchell, G. (2012). Revisiting Truth or Triviality: The External Validity of Research in the Psychological Laboratory Perspectives on Psychological Science, 7 (2), 109-117 DOI: 10.1177/1745691611432343

Elsewhere: Gregory Mitchell contributes to the debate on replication of psychology findings in this months Psychologist magazine

How 'who you could have been' could shape your workplace identity

Carter is at a formal drinks for a colleague back from secondment, part of a fast-track management scheme. he remembers opting not to apply for the scheme five years ago and wonders how things would be now had he taken that plunge: the overseas experiences, the pressures, the opportunities. What would that Carter be like? In subsequent months he finds himself returning to this idea, finally setting up a meeting with his manager, who is surprised to hear him reveal that he feels dissatisfied and wants to reinvigorate his career.

Carter has encountered an alternative self: a version of him that could have been. This concept, unpacked by Otilia Obodaru in a recent Academy of Management Review article, can be contrasted with most theories of self that work within a temporal framework - the actual past and present, extrapolating the future from an actual now. The idea of an alternative self integrates research on counterfactual thinking – 'if I had gotten that bus, I would be there by now' – into the psychology of self.

Developing an alternative self and integrating it with identity requires a few steps. First, you need a turning point, a fork in your life where you took one road over another. As the 'job for life' has given way to more boundaryless careers, there are more work-related turning points to reflect on than ever. Secondly, you must undo that turning point, imagining 'what if?', easiest to do when the event was controllable, like Carter's choice not to apply for a role. Finally, the alternative self must have opportunity and motive to be rehearsed mentally or to an audience. Identity research suggests a self-narrative tends to be taken up when relevant to ongoing desires or fears; perhaps Carter has been tiring of his fixed location and wondering if he will ever get out of the city.

Not everyone has an alternative self, the article quoting one interviewee from previous research, confessing "I'm a priest... I can't imagine not being one. I have no idea what I would do if I wasn't a priest." But many do: Obodaru cites research that reports of long-term regrets have increased fairly linearly decade on decade from around 40% of people in the 1950s to close to 100% in the last decade. Note that this measures only 'better alternative selves'; worse ones are also possible, such as those that Alcoholics Anonymous encourage their members to reflect on - the active alcoholic they chose not to be. Having an alternative self means you can compare them to your actual self, generating emotional responses, affecting satisfaction, and leading to better self-knowledge about strengths or weaknesses.

As the AA example makes clear, organisations can encourage or dampen the formation of alternative selves, by drawing attention to turning points, inviting the undoing, or giving space for rehearsing what that alternative would look like. At its best, this can lead to insight and greater resolve, such as collectively considering 'what if we had never dared to start the business together?' It can also lead to the 'crystallization of discontent' and a motivation to change circumstances. In this sense, the road not taken doesn't always vanish: it can live on in our minds, affecting our present and shaping our future.

Obodaru, O. (2012). The Self Not Taken: How Alternative Selves Develop and How They Influence Our Professional Lives The Academy of Management Review, 37 (1), 34-57 DOI: 10.5465/amr.2009.0358

The truth about nice guys, mean girls, and pay

Analyses suggest that a personality high in agreeableness is associated with lower earnings. This might seem surprising, given that agreeableness is associated with interpersonal effectiveness, increasingly important in jobs. But at least it helps explain why women experience pay inequality, given that women tend to have warm qualities; if they want to earn more, they better toughen up, right? If this seems reasonable, you'd do well to read on, and appreciate the work of a recent study that seeks to uncover more about why disagreeableness breeds pay, and why the situation for women is rather different.

Timothy Judge, Beth Livingston, and Charlice Hurst investigated the factors influencing pay using three large data sets, each containing data on between 500 and 2000 adults. Each collected personality information using slightly different measures; as each study corroborated the other, I treat this difference as a strength of the paper, as convergent evidence from multiple measures precludes the possibility that the instruments used were generating funky results. In each case, men tended to earn more than women (in one data set this was made explicit as approximately $5,000 less per year), and disagreeable people earned more. However, this mean premium was mostly due to the wage benefit that men received; for women, the premium was much slighter. Across studies, agreeableness made a big difference to male incomes, and a minor one to female ones.

So what makes the mean premium possible? It's not entirely clear - the study investigated some possible reasons such as that disagreeable people find their way into higher status or more complex jobs, but the data didn't support that conclusion. They did find that disagreeable people place more importance on pay and less on communal relationships than others, which sheds light on their priorities but not on how these are achieved. Some possibilities include agreeable people being more prepared to compromise and concede, for example on pay negotiation, or for decision-makers to falsely place warmth and competence as two ends of a continuum (rather than independent factors) and conclude that a people person may be less capable. Highly agreeable men would not only lack those edges, but, as Judge and colleagues point out, are doubly disadvantaged, as they are operating against gender stereotypes by being a soft male. Conversely, the edge that a disagreeable woman gains is blunted by their operating in ways that aren't socially sanctioned. The authors reflect that "exhortations for women not to be nice...might be overblown", and the solutions to gender pay inequality lie foremost with decision-makers.

Perhaps the causality is reversed – well-paid jobs make people less agreeable? There are a few points against this: firstly, the failure to find linkages between job type (such as status) and agreeableness. More convincingly, the investigators ran an additional, experimental study, where 480 student participants made choices in an imaginary scenario as to who they would recommend for a management fast-tract. The pair of candidates only differed according to keywords inserted in the text that speak to the quality of agreeableness, such as modest/immodest. The same pattern emerged – more recommendations for disagreeable people, with a much stronger effect for men than for women. Taken on its own, this suggests that disagreeableness is driving job outcomes, rather than the reverse.

So do nice guy finish poorly, and women last? Well, it depends what matters to you. Judge's third study found that agreeableness - and to a lesser extent, being a woman - was positively associated with life satisfaction, stronger social networks, and community involvement, and negatively associated with stress. Essentially, the disagreeable-man priorities are having exactly the impact you would expect; as the authors conclude, "if disagreeable men win the earnings war, it is a victory that may come at some cost."

Judge, T., Livingston, B., & Hurst, C. (2012). Do nice guys—and gals—really finish last? The joint effects of sex and agreeableness on income. Journal of Personality and Social Psychology, 102 (2), 390-407 DOI: 10.1037/a0026021

The 'taint' of sex shops can simultaneously repulse and compel its employees

If you worked in a sex shop, how would you handle the taint associated with that line of 'dirty work'? People employed in such occupations take various approaches, including refocusing attention away from negative features, inflating the weight of positive features, or reframing the stigmatised elements to neutralise or even valorise them.

A recent paper by Melissa Tyler suggests another response to the physical, social and moral taints associated with sex shops. Her ethnographic account focusing on shops in the renowned London district of Soho, details three months spent interviewing fourteen shop workers, together with field observations of customers and the local atmosphere.

The evidence suggested a real significance for place in how individuals framed their experiences. Appending 'Soho' to 'sex shop' charges its significance, making many of the interviewees more reluctant to share their occupation with loved ones, as they would be concerned 'not because of what I'm selling; more worried about me being in Soho'. Simultaneously, the interviews found evidence that the collection of Soho sex shops was considered a community of coping, where 'everyone looks after each other's back'. Soho is both the source of the tainted associations of the work that goes on and a resource to protect those in that work.

Tyler was particularly interested in how people might relate to this kind of work in a way best described by the concept of abjection. Abjection is that which 'beseeches, worries and fascinates' (Kristeva, 1982) at the same time, due to features that simultaneously attract and repel. One interviewee described the odder customers they encounter as a source of discomfort, yet also as a rare experience of people who they would never meet otherwise, noting that 'it's living isn't it?'. Another savoured the abnormality of the job, rather than avoiding or normalising it, explaining that 'I need to be doing something different... and I think I've captured that working here'. Another comment epitomised abjection: 'there are things about it that I absolutely hate and sometimes these are the same things that I love about it'.

Tyler suggests that future researchers may want to investigate the category of 'abject labour', where individuals are drawn to work that society considers dirty not in spite of its darker features, and not in unqualified embrace of it, but because they are taken in by its simultaneous attraction and repulsion.

Tyler, M. (2011). Tainted love: From dirty work to abject labour in Soho's sex shops Human Relations, 64 (11), 1477-1500 DOI: 10.1177/0018726711418849

Controlled study finds mind-body stress reduction techniques effective in workplace

A large randomised controlled study has found yoga and meditation techniques to be effective for stress reduction in the workplace. The study screened employees from a single company at two location to find healthy individuals who identified themselves as stressed and did not practice those techniques. This led to 239 employees who were randomly assigned to a weekly program of viniyoga practice, a similar program focused on mindfulness meditation, or to a control group who were simply given information about resources available to help with workplace stress. After 12 weeks, participants in both programs had significantly lower stress, as well as reduced difficulties in sleeping, whereas the control participants did not.

The study also measured biological features, such as heart rate variability measured post-intervention, where the participant had to imagine an upcoming stressful event and try and apply the relevant technique (mindfulness, yogic techniques such as breath control, or simply their default coping strategy if a control). Again those participants who had been through the intervention had better outcomes, in terms of heart rhythm coherence, a measure of autonomic balance linked to better functioning.

Key to these findings were the time commitments taken on by participants: the weekly commitment was in most cases just an hour, with a total time investment of 12-14 hours leading to these health effects. We've written about even more bite-sized approaches to introducing health activities into the workplace, which itself is being evaluated in a trial form. As our scientific understanding of the valuable impact of these often-ancient activities deepens, it's very welcome that we are simultaneously investigating the pragmatic concerns: understanding which strategies are viable for introducing these techniques on a large scale into a workplace.

Wolever, R., Bobinet, K., McCabe, K., Mackenzie, E., Fekete, E., Kusnick, C., & Baime, M. (2012). Effective and viable mind-body stress reduction in the workplace: A randomized controlled trial. Journal of Occupational Health Psychology, 17 (2), 246-258 DOI: 10.1037/a0027278

Is it true that our perception of telephone waiting time depends less on the actual time than other factors?

A customer's experience of dealing with a call centre on the phone can colour their attitude towards the organisation. A recent study claims that customer satisfaction with how long such calls take depends less on call time than on the quality of service we receive, suggesting that companies' focus on an 'ideal call time' may be misplaced. An interesting claim, but is it borne out by the data?

The team, writing in the journal Scientific Research, worked with a call centre to analyse data from 3013 calls: the true call duration and whether the customer felt they received (using a simple yes/no response in each case) good service, sufficient information, and, the key measure, a timely service, termed time satisfaction. An initial analysis showed time satisfaction was correlated quite highly with satisfaction with service and information, and a more modest negative correlation with call duration. The size of the effects weren't directly compared.

A follow-up analysis split the data into four groups based on actual call time; for instance the 'low' group contained calls under two minutes in duration. In every group, a 'yes' for time satisfaction was much more likely to be found alongside yeses for service and information. Meanwhile, the relationship between time satisfaction and actual time was much milder, and in the low time group the effect was too weak to be significant. The authors argue that 'with waiting times being so low, time lost its value and that satisfaction with information and service were more important'.

But wait. Let's imagine that data had been split by call time, but rather than four groups there were many; so many that a single one only contained calls lasting 10m30s to 10m31s. We'd be unsurprised if within that group (and all its counterparts), call time was irrelevant; the range of possible times is so restricted that there is no interesting difference left. The chosen analysis produces a milder version of this 'restriction of range', disproportionately reducing the chances of detecting any effects for actual call time.

The study shows that satisfaction on timely call handling is coloured by factors aside from actual call time, and it's good to remind organisations that perceptions are not formed solely by such objective features. However, this research design doesn't actually put us in a position to rank these different aspects of a call.

Garcia, D. (2012). Waiting in Vain: Managing Time and Customer Satisfaction at Call Centers Psychology, 03 (02), 213-216 DOI: 10.4236/psych.2012.32030

Too much focus on 'learning from failure' can make us unhappy

When we fail, how we feel and what we end up learning from it depends upon our coping strategy, according to new research. In particular, focusing exclusively on 'learning from failure' may make us miserable in the process.

This research explored experiences of working scientists, investigator Dean Shepherd and colleagues noting how this domain involves facing disappointing project failures, see e.g. the low rates of success for bringing drugs to market.

The researchers personally contacted employees from institutions in Germany that worked in areas such as pharmacy, zoology and ageing, with 257 scientists ultimately completing surveys consisting of standard and newly developed measures. The team were interested in outcomes from failure: positive, in the form of learning how to better run future projects or how to treat co-workers when their work is floundering, and negative emotional fallout that results in avoiding project team workers or feelings of disappointment. Both are vital, as learning creates organisational knowledge and negative emotions are associated with lower emotional commitment to the organisation – a finding observed within this study.

Learning from a failure was higher when more time had elapsed since the failure itself, suggesting time provides perspective and insight. Learning was also influenced by a respondent's coping strategy or 'orientation': those who affirmatively responded to items such as 'In my mind, I often go over the events leading up to the project's failure' are considered to have a high loss orientation, and these individuals reported higher levels of post-failure learning.

As time elapsed, however, respondents with high loss orientation swung from a low level of negative emotions to a high one, suggesting that healthy reflection gives way to unhelpful rumination. Restoration orientation, a different strategy exemplified by the item 'I keep my mind active, so it does not focus on the loss of the project', is associated with lower levels of negative emotion, but doesn't provide the learning boost provided by a preoccupation with loss. A third strategy of oscillation orientation involves the willingness to actively switch from mindset to the other, giving one's mind a rest before thinking about the project. Employing this strategy led to both more learning and a time-bound decrease in negative emotion.

As important as it is to learn from our mistakes, making this our overriding focus may be counterproductive. The authors advocate giving more space for a restorative approach, accepting that it can be good not to think about failure, and actively switching mindsets to gather insights while improving attitude toward the project over time. Their data also shows that a culture that considers failures as normal, taking it in its stride, leads to lower negative emotions overall, so there are steps that organisations can take as well.

Shepherd, D., Patzelt, H., & Wolfe, M. (2011). Moving Forward from Project Failure: Negative Emotions, Affective Commitment, and Learning from the Experience The Academy of Management Journal, 54 (6), 1229-1259 DOI: 10.5465/amj.2010.0102