What's the evidence for coaching?

Extending the conversation about evidence in psychology, the DOP conference held several sessions looking at this exactly this issue as it pertains to an area close to the occupational domain: coaching.

A discussion session led by Rob Briner asked the simple question: does executive coaching work? As a whole, the field contains many claims about its effects, which Briner demonstrated using choice quotes taken from public websites. Coaching can apparently make you more effective at work, help you lead your team better, help resolve interpersonal tensions, uncover strengths and help overcome weaknesses. Yet, as became clear during this workshop, very few practitioners have the evidence at their fingertips to support such claims. And as we will see, this is because such evidence is scant.

Another challenge to a truly evidence-based coaching is that coaching enterprises are often entered into without a clear definition of what success would look like. When the problem is extremely ill-defined, applying a 'treatment' (in the scientific sense) and then seizing on any changes as proof of this treatment being effective is problematic. That said, Briner made it clear that evidence-based does not mean 'randomised control trial or nothing', and outlined four types of evidence that matter:

• Practitioner experience
• Local context
• Evaluation of research
• Opinions of those affected

There is value to all of these, and in some situations (for example when dealing with new approaches or revisions to these) it isn't reasonable to expect an activity to have peer-reviewed data behind it. But coaching is hardly new, so we should be moving to a position where the core claims are in some way validated.

The subsequent day's symposium on evidence in coaching was therefore well-timed. Amongst some presentations of interesting but specific investigations, Yi-Ling Lai presented her PhD research comprising a systematic review of the evidence on coaching. As these reviews should - and we'll look at this more thoroughly in a future digest - it followed a highly systematic process to identify what should and shouldn't be included in this study. The danger with non-systematic reviews is that of cherry picking data to fit a narrative (intentionally or no), and this analysis avoided this issue, and presented what appears to be a reliable account of the current state of the field.

The review was highly useful in understanding what practitioners felt mattered most in coaching relationships - factors such as emotional support and trust, and the overall quality of the coaching relationship, rather than merely the content of the coaching sessions, was believed to be key. However, if anything, the review underlined the fact that we are still missing an adequate amount of hypothesis-driven research on effects and outcomes, at least of the sort that would support the claims that are frequently made about coaching. In line with the consensus that the previous day's discussion led to, the take-away is to talk about and market coaching activities in a way that you feel you can defend, using the four forms of evidence and being honest about unknowns.

The effect of coaching isn't understood in the way that aspirin is, nor is it likely to ever be. This isn't a problem, up until we make it one.

Further reading:

Grant, Anthony M;, Passmore, Jonathan;, Cavanagh, Michael J;, & Parker, Helen. (2010). The State of Play in Coaching Today: A Comprehensive Review of the Field International Review of Industrial and Organizational Psychology , 25, 125-167 DOI: 10.1002/9780470661628

(direct pdf link available here)

Personality similarities between people who share an employer are as strong as between those who share a nationality

Is bigger better when it comes to data? After all, a well-designed study can demonstrate powerful and meaningful effects even within a modestly sized sample. Still, if you get your hands on a large enough data set - and you know what you are doing - you can potentially get a lot out of it.

In our penultimate report from the DOP 2014 conference, Prof Dave Bartram - someone who certainly knows what they are doing - presented work from a data set clocking in at a formidable 92,561 job applicants. At a previous conference Bartram reported on how countries have more homogenous personalities: two British people will be slightly more similar than a Brit with another nationality.

Evidence suggests a similar thing happens with organisations, and Bartram's team was interested in using this mammoth data set, with individuals in 35 countries applying to 490 different organisations, to look at how these effects interact. Can we make a better guess to your personality based on the country you work in, or the organisation you work in? How much of the organisation effect is explained by the industry you work in? Are country and organisation effects distinct, or interrelated - perhaps through the industries that a country focuses on?

The personality measure used was the Occupational Personality Questionnaire, which takes a fairly fine-grained approach that reports across 32 personality traits. A multi-level modelling analysis showed that 12% of the personality variance was explained by country and organisational membership, each accounting for a similar proportion. With 88% of the variance unaccounted for, there is still a range of personality within any given organisation or country, but the commonality is definitely present. Industry effects, meanwhile, were very small: around 2%. So the specific organisation you are in or applying to says more about your personality than the broader industry does.

Follow-up analysis showed that these country/organisation effects varied from trait to trait. Some, such as persuasiveness, competitiveness, and appetite for busy work conditions, were highly sensitive to country and organisational membership, which explained up to 20% of their variance; other traits far less so.

The organisation and country effects didn't tend to turn up for the same traits, either: the correlation between the two was close to zero. Instead, each environmental factor tended to target different types of traits. For instance, traits related to extraversion and conscientiousness were up to four times more related to organisation, whereas those related to emotional stability were up to five times more related to country. This suggests that the aspects of personality that comprise a 'national culture' are generally distinct from those that translate to an organisational culture.

Research has shown that organisations can develop a typical personality type, often shaped by their founders, whose own personality traits directly influence how the organisation operates. This big dataset shows that organisations attract people who are as similar to each other as are national compatriots. The ways in which people cluster due to organisation is different from countries, focusing more on traits that relate to entepreneurial, creative, and communication styles that clearly differ from workplace to workplace. Personality is multifaceted, and those facets appear to have a different relevance for the different aspects of our environments.  This kind of large-sample research is one means for us to get a better understanding of how we are different, and how we are similar.

Trust at Work: Broken Promises and the Manager's Mirror

In our last post we introduced insights on trust in organisations from the symposium run by the Centre for Trust and Ethical Behaviour. We learned that surface features such as organisational prestige and size, as well as historical features such as your own experience or the organisations track record, influence how we look at organisations from the outside.


In this post, we continue by looking at how trust – or lack of trust – endures even in the face of contradictory events. And we see the particular importance of one person in shaping our trust in the organisation: our manager.

Michelle McGrath's presentation explored promises made and broken in the workplace. Her work focuses on the psychological contract – the unwritten contract between the workplace and an employee that they carry with them and use to view their working existence. One person may see their arrangement with the employer as a principally transactional one, where I do A in order to receive B. Another believes they are entering into a relationship with the employer, treating the situation as one based around a strong bond and mutual trust.

McGrath interviewed 30 people using a Critical Incident technique to identify and delve deeply into situations where they felt the employer had broken or exceeded a promise. Each participant was also categorised with respect to their psychological contract. Those with a relational contract treated over-delivery of a promise as a positive example of how the organisation valued them. And they were forgiving of promise-breaking too, rationalising it as situational rather than reflective of the organisation's agenda.

Those with a transactional contract, meanwhile, were unimpressed when a promise was exceeded. In their eyes, it just showed that the organisation was erratic, and they felt that 'anything good that happens is always short-lived.' Interestingly, this resigned attitude also extended to promise breaking: it was annoying but 'what else would you expect?' To an extent, the meaning of the contract overrode the events themselves, with participants continuing to see the employer through the same lens.

One caveat to this work is that the sample didn't contain extreme examples of promise-breaking, nor highly disaffected employees. After all, we know that when a relational contract is damaged sufficiently it can cause serious problems. The metaphor that comes to mind is that of a well-filled tyre on a fast car: encountering one or two pebbles on the road won't interfere with things - but a patch of broken glass is still a disaster in the making…

Attitude to managers account for 35%  per cent of the variance in organisational trust, making Alison LeGood's presentation on the topic highly relevant. Her evidence suggests that in some way we treat the strengths and failings of our managers as a mirror of how the organisation at large behaves, meaning that we take an ethical manager to imply an ethical organisation. This was demonstrated within a study that asked individuals to rate their managers - 201 within mid to senior level positions - on behaviours that fell across three areas:

* Integrity, including behavioural consistency
* Ability, such as demonstrating and delegating control
* Benevolence, such as open communication and delegating concerns

Individuals also rated different facets of trust in the organisation, which turned out to correlate with relevant behaviours of the manager: for instance, believing that your manager shows open communications makes it more likely you trust the organisation as a whole to be benevolent. For the Integrity and Ability factors, the relationship was stronger for more senior managers. But across every level Benevolent manager behaviour was associated with perceptions of a kinder workplace.

Whether employees assume their manager truly reflects the organisation's agenda, or are simply using them as a proxy to offer some information to navigate the complexities of organisational life, our managers are tied to whether we trust our workplaces.

The symposium provided a multi-faceted look at trust in organisations, that I hope these write-ups demonstrate. It alerts us to structural features of how an organisation is (size, prestige) and how it operates (communications, history) all shape the ‘outsider’s eye.’ It emphasises the power of developing relational psychological contracts, sturdy enough to absorb the occasional disappointment.  It reminds us that individual tendencies to trust are important, but the behaviour we see in the organisation – especially from our managers – is even more so. Some trust comes for free, but more is earned, and all must be kept.

It's About Trust: Expectations & Interactions

Trust arrives on foot, but leaves on horseback. These were the closing words of a fascinating symposium dubbed "The How, Why & For Whom of Organisational Level Trust," introducing research from Coventry University’s Centre for Trust and Ethical Behaviour. We’re covering the symposium in this post and another later this week.

Trusting someone means you are prepared to let yourself be in a place of vulnerability, in the belief that the other person will not let you down. Low trust is associated with a range of organisational problems, such as low motivation, low commitment, and cynicism, as well as intention to leave. And higher trust has active benefits - in an article freely available online, it is shown to be associated with putting in extra effort in the form of organisational citizenship behaviours such as helping others or speaking out about improvements, and ultimately in better organisational performance. Trust matters. So how do organisations become trusted?

The symposium kicked off with Prof Rosalind Searle’s talk exploring what draws a new hire to put more trust in their new workplace - for example, when they perceive the organisation as prestigious. To my surprise, those joining a smaller organisation were less trusting. Granted, it can be reassuring to know that your employer is well-established, hopefully with mechanisms and policies developed over time to give you some protection. But Searle’s claim also suggests smaller workplaces aren't fully capitalising on their more intimate scale and personal familiarity. Further research is needed to identify the specifics of what is dampening trust here.

There is another crucial factor that influences our general trust in a workplace, and that is our 'propensity to trust:' an enduring trait reflecting the stable component of how trusting we are across contexts. While this is something that organisations can’t directly influence, it can be useful to understand the impact of low propensity to trust, especially if your recruitment for a role may lean towards someone with that trait.

Searle also took us through another way we relate to organisations: as a consumer or service user. In this role, we have to take even more on trust than as an organisational insider. For instance, most of us don't scrutinise the food sourcing practices of our supermarkets, we just trust that their meat is what they say it is. When that trust is lost, it can have substantial consequences.

Evidence suggests many factors influence consumer trust globally, differing from region to region. For example, endorsements from other users are crucial in Asia, whereas relationship history - how you personally have been treated to date - matters as much or more to US and European consumers.  It’s possible this could dovetail with the collective-individualistic differences between cultures, which we know influence some consumer behaviours such as complaints. Meanwhile, transparent communications - such as breaking bad news stories or sharing details of internal workings such as mergers -  is particularly important in business-to-business contexts.

Our surface interactions with organisations, then, are shaped by certain expectations and information sources, as are our attitudes to them as a new entrant. But once we are part of the organisation, how is trust earned and maintained? Check in later this week for our second report.

Can psychology help pick winning politicians?

When she received a letter from the British Conservative Party Prof Jo Silvester had no idea that it would draw her into the world of politics and its psychology for years to come. In her keynote at the 2014 DOP conference, she described how she ended up helping the three main national parties modernise how they understand and select capable politicians.

Silvester drew our attention to the abundance of research on the characteristics needed for political success: just look at the biographies and analyses available on any US president you care to pick. However, most of it uses 'at-a-distance' methodologies - inferring qualities of a notable figure through analysis of political speeches and interviews, or ratings of the person's in-office decisions by expert historians. It's particularly hard to find politicians’ self-ratings, and this is understandable - they operate in a sensitive realm and are wary of divulging information that might come back to haunt them ("Brown Control-Freak: It's Official" would be top tabloid fodder). All in all, there is insufficient research on what drives good political performance, especially from a work psychology perspective.

So it’s perhaps unsurprising how political selection processes have also developed without much reference to improvements in mainstream employment selection, such as the ubiquity of explicit, formal criteria and processes for selecting between candidates. Silvester’s letter was from the Conservative's Director of the Development and Candidates Department, who was looking to improve their candidate approvals process and in particular to facilitate an increase in women MPs from their abysmally low numbers. Silvester quickly discovered there were no clear selection criteria and a heavy reliance on the personal judgment of committee members.

Taking the classic components of occupational psychology - role analysis, construction of competencies, and development of assessment activities - Silvester reconfigured this process into an assessment centre involving cognitive tests, structured interview and a group exercise. Assessors were trained and an evaluation showed that the centre recommended equal numbers of men and women. Additionally, ratings at the assessment centre correlated with candidate performance at the following (2005) election - Communication skills with overall votes, and Critical Thinking with both overall and swing of the vote. This suggests that the process was measuring something politically meaningful: vote-winning.

Silvester was subsequently contacted to work in a Labour-led initiative across local government, and then a project with the Liberal Democrats. These projects also identified important political competencies, and all parties showed substantial agreement that politicians needed to be resilient, people-focused and also highly analytic. There were some differences in emphasis: only local politicians emphasised Politicking as a stand-alone skill; and where the local and Lib Dem politicians emphasised Representation, the Conservatives placed their emphasis on Leadership.

The local government research also investigated personal traits through politician self-ratings, which as we’ve noticed is a rare harvest. They found that politicians judged by colleagues to be high performers were more conscientious, less neurotic, and more politically skilled. Against predictions, extraversion was not associated with success. Whereas gregariousness may be a stereotypical political trait, calmness and diligence are more key to the job.

In some ways, the talk highlighted how politics lags behind other sectors in terms of its HR practices. But Silvester cautioned us not to assume that best practice elsewhere makes sense here. Thanks to the obligations of democratic principle, a political party simply cannot operate like a business or even other public sector areas: objective criteria can take a candidate so far, but final decisions must be made through the will of the people. It also struck me that concerns about a narrowly clustered political class selected from Oxbridge grads and 'Spads' (special advisors) might only become compounded by an overtly assessment-focused approach to selecting candidates: career politicians could have another objective test to prepare to ace once their PPE finals are done. There is a balancing act to ensure that politicians are capable at their job but also reflect the body of the nation. As such, this is a fascinating field with many questions ahead.

If you find this area as interesting as I do, you may want to check out an interview with Prof Silvester in The Psychologist magazine, available to read here.

Perfectionism: The Good, The Bad, and The Way Beyond

Can perfectionism ever be useful? This was the question floated at the outset of an arresting keynote at the 2014 conference of the BPS's Division of Occupational Psychology. Paul Flaxman began by asking his audience to jot down helpful and hurtful features of perfectionism, and a show of hands demonstrated that many struggled to see a positive angle to it. They aren’t alone - many clinical specialists who study perfectionism share this view. But over the hour, Flaxman informed us about data that throws lights on the negative and positive facets of perfectionism, including his own research on perfectionism in the workplace.

Yes, there is evidence showing perfectionism to be associated with various negative outcomes. We've covered some of Flaxman's research before, in which people with perfectionist traits were found to recover from stress during vacations, but see that benefit dissipate quickly once they return to work. In general, perfectionism is also associated with weak productivity thanks to putting off completion of tasks or hitting walls in creative areas, and to personal frustrations.

However, evidence suggests that there are actually two kinds of perfectionism. At the heart of the problematic kind is a concern for how others see you, leading to doubts in your own worth. This so-called 'evaluative concern perfectionism' leads individuals to avoidance as a method of coping with stress (see link for problems this can cause) and review of the data suggests these people experience higher levels of hassle in life, along with more distress. But crucially, there is another strand of perfectionism, one that is primarily self-focused and concerned with personal standards. This strand is associated with active coping strategies, and with reaching higher levels of achievement. However, for these people, the higher amounts of life hassles - and possibly distress, although the data is less clear - remain. Flaxman emphasised that while the two strands involve different internal states, the behaviours can look identical from the outside.

One useful way to look at perfectionism is as an underlying vulnerability factor. Day-to-day perfectionism can chug away in the background, influencing but not determining behaviour, and if it's self-focused, it may facilitate better performance. But when experiencing achievement-related stress, such individuals encounter high distress. At its extreme, in the clinical context, failure is 'a fatal blow to the self', and can be associated with self-harming actions.

Flaxman urged his audience to foster more workplace-focused research in this area, as the field is still overwhelmingly clinical or student-focused - despite the fact that many of the survey instruments used explicitly reference work-related stressors, which often have to be removed before data is collected! And he showed evidence that interventions can be successful: his work with Acceptance and Commitment Therapy, a mindfulness-based approach, can help improve psychological flexibility, increase resilience, and reduce perfectionist attitudes.

Strengths-based coaching - Work in Progress

Coaching is a multi-billion pound business globally, and more and more attention is directed toward demonstrating its impact. A lot of available research deals with the coachee's perceptions of change rather than using more objective measures of impact, but a new paper, currently under review, uses data from a range of stakeholders to see how coaching translates to genuine organisational impact. Here we have an early look at this piece of research.

Doug MacKie's research attempts to settle several methodological issues. The first is that coaching has many potential impacts, not all of which are easy to anticipate. As I've said before, "when the problem is extremely ill-defined, applying a 'treatment' (in the scientific sense) and then seizing on any changes as proof of this treatment being effective is problematic."

To solve this, MacKie used a pre-defined and well-understood set of criteria, the Bass and Avolio Full Range Leadership model. Widely used, it covers behaviours that are transformational, transactional - both desired - as well as behaviours from the more harmful avoidant and laissez-faire styles. As this model has been implemented in a 360-degree feedback format, the MLQ-360, it's possible to gather the data from others.

A further problem is what it is about the coaching experience that is actually useful. So, this study utilised manualisation, meaning that session by session, the coaching proceeded "by the book".

In the study, 37 senior managers were recruited into two groups. Members of the first received coaching while those in the second were placed on a waiting list. At the start of the study participants were rated on their leadership skills by themselves and others known to them using the MLQ-360 tool. The coaching group then received six sessions of strengths-based coaching, which involves identifying strengths, and developing them through pursuit of appropriate goals.

Both groups were then retested with the MLQ-360. Somewhat anomalously, both the coaching *and* waitlist groups showed an improvement in the overall leadership score, but the effect size for the coaching group was three times greater, and statistically significantly different from the wait group.

At this point, the waitlist group received their coaching, and at the end of this all participants were rated on leadership for a third time. The group that got coaching second saw a rise in leadership scores, and the first-coached group continued to improve in performance even though they received no further coaching. From the start of the study, therefore, leadership scores rose at every measurement point whether participants had just received coaching or not.

Although we can interpret the continuing climb of the first-coached group as showcasing just how powerful coaching is, there is also the risk that the Hawthorne Effect is kicking in (see link, briefly, people temporarily increase performance when there are novel observers examining them). However, MacKie also measured how closely coachees adhered to the coaching process, both through their own reported accounts, and through scoring of manuals and workbooks used in the process. As both measures were significant predictors of final leadership scores, it gives us some assurance that the specific coaching method has some impact. Data demonstrating improved leadership using a non-self report measure is heartening to those who practice or receive coaching.. Let's await the reviewers' verdict.

As MacKie states, we may now be at the time “to compare conditions and methodologies to find the optimum blend of critical components of effective coaching”. While there is increasing evidence that coaching is an effective intervention, compared to doing nothing, I'm after more compelling evidence to why this approach to facilitating human development is valuable.

Does time spent coaching lead to better outcomes than if it were spent reading a self-help book, writing about challenges, or discussing things with a friend or non-coaching business advisor? MacKie's data on adherence is a step in this direction, but to determine coaching's return on investment we still have a way to go.

Further reading:
Bass, B. M. (1999). Two decades of research and development in transformational leadership. European Journal of Work and Organizational Psychology, 8, 9-32