The questionable evidence base of SCoPEd

David Murphy, Associate Professor at the University of Nottingham, interrogates the claim that the SCoPEd framework is ‘evidence-based’.

---

 

A cornerstone of the SCoPEd approach to generating their competences is to arrive at their findings having drawn from an ‘evidence base’. This is apparent in two separate claims made in the opening two paragraphs. Here the SCoPEd projects’ members say, “The aim of the SCoPEd project is to agree a shared, evidence-based generic competence framework to inform the training requirements, competences and practice standards for counsellors and psychotherapists working with adults.” And they go on in the next paragraph to say that, “The draft framework has utilised an evidence-based process of mapping existing competence frameworks, professional standards and practice standards to identify areas of overlap and areas of difference between counselling and psychotherapy.”

In this blog, the first of two tackling the issue of an ‘evidence-base’, I shall point to the fact that any reference to the use of evidence is questionable and more likely not, and therefore lacks integrity. First, let’s look at one of these statements again in more detail, “The aim of the SCoPEd project is to agree a shared, evidence-based generic competence framework to inform the training requirements, competences and practice standards for counsellors and psychotherapists working with adults”. Here the aim clearly suggests that the differentiated competences between the role of counsellor and psychotherapist will be grounded in evidence. But what do the SCoPEd team mean by evidence? And what might be an example of ‘evidence based competence’ that could be differentiable and identify clear evidence for a difference between the activities carried out under these two adjectival titles?

To first address the issue what evidence is being used, it seems that the list of sources used to arrive at the differentiated competence framework have been taken from existing lists of competences used, or even sponsored, by the same institutions that are trying to the find the evidence for the claimed differences between these titles. One of the issues with this process is that the documents tend to identify something that isn’t real. They are a fiction in regards to what people actually ‘do’. This is a well-known problem with the development of competences designed for a ‘labour market’ and they fail to adequately identify both the specifics and idiosyncrasies of actual praxis. What would be better, and would provide real evidence, is to look closely at what people actually ‘do’ in their work under these different titles.

Doing this would, I am sure, show something quite different from what is presented in the framework and might even show some new or unexpected findings. But the chance of discovering something new through this process has been closed off right from the outset because the aim is really not as it appears on the surface. The idea that counsellors are not doing the same work as psychotherapists flies in the face of all the evidence one can find from spending even just a little time listening to counsellors talking about their work. The chance to find this out has been forgone because a decision was made, at the outset, to look for the evidence in the data that already tells the SCoPEd team what they wanted to know.

‘But what does the actual evidence say?’

Let us look at just one example of a differentiation offered in the SCoPEd report. That is, Competency 3.10 under Theme 3 – Relationship, and refers specifically to the idea of ‘alliance ruptures and repairs’ (competence 3.10, 3.10b, 3.10c). In the SCoPEd document it is suggested that where there is a lack of ‘empirical evidence’, ‘grey literature’ has been used. It is difficult to know what evidence or grey literature has been used for the differentiation of this particular competence; not least because the research on therapeutic alliance, including the more specific topic of alliance rupture and repair, is one the largest bodies of empirical research in the entire psychotherapy process-outcome literature. Yet there is no citation to the empirical evidence that has been consulted. However, as this area of the psychotherapy literature is so extensive we probably should assume that the SCoPEd team will have consulted a recent meta-analysis of alliance rupture repair and outcome studies.

Looking at the differentiated competences (3.10, 3.10b, 3.10c) it seems that the SCoPEd project claim that psychotherapists are more skilled in dealing with alliance ruptures and repairs than are counsellors. But what does the actual evidence say about such a claim? Well let’s take a quick look. In a recent meta-analysis published in the American Psychological Association journal Psychotherapy (not a lightweight publication by any means) the very issue of alliance rupture and repair was considered as was its association to the improvement or progress that clients might make. Let me just say that whether or not one is against or for the quantitative paradigm, given that SCoPEd is an advocate for evidence it is probably worth noting the findings of this type of research because that is the premise on which it claims to be able to differentiate competence.

Esteemed alliance researchers Eubanks, Muran and Safran (2018) examined the association between alliance rupture repair and outcome in 11 studies involving 1,314 clients. Overall the effect size for rupture resolution and positive outcome was pretty impressive (d = .62 for those interested). Clearly good news for those theoretical approaches that regard the alliance as an integral feature of their practice (mainly psychodynamic but also CBT and some integrative therapies). A number of the studies included in this meta-analysis included ‘trainee therapists’ (so not even ‘qualified’ counsellors). The researchers tested the data to see if there were differences between the trainees and qualified/experienced therapists in this association between repairing ruptures and outcome. This was a test of moderation to see if this can account for the variance in the effects. The findings showed those studies that included primarily trainees do not differ significantly from those studies of more experienced qualified therapists in showing an association between rupture repair and outcome. This finding cannot provide evidence for supporting the claim that the more qualified/experienced a therapist is the more likely they will show an association between rupture repair and outcome.

‘the competences drawn up by SCoPEd have been deduced rather than induced from the close observations and recording of what therapists actually do’

So, the evidence from this most recent, up-to-date, and rigorous meta-analysis is overwhelmingly in favour of there being no difference between experienced/more trained therapists and the unqualified/trainee therapists in the associations they show between alliance rupture repair and outcome. But wait, the psychotherapists will say, obviously we refute this claim. Psychotherapists will work with ‘more complex and more distressed clients’ so there would be a difference if you looked at that, wouldn’t there? Well it seems the answer to that might also be available in the ‘evidence’ provided by this cutting edge meta-analysis. The evidence is as follows. Eubanks, Muran and Safran (2018) addressed the issue that more complex client work (such as that with clients given a diagnosis of a personality disorder) might mean that there would be less chance of a positive association between rupture repair and outcome. Again this was not found to alter the effects, meaning that whether or not a study included a majority of clients with a diagnosed personality disorder or not, the association between rupture repair and outcome did not differ significantly from those studies where this was not the case. So the ‘evidence’ suggests that even if psychotherapists claim to work with more distressed cases it seems this cannot be used as evidence to suggest they require more skills in working with alliance ruptures.

But wait, surely psychotherapists would be ‘trained specifically to deal with ruptures and repair’ and that is why their training is at a higher level. So now the question is what effects does having specific training in managing alliance ruptures have on the association of alliance rupture repair with outcome? Well, the same researchers completed a second meta-analysis comprising six separate studies and 276 therapists that had undergone such specific training. The findings again point to some interesting ‘evidence’. Once more, status as a trainee or qualified therapist undergoing this specific rupture repair training did not moderate the association between rupture repairs and outcome, suggesting that the finding is not affected by level of qualification. However, interestingly those training in CBT had a much stronger association between rupture repair and outcome than did the psychodynamically oriented rupture repair training. In fact, for psychodynamically trained therapists, the association was in a negative direction – suggesting the less effective they were in rupture repair, the better were client outcomes!

So what are we to make of this? Well it seems that if we are to look at the empirical evidence, that drawn from the very latest and most up-to-date evidence from meta-analysis, it appears to be suggesting that there’s little difference in association of outcome and rupture repairs in terms of the level of training a therapist has had. Of course, if we look to the ‘grey literature’ it is highly likely that we will find such differences; not least because those differences are simply statements of intent and not reality of practice. The issue here is that the competences drawn up by SCoPEd have been deduced rather than induced from the close observations and recording of what therapists, regardless of their level of qualification, actually ‘do’.

But let’s be honest, the ‘aim’ of SCoPEd is, at best, to determine what the differences are between ‘counselling’ and ‘psychotherapy’ in order to protect the various financial interests of those involved in the accrediting the training of counsellors and psychotherapists. At worst the aim is about laying the groundwork for the next attempt at protecting these titles under statute by pursuing the Statutory Regulation of counselling and psychotherapy. Referring to an ‘evidence-base’ is merely an attempt to give this project legitimacy in the eyes of the membership, who are kept in the dark about the real agendas playing out. This is an attempt to try and convince the members of BACP, UKCP and BCP – run down into the ground by the lack of employment opportunities after training – that they will be better off having these titles first differentiated and then protected. But in truth this will do nothing to protect or enhance the employment opportunities for the tens of thousands of under-employed counsellors/psychotherapists already trained and looking for work, and will do nothing to protect the public from rogue practitioners.

 

Reference

Eubanks, C. F., Muran, J. C., & Safran, S. D. (2018). Alliance rupture repair: A meta-analysis. Psychotherapy, 55, 508-519.

---