The Persistence of Error

There is an embarrassing, unanswered question about theories and models in Psychology that is screaming to be answered. If the evidence in support of Psychology’s models and theories is so meagre and feeble, how have they survived for such a long time?

The scientific method is intended to be a fail-safe procedure for abandoning disconfirmed hypotheses and progressing with hypotheses that appear not to be disconfirmed. The psychologists who dream up these theories and test them claim to be scientists, so what the heck is going on?

One reason that theories and models become semi-permanent features of textbooks and degree programmes is that simple rules at the very heart of science are persistently broken. If a theory is tested and found wanting, then one of two things happens: either (1) the theory is revised and retested or (2) the theory is abandoned. The history of science suggests that (1) is far more frequent than (2). Investigators become attached to the theories and models that they are working with, not to mention their careers, and they invest significant amounts of time, energy and funds in them, and are loath to give them up, a bit like a worn-out but comfortable armchair.

We’ve all been there – seen it, done it, even have the T-shirt:

t-shirt-homme-le-petit-encule

Nothing dishonest is happening in most such cases, simply an unwitting bias to confirm one’s theoretical predilections. This is the well-known confirmation bias studied by, yes, you guessed it, psychologists (e.g. Nickerson, 1988).

The process of theory or model testing is illustrated in the diagram. The diagram shows how the research process insulates theories and models against negative results, leading to the persistence of error over many decades. Continuous cycles of revisions and extensions following meagre or negative results protect the model from its ultimate abandonment until every possible amendment and extension has been tested and tried and found to be wanting.

Screen Shot 2018-09-07 at 08.51.32What textbooks don’t tell you: the persistence of error – the manner in which a model or theory is ‘insulated’ against negative results

Several protective measures are available to insulate investigators from ‘negative’ results:
(1) Amend the model and test it again, a process that can be repeated indefinitely.
(2) Test and retest the model ignoring the ‘bad’ results until some positive results appear that can happen purely by chance (a type 2 error).
(3) Carry out some ‘statistical wizardry’ to concoct a more favourable-looking outcome.
(4) Do nothing, i.e. do not publish the findings, and/or:
(5) Look for another theory or model to test and start all over again!

Beside all of these issues, there is increasing evidence of lack of replication, selective publication of positive findings, and outright fraud in psychological research, all of which militate against authentic separation of fact from fantasy (Yong, 2012).

Little attention has been paid to the cultural, socio-political and economic conditions that create the context for individual health experience and behaviour (Marks, 1996). Thousands of studies have accumulated to the evidence base that is showing that socio-cognitive approach provides inadequate theories of behaviour change. Any theory that neglects the complex cognitive, emotional and behavioural conditions that influence human choices is unlikely to be fit for purpose. Furthermore, health psychology theories are disconnected from the known cultural, socio-political, and community contexts of health behaviour (Marks, 2002). Slowly but surely these issues are becoming more widely recognized across the discipline and, at some point in the future, could become mainstream.

As we have seen, critics of the socio-cognitive approach have suggested that SCMs are tautological and irrefutable (Geir Smedslund, 2000). If this is true, then no matter how many studies are carried out to investigate a social cognitive theory, there will be no genuine progress in understanding.

Weinstein (1993: 324) summarized the state of health behaviour research as follows: ‘despite a large empirical literature, there is still no consensus that certain models of health behaviour are more accurate than others, that certain variables are more influential than others, or that certain behaviours or situations are understood better than others.’ Unfortunately, there has been little improvement since then. The individual-level approach to health interventions focuses on theoretical models, piloting, testing and running randomized controlled trials to demonstrate efficacy.

It has been estimated that the time from conception to funding and completing the process of demonstrated effectiveness can take at least 17 years (Clark, 2008). Meta-analyses, reviewed here, suggest that the ‘proof of the pudding’ in the form of truly effective individual-level interventions is yet to materialize.  Alternative approaches for the creation of interventions for at-risk communities and population groups are needed. A fresh approach requires a general theory of behaviour that encompasses human intentionality, desire and purpose within an ontology of change.

Psychology Bankrupt?

Featured

Is Psychology a bankrupt science? The majority of theories are wrong, the majority of methods do not work and the majority of studies cannot be replicated. In A General Theory of Behaviour I argue for a complete redesign of the discipline.

There are several reasons why the most popular techniques used by psychologists to help people change are ineffective. The evidence does not justify any confidence in the theories, in the methods used or in the explanations provided. Meta-analyses of theory testing studies paint a gloomy picture. The overall pattern of findings suggests that current psychological theories and models cannot provide a viable foundation for effective interventions.

One core limitation with many theories and therapies is their use of the ‘Social-Cognitive Model’ (SCM). The SCM holds that a person’s ability ‘get better’ or to change is a social-cognitive problem, i.e. the person is said to have the ‘wrong’ thoughts and beliefs. According to the theory, these ‘unhelpful’ cognitions must be changed to produce a change in behaviour. But what if the beliefs are correct, or are only a small part of the whole picture, and what if they have little relevance to the behaviour or symptoms that the person is wishing to change?

Other reasons for the failure of the SCM in real-world behaviour change are briefly described below.

Individualistic Bias
Choice and responsibility are internalized as processes within individuals similar to the operating system of a computer. The human ‘operating system’ is assumed to be universal and rational, following a fixed set of formulae that the models attempt to describe. Yet even within its own terms, the programme of model testing and confirmation is failing to meet the goals it has set.

Lack of Ecological Validity and Questionable Statistical Methods
Thousands of published studies have used null hypothesis testing with small samples of college students or patients. The power, ecological validity and generalizability of these studies is questionable. We do not really know their true merit because of uncertainties about representativeness, sampling, and statistical assumptions. Rarely are alternative approaches to theory testing utilized, for example, Bayesian statistics and power analyses, to assess the importance of the effects rather than their statistical significance (Cohen, 1994).

businessman-would-be-bankrupt_1253-1590

Self-report measures
Most studies use self-reported measures of intention and behaviour rather than objective measures. Always a huge problem! It means that the academic studies have little contact with the universe of real-world, objective behaviour.

Neglect of Culture, Religion and Gender
Religion, culture and gender are neglected by most socio-cognitive models. The models aim at universal application that is unachievable.

Unfalsifiable
Some strident critics have suggested that the models are tautological and, therefore, unfalsifiable (Smedslund, 2000). A tautology is a statement that is necessarily true, e.g. ‘Jill will either stop or not stop smoking’ or “The earth is round (p<.05)” as one famous paper would have it (Cohen, 1995). Whatever data we obtain about Jill’s smoking, the statement will always be true – a very safe prediction. Smedslund (2000) deduced that, if tautological theories are disconfirmed or only partially supported by empirical studies, then the studies themselves must be flawed for not ‘discovering’ what must be the case!

Bad models can only be supported by bad research. Others have argued that behavioural beliefs (attitudes) and normative beliefs are basically the same thing. Ogden (2003) analysed empirical articles published between 1997 and 2001 from four health psychology journals that tested or applied one or more social cognition models (theory of reasoned action, theory of planned behaviour, health belief model, and protection motivation theory). Ogden concluded that the models do not enable the generation and testing of hypotheses because their constructs are unspecific. Echoing Smedslund (2000), she suggested that the models focus on analytic truths that must be true by definition.

Unsupported Assumptions
The transtheoretical model has received particular criticism. Sutton (2000b) argued that the stage definitions are logically flawed, and that the time periods assigned to each stage are arbitrary. Herzog (2008) suggested that, when applied to smoking cessation, the TTM does not satisfy the criteria required of a valid stage model and that the proposed stages of change ‘are not qualitatively distinct categories’.

Procedural Issues
Studies measuring social cognitions rely upon questionnaires which presuppose that cognitions are stable entities residing in people’s heads. They do not allow for contextual variables which may influence social cognitions. For example, an individual’s attitude towards condom use may well depend upon the sexual partner with whom they anticipate having sexual contact. It may depend upon the time, place, relationship and physiological state (e.g. intoxication) within which sex takes place.

French et al. (2007) investigated what people think about when they answer TPB questionnaires using the ‘think aloud’ technique. French et al. found problems relating to information retrieval and to participants answering different questions from those intended and they concluded that: ‘The standard procedure for developing TPB questionnaires may systematically produce problematic questions’ (p. 672).

Neglect of Motivation
Another problem with the SCMs is that they do not adequately address the motivational issues about risky behaviours. Surely it is their very riskiness that in part is responsible for their adoption. Willig (2008) questioned the assumption that lies behind behind much of health and sex education ‘that psychological health is commensurate with maintaining physical safety, and that risking one’s health and physical safety is necessarily a sign of psychopathology’ (p. 690).

Redesign of the Discipline
Many people love taking risks; they find taking risks enjoyable, exciting, and exhilarating. If you doubt this fact, take a stroll into any casino or race track, or wait at the bottom of Mount Everest for the body bags.

Until psychology addresses the causes of behaviour, it will never succeed in helping people to change. For this, we need a complete redesign of the discipline. For an in-depth analysis, see my book A General Theory of Behaviour.

Changing Behaviour

Featured

The vast majority of people change their behaviour with no external help. They just do it. ‘Change experts’ include psychologists who advocate behaviour change techniques in their interventions. A behaviour change technique (BCT) is any systematic procedure (or a category of procedures) included as an active component of an intervention designed to change behaviour. The defining characteristics of a BCT are that it is:

• Observable
• Replicable
• Irreducible
• A component of an intervention designed to change behaviour
• A postulated active ingredient within the intervention (Michie et al., 2011).

The description, classification and investigation of BCTs has become a cottage industry. Places like UCL, Aberdeen and Cambridge Universities, together with IBM, have received several millions of pounds from the Medical Research Council and Wellcome Trust to construct an ‘ontology’ of behaviour change.

According to the project website, “Behavioural Scientists are developing an ‘ontology’: a defined set of entities and their relationships” which will be used to “organise information in a form that enables efficient accumulation of knowledge and enables links to other knowledge systems.”

bs-diagramdetailedThe top level of the ‘Behaviour Change Intervention Ontology’ (project website)

An ontology is a set of concepts and categories in a subject area that shows their properties and the relations between them. An ontology can only be helpful when nothing of importance to the system as a whole is left out.

A ‘BCT Taxonomy’ has been employed to code descriptions of intervention content into BCTs (Michie et al., 2011, 2013). The taxonomy aims to code protocols in order to transparently describe the techniques used to change behaviour so that protocols could be made clearer and studies could be replicated (Michie and Abraham, 2008; Michie et al, 2011). A taxonomy also can be used to identify which techniques are most effective so that intervention effectiveness could be raised and more people would change behaviour.

The production of a structured list of BCTs provides a ‘compendium’ of behaviour change methods which helps to map the domain of behaviour change and inform practitioner decision-making. However it also risks becoming a prescriptive ‘cook-book’ of what therapeutic techniques must be applied to patients presenting with a specific behavioural problem.

Another problem with the compendium approach is that BCTs are not all optimally effective when combined in ‘pick-and-mix’ fashion. There needs to be coherence to the package that is provided by a theory that offers power and meaning and connects the components into a working set.

I can illustrate this point by considering an intervention for smoking cessation, Stop Smoking Now (Marks, 2017). This therapy is an effective method for clearing the human body of nicotine. The desire to smoke and any satisfaction from smoking are abolished using different forms of CBT and mindfulness meditation. Stop Smoking Now includes 30 BCTs integrated within a coherent theory of change based on the concept of homeostasis. In Stop Smoking Now a structured sequence of BCTs is provided that takes into account the nesting of BCTs such that guided imagery works best in combination with relaxation and both of these work best following enhancement of self-efficacy, achieved using self-recording, positive affirmations and counter-conditioning.   In addition, our field evidence shows that the outcome is enhanced by having a personable delivery from a charismatic person who builds a positive therapeutic alliance.                  

bs-diagramdetailedWith so many missing elements, this an Incomplete Model of Behaviour Change

Where is the client person in the ‘Behaviour Change Intervention Ontology’, and what about their feelings and their own striving for new balance and equilibrium?  Where is the therapist and the therapeutic alliance?  The quality of the change agent, their clinical and interpersonal skills and the quality of the therapeutic alliance can be more important than the BCTs (Hilton & Johnston, 2017) .With so many missing elements, this is beginning to appear like a top-down model of behaviour change. One may be excused for wondering whether the people designing the ‘ontology’ have any real-world hands-on experience of delivering interventions.

Hagger and Hardcastle (2014) suggest that “Interpersonal style should be included in taxonomies of behavior change techniques”. The whole point is that the therapeutic alliance is something the therapist and the client need to strive for. The alliance creates a more equal power balance between therapist and the client. It is more important than another technique, another item on the list. It is more about the ‘chemistry’ of the client-therapist relationship than about a finely polished set of BCTs. The trouble is that the advocates of the BCT compendium/ontology appear unwilling to engage with the problem. Somewhat ironically, they are resistant to change. However, the problem will not just go away, but rears its head each and every time a therapist swings into action.

Behaviour change involves a collaboration between the client wanting to make the change, with their own desires and feelings, and the change agent/therapist. The therapeutic alliance between the two parties is crucial to the project’s ‘outcome’.  Therapist’s attributes such as being flexible, honest, respectful, trustworthy, confident, warm, interested, and open contribute to that alliance. From all of this it can readily be seen that the situation is far more complex than the proposed ‘Behaviour Change Intervention Ontology’. It is never as  simplistic as an ‘Intervention’,  ‘Mechanisms of Action’ and ‘Target Behaviour’.

To use an analogy, there is so much more to baking a cake than a set of ingredients. Of course one needs a set of ingredients (the BCTs) but one also needs a baker – the behaviour change agent (BCA). The BCA/therapist must be fully trained to prepare, mix and cook the ingredients, to be fully competent to deliver the BCTs in a stylish manner. The qualities of effective therapists have been studied for at least 50 years. The stock piling of a compendium of BCT ingredients without attending to the mixing and ‘baking’ of the ingredients by the BCA on the front line is a recipe for disaster.

smart chef character cooking behind kitchen table with various o

Including therapist attributes of flexibility, authenticity, respect, trustworthiness, confidence,  warmth, interest, and openness, along with the client’s goals, desires and striving provides a more accurate and comprehensive approach to behaviour change.

cropped-screen-shot-2018-08-17-at-15-52-50.png