Evidence based practice when assessing sexual behaviours in children and young people: Ideology versus research – Sophie King-Hill & Kieran McCartan

Evidence based practice is a phrase that is used a lot in various services offered for children and young people. However, it is worth questioning how much evidence informs practice and how much of this is based within the expectations and ideologies in which the practice is situated. It is important to recognise that evidenced based practice and/or policy often means adaption based on research and transfer of evidence rather than blanket acceptance. This means that practice and policy is “evidence informed” or “evidence inspired”, that is, past research influences the conceptualisation of policy and practice, which is not the same as “evidence based”, an important distinction. An evidence-based approach is the full adoption of tried and tested practice that has been rigorously evaluated, and this means that the original evidence base needs to stand up to examination and that work needs to be done to conceptualise its adoption. This piece is going to investigate, and consider, some of these challenges via a discussion on the implementation of the Brook Sexual Behaviours Traffic Light Tool (TLT) into the UK context.

The Brook Sexual Behaviours Traffic Light Tool

The Brook TLT uses a traffic light system adapted from True Relationships & Reproductive Health, Queensland, intending to enable professionals to assess and categorise sexual behaviour in Children and Young People (CYP). The TLT intends to give professionals from various agencies a consistent and unified criterion when assessing and categorising sexual behaviours and provides a common language by which to refer to them (Brook, 2013).

The challenge of evidence-based practice: Developing a new evidence base or reusing an old one

Sophie, co-author of this article, carried out research evaluating the TLT. This research explored the challenges for knowledge translation when the TLT was being implemented within a locality in the UK in 2015. Exploration of the origins of the tool in Australia were traced back, as was the evidence base that informed the development of the tool and its subsequent transfer to the UK. Policy assessment frameworks from Dolowitz and Marsh (1997), Pawson and Tilley (1997) and McConnell (2010) were used to measure the success of the TLT and the legitimacy of its underlying evidence base.

The introduction of the TLT in the UK has raised questions as to whether sexual behaviour assessment tools in an area of practice as contentious and mutually exclusive as children and sexuality can ever be solely based upon evidence from one context, or whether they are forever bound to the context and ideologies they are situated in. It is important to understand the implications of “culture” and “context” in adapting good practice. Good practice may only be good practice in certain countries or regions because of local/national contexts that are specific to policy and practice in those areas, without which it would not necessarily work. A good example of this is Project Protection Dunklefeld (Beier, et al, 2015) which works in part because of German policies on the disclosure and reporting of sexual offence policies; therefore in countries like the UK, that have different disclosure policies, this approach cannot work.

The TLT in Australia was underpinned by a resource that drew on literature in the field and gained experiential and contextual evidence from the education and health systems. This used an approach advocated by Puddy and Wilkins (2011) called the ‘Best Available Research Evidence’ (BARE). This approach acknowledges that research in some areas is difficult to obtain, such as the sexual behaviours of children and young people. The TLT was originally developed to reduce the burden of over reporting and in response to the reoccurring question by parents, carers, and teachers of ‘is this normal?’. No new empirical evidence was gained from  using the BARE approach and the theoretical positions and secondary data that were used in the tool’s development, alongside the experiences of parents and professionals, are likely to have given the tool relevance in the context into which it was being introduced in Queensland.

Gathering evidence in this way may have lent itself to the nuances of the context into which the tool was being embedded and using the BARE approach in that context may have picked up on the subtleties of the sub-cultures into which it was being introduced. Does this mean, therefore, that as the tool is rooted in the Australian context it needs to be reconceptualised for a UK context, and if so, is this possible? This is made more complicated by the fact much of the literature that was used to inform the development of the TLT originated from the USA which may again have skewed the development of the tool due to the locational differences.  Essentially, we have a tool which was developed in an Australian context, based on American evidence, being introduced to the UK.

The implications for the use of the existing evidence base in a different context

The process of transferring the tool to the UK involved further input and changes being made by another BARE approach. Little literature from the UK was used to inform the adaptation of the tool and various steering/focus groups were used to direct the transition to the UK. This process requires consideration when assessing the evidence base for a tool such as this; the process and evidence base appears to be further ‘diluted’ and informed by anecdotal inputs, and literature and studies from other countries. Nevertheless, a larger evaluation study of the TLT in the UK carried out by King-Hill (2020, in press) found that the TLT was welcomed by professionals working with children and young people (CYP) across the board and was felt to be a useful resource. This research led to the following findings: 

• If the subject matter is a difficult one, such as sexual behaviours in CYP, and there is no such resource yet available, then any assessment tool which is seen to fill this gap and reduces the need for independent decision-making by practitioners, is likely to be welcomed and valued, even where the process and evidence base has been diluted.
  
  
• The tool appeared to wash away the context in which the behaviour was happening and became a ‘one size fits all approach’. The TLT is simplistic and easy to use (so immediately appealing to practitioners) with listed behaviours within age categories, yet it is attempting to assess a complex subject area where every situation will be different.
• The intended outcomes for the introduction of the TLT to the UK were less than specific which made ascertaining value difficult. Judging the wider success of the TLT is therefore challenging as it is based around non-specific outcomes and is situated within the value laden area of sexual behaviours and CYP. This makes evaluation of its effectiveness difficult.

Tracing the evidence base for a tool such as this has been a useful endeavour and indicates that challenges can be made to the legitimacy of tools that are used in areas of practice that are steeped in subjective judgements and assumptions, and gives insight into whether processes that are in place are fit for purpose. Tracing back the evidence in such a way has allowed for examination of the systematic rigour of the research underpinning the tool and provided an audit trail of the evidence that has informed it.  

Rather than critically engaging with the tool, the UK lifted TLT ‘off the shelf’ and applied it with relatively minimal consideration to the UK context, with an acceptance that it was good practice.   If this approach was to be used in the adoption of other tools or models for policy and practice (i.e. not embedded in robust empirical evidence), does this then render them unviable? Or, as in the case of the TLT in the UK, is having an easy to use tool which makes practitioners feel more supported in their decision-making, better than having nothing at all? 

Conclusion  

Gathering evidence to inform policy is a difficult process, especially in an area steeped in contextual and subjective values, which is particularly true in respect to research on sexual behaviours in children and young people.  It seems  we must consider whether the approaches that were used in the development of the tool in Queensland and subsequently in the UK were fit for purpose;  sporadic, non-contextual, non-reflective research and evidence can, after all be detrimental. The introduction of the TLT directly replicated Australian practice, therefore the work was evidence-based not “evidence informed” or “evidence inspired”, but was simply replicating Australian good practice (i.e. the evidence base) enough? Should there have been a more nuanced, evidence-based approach that examined the UK context the implementation of the TLT that allowed a well-developed, fit for purpose and appropriate model?

Using the evidence base is important and good practice, but what is important is that it should not be blindly copied across to different contexts.  Instead, it needs to be considered and adapted to the needs, limits, professional knowledge and current working practices/policies of the organisations introducing it. Evidence based means understanding the evidence base, not just being able to use it.

Sophie King-Hill, Senior Fellow, University of Birmingham

Kieran McCartan, Professor in Criminology at UWE, Bristol

 

References

Beier, K.M., Grundmann, D., Kuhle, L.F., Scherner, G., Konrad, A. and Amelung, T. (2015), Evaluation of Therapy with Pedophiles in the Dunkelfeld. J Sex Med, 12: 529-542

Brook (2013) The Traffic Light Tool Available from: https://legacy.brook.org.uk/brook_tools/traffic/index.html?syn_partner=&gclid=EAIaIQobChMI3bS98-fS7AIVWODtCh3bdwQREAAYASAAEgJOZ_D_BwE

Dolowitz, D.P. and Marsh, D. (2000) ‘Learning from abroad: The role of policy transfer in contemporary policy‐making’, Governance, 13(1), pp. 5-23.

McConnell, A. (2010) ‘Policy success, policy failure and grey areas in-between’, Journal of Public Policy, 30(03), pp. 345-362.

Pawson, R. and Tilley, N. (1997) Realistic evaluation. London: Sage.