Evaluation of the Improved Mental Health service

Sonia Barnes
Principal Research Adviser, Department of Corrections

Author biography:

Sonia designed and managed an evaluation of the Mental Health and Reintegration Services during her secondment to Corrections in 2017. Prior to working for Corrections she worked in research and evaluation roles for Police in NZ and in the UK. She has completed research and evaluation projects in a range of areas including family violence, mental health, youth, ethnic diversity, neighbourhood policing and police training.


Introduction

The Improved Mental Health (IMH) service started operating in Rolleston and Christchurch Men’s prisons and at Palmerston North Community Corrections in April 2017, and was operating across all sites by the end of August 2017. The purpose of the service is to improve the mental health of offenders experiencing mild to moderate mental health needs. In brief, IMH clinicians have been contracted to conduct mental health assessments of offenders, make referrals and/or develop care plans and, where appropriate, provide treatment of up to 10 sessions. On completion of treatment it is expected that the clinician will conduct another assessment to measure improvement. The clinicians are also expected to help build capability of Corrections staff in relation to managing offenders with mental health needs. Further detail about the IMH service is provided in Gilbert Azuela’s article .

In December 2017, an evaluation was conducted to investigate how well the service was operating, and to report on progress towards achieving the expected outcomes. This evaluation drew on the analysis of interviews with offenders, staff and clinicians, and administrative data. Interviews were conducted in six prisons and three Community Corrections sites [1] in August and September 2017. Administrative data used in the evaluation included data from reporting spreadsheets maintained by IMH clinicians, data from REFER Online (the Department’s electronic referral system) and the Integrated Offender Management System (IOMS).

The timing of the evaluation meant that only limited information could be obtained as the service had been in operation only a few months. In particular, there was limited data on the exit and transition processes, or how staff interacted with the various services.

Summary of findings

Overall, findings from the evaluation indicate that the service is operating as intended, and is contributing to improvements in offender mental health. There is also evidence of increased general staff awareness of mental health issues.

Corrections staff are referring offenders with mild to moderate mental health needs (for example stress, depression, anxiety) to the service, and in most cases are using the electronic referral system, as intended, to make these referrals. IMH clinicians are working with these offenders – generally over a period of up to five weeks – to improve their mental health. Most offenders are showing a reduction in their level of psychological distress after engaging with the IMH service. The evaluation also provides evidence of positive changes in how offenders are managing their emotions and behaviour, and improving their ability to respond to stress as a result of engagement with the IMH service.

The focus and extent of efforts to increase the capability of Department staff to support offenders with mental health needs is variable, but there are early indications that Department staff are becoming more confident in identifying offenders that may benefit from the IMH service.

Detailed findings: Progress towards outcomes

The main benefit expected from the introduction of the IMH service is that it will lead to an improvement in offender mental health. The evaluation found extensive evidence to suggest that positive changes in offender behaviour resulted from participation in IMH.

In particular, offenders commented on:

  • improved ability to discuss their emotions and what was going on for them
  • greater awareness of why they were feeling what they were feeling
  • improved ability to identify triggers for heightened emotions
  • reduced feelings of isolation
  • improvements in mood.

“Before I saw her I don't think I’d laughed or smiled for about six months, I’m actually laughing now. That's amazing when you can start laughing again. I put that through to having someone different to talk to.” (Prison IMH service user)

Offenders also described examples of managing difficult situations more effectively, and particularly without the inclination to use violence or other harmful behaviours. Offenders with anger control issues were proud to describe how they had managed incidents which, in the past, may have triggered violent outbursts, putting the wellbeing of Corrections staff and themselves at risk. For some, the positive change was knowing when and how to turn and walk away. For others it extended to engaging with the system in a more constructive way, like starting a discussion, or writing a letter.

“… I’m a way better, calmer prisoner here. Even when I’ve had difficulties with management, I had a few skirmishes, I did it all properly, I quoted the Corrections Act, put it into a letter, not once did I say f**k, and I got what I wanted.” (Prison IMH service user)

Some offenders had re-connected with family/whānau, describing those relationships as important in motivating them to continue a stable lifestyle.

Kessler10 assessments were conducted on both entry to and exit from the service for 112 offenders (21% of offenders who had exited the service at 24 October 2017). Analysis of level of change between entry and exit scores supported the findings of positive changes in behaviour.

The graph above shows the proportion of offenders within each category of level of psychological distress (low to very high) when assessed prior to engaging with the IMH service, and on exit from the service. Prior to engagement with the service, a higher proportion of offenders had high or very high levels of psychological distress [2] than low or moderate levels of psychological distress. On exit from the service, many who originally scored in the “high” and “very high” distress bands had moved down the scale to “low” and “moderate” distress bands; significantly fewer offenders were assessed as having high or very high levels of psychological distress.

Detailed findings: service operation

Referrals

By the end of October 2017, IMH clinicians across all sites had received nearly 1,400 referrals relating to 1,317 offenders (a small number of the offenders were referred to the service more than once, mostly as a result of transfer to another site). Most of the IMH service referrals (78%) were made in prisons.

The most common referral reasons were anxiety and depression [3]. Findings from the qualitative evaluation suggested that problems relating to stress or anger were also common reasons for referrals.

Analysis of referrals received by IMH clinicians suggested the recommended referral pathway (using Corrections’ “REFER Online”) was not always used.  This in turn helped identify barriers to use of this electronic referral process: corrections officers were found to be not sufficiently confident to refer using REFER Online as a result of limited access to computers in their units, uncertainty about the correct consent form to use, and fears around breaching confidentiality relating to the information required to be entered. IMH clinicians reported that they had been helping Corrections staff to upskill in using the online referral system.

At Community Corrections sites, probation officers tended to talk directly to the clinician when they identified an offender who might benefit from their support. Under these circumstances the clinician would typically assess the offender’s suitability on the spot, then ask the probation officer to complete a referral form later if the offender met the IMH service criteria. Possibly as a consequence of the more direct process, inappropriate referrals for the IMH service were recorded less frequently in the community (2%) than in prisons (10%).

It was intended that prison health centre managers would triage referrals in the electronic system to ensure that only appropriate referrals would be sent to IMH clinicians. Triaging was working well at some sites, but not all. Where triaging was not well-practiced, clinicians were expending time on unnecessary assessments.  When offenders were assessed but not accepted to the service, it was mainly because their mental health needs were at a level of severity that referral to the prison forensic service was the more appropriate path (some were already receiving input from forensics).

Assessment

IMH clinicians are required to use the Kessler10 tool to assess the level of psychological distress of offenders at entry to, and exit from the service. The assessment involves asking an individual 10 questions about psychological distress and then scoring each of the questions on a 1 (none of the time) to 5 (all of the time) scale. Item scores are then summed to provide an overall assessment: the lowest possible score is 10 and the highest is 50.

At 24 October 2017, IMH clinicians had engaged with 70% (973) of offenders for whom they had received referrals. For about a quarter of these offenders, an assessment of psychological distress on entry to the service had not been recorded. For the purposes of analysis, the entry assessment scores that had been recorded were grouped according to the following categories:[4]

  • 10 - 15 low level of psychological distress
  • 16 - 21 moderate level of psychological distress
  • 22 - 29 high level of psychological distress
  • 30 - 50 very high level of psychological distress.

Of the offenders for whom pre-assessments had been recorded, most were assessed as having high (30%) or very high (49%) levels of psychological distress. There were no apparent differences in the profile of psychological distress scores between offenders receiving the service in prisons and those in the community.

Treatment

All clinicians delivered interventions to offenders in one-on-one sessions. Working with offenders began with the IMH clinician building rapport and establishing a relationship.

In prisons, IMH clinicians generally worked with offenders in offices or rooms in the offender’s unit. At two of the sites, however, services were generally delivered at the health centre. One clinician observed that offenders engaged better when seen at the health centre because it afforded greater privacy.

Clinicians reported employing a range of tools and techniques such as “talking therapies”, use of props to prompt discussion, developing plans for coping, mindfulness sessions and providing resources, strategies, and tools.

Feedback from offenders on what they found useful in working with the clinicians focused on (in order of frequency mentioned):

  • Talking one-on-one with the clinician
  • Receiving books, texts or other resources
  • Help overcoming issues related to past experiences which had acted as barriers to moving forwards in their lives
  • Setting goals
  • “Anger management”
  • Discussing commencement of medication, or the need to adjust dosage
  • Referrals to or liaison with other services
  • Relaxation techniques (e.g., controlling breathing)
  • Journal writing
  • Mindfulness.

Clinicians reported that they saw offenders weekly or fortnightly but would sometimes see them more often when they were in crisis. The frequency of sessions generally reduced as an offender’s mental state improved.  However, clinicians reported that frequency of sessions sometimes had to be reduced for workload management reasons, as level of demand for the service at times was high.

Exiting the service

The majority of offenders who had received the service, and had since exited, engaged in up to five sessions with IMH clinicians over a period of up to five weeks.

For the IMH service in prisons, only 15% of offenders who exited were recorded as having “completed [5]” the service.  This relatively low level of completion reflected a range of factors which prematurely curtailed involvement of prisoners. Common events included offenders being released (22%), transferring to another prison (13%), or simply choosing to cease involvement (20%).  Completion rates were lower in the community; only 5% of offenders who engaged with the IMH service were recorded as having “completed” the service. Similar types of reasons for early discontinuation were observed: the offender declined to continue (26%), they breached conditions and were re-sentenced (21%), they failed to attend sessions (13%), or their sentence ended (11%).

While delivery of incomplete service is not ideal, it is reasonable to assume that a few sessions will nevertheless have effects superior to having offered no service at all.  It is also likely that, as the service matures, retention rates will improve, as will the operational efficiencies that enable referrals to be made earlier in the sentence.

The IMH service is available to offenders only whilst they are serving their sentence.  A number of offenders commented on the speed with which they were able to see the Corrections-contracted IMH clinicians, once referred, which compared favourably with long waiting times experienced in the past in accessing similar services when not under Corrections management.  However, this raised an issue of the feasibility of referring offenders on to other services once that offender’s sentence had ended.  Both clinicians and offenders mentioned anxieties about options for on-going support for those who needed it but who had completed their sentence.

Education of Corrections staff

The IMH clinician role allocates five hours per week for the education of Corrections staff.  This training is intended to build staff capacity to identify and respond productively to offenders who present with mental health needs. Lack of knowledge was seen as a barrier to staff recognising signs and symptoms that need to be reported to clinicians. Clinicians were of the view that upskilling Corrections staff in recognising mental health issues would improve the timing and suitability of referrals to the service. Corrections officers supported those views in their feedback.  IMH clinicians generally acknowledged that there was “room for development” in staff understanding of mental health issues.

Conclusion

The evaluation revealed that offenders and staff involved with the service were overwhelmingly positive about its value for improving offender mental health.  Evaluation evidence indicated that levels of psychological distress had been reduced for offenders who had completed the service.

The evaluation also identified some areas of service operation that could be improved. These included the need to increase the use of the electronic referral system, refinement of referral and triaging processes, the need to increase treatment completion rates, and issues associated with referrals of offenders once their sentences came to an end.

Despite challenges with the referral process, clinicians are working with offenders with mild to moderate mental health needs (such as anxiety and depression) as intended, and using a range of tools and techniques with them in individual treatment sessions. Offenders commented on the usefulness of these sessions and there are early indications from analysis of the Kessler10 assessments that offenders are showing reductions in levels of psychological distress. However, Kessler10 assessments are only being conducted for about one fifth of those who exit the service. This needs to increase substantially, as a measure of change in level of mental distress is not only informative for clinicians, but is also a useful indicator for monitoring the service.

[1] Auckland Region Women’s Corrections Facility, Auckland Prison, Christchurch Men’s Prison, Christchurch Women’s Prison, Manawatu Prison, Rolleston Prison, Otara Community Corrections, Mangere Community Corrections, Palmerston North Community Corrections.

[2] It is important to note that someone may present with a mild to moderate mental health need (e.g., depression) but experience a “high” or “very high” level of psychological distress.

[3] This data was entered as free text. A coding exercise was undertaken but 35% of this data was recorded as “other” so these findings should be treated with caution.

[4] Turning Point Alcohol and Drug Centre (2010). Screening and Assessment. Retrieved 4 January 2018

[5] Analysis of exit reasons is based on interpretation and coding of large volumes of free text entries and therefore these findings should be treated with caution. An exit reason was coded as “completed” if the free text entry indicated that the goals of treatment had been met.