Indicators

“Numbers have an important story to tell. They rely on you to give them a clear and convincing voice.”

– Stephen Few

The KPI Programme’s indicators are measures designed to summarise and provide data-based insights into how Aotearoa’s mental health and addiction services are meeting the needs of tāngata whai ora and whānau. Based on sector priorities, the indicators enable Te Whatu Ora and NGO services across the county to examine their PRIMHD data in a comparable way so they can learn from each other and implement continuous improvement initiatives that can be monitored over time.

The Governments high level long-term plan for system transformation – Kia Manawanui Aotearoa (2021) identifies data and information as a core action for driving change and continual improvements that are community-led and responsive to the needs and experiences of Māori, tāngata whai ora, whānau. Integrating data across populations is essential to understanding people’s pathways through services. This is why, wherever possible, indicators will include visualisation tools that enable users to view data both within their communities of interest, as well as other population groups.

Data can be used by anyone to improve services. Using indicators in conjunction with local data collections, as well as feedback from tāngata whai ora, whānau and people working in services helps ensure the data is understood in the right context and informs continuous improvement actions that drive better wellbeing outcomes. The making a difference with data resources are available to refresh data and information use knowledge.

Data dashboards

Each indicator has dedicated data dashboard visualisation tools. The dashboards are interactive and allow users to filter and explore PRIMHD data relevant to each indicator. All indicator dashboards include data filtered by prioritised ethnicity, gender and age.

Click on any indicator to view the data or read more about the technical definitions.

Have you noticed a difference in your local data and the KPI Programme data dashboards? Visit the FAQ page to find out why.

Seclusion

Seclusion indicators by 100,000 population and per 1,000 bednights

Find out more

View data

Wait times

Days to first and third in-scope activity

Find out more

View data

7-day follow-up

Acute inpatient post-discharge community care

Find out more

View data

28-day readmission

Acute inpatient 28-day readmission rate

Find out more

View data

Continuity of Care

Tāngata whai ora with recorded NGO activity in the 28 days before an inpatient stay

Find out more

View data

Whānau engagement

Percentage of tāngata whai ora, who experience an in-scope activity, and had a whānau engagement activity recorded

Find out more

View data

Seclusion

Data source

PRIMHD and StatsNZ population projections

Technical notes and shared definitions

A seclusion activity has:

  • ActivityTypeCode = T33
  • ActivityUnitType = “SECLUSION”

and is recorded against a referral with:

  • TeamSetting = “Inpatient based”
  • TeamType of
    • Inpatient
    • Forensic
    • Intellectual Disability Dual Diagnosis
    • Speciality

A bednight activity (for the purposes of the seclusion KPI) has:

  • ActivityTypeCode not TCR
  • ActivityUnitType = “BEDNIGHT”
  • ActivityUnitCount > 0
  • And where there does not exist a LEAVE record for the same date
  • Same team restriction as above
  • Note that there are no checks on ActivityTypeCode against the TeamType

A bednight is:

  • A single midnight (00:00) that is crossed by a bednight activity
    For example, a bednight activity with start date 2020-01-20 17:30 and end date 2020-01-21 09:00 would have one bednight; a bednight with start date 2020-01-20 00:00 and end date 2020-01-21 09:30 would also have one bednight.
  • Where bednight activities have been recorded with both start and end times of 00:00, we count only one of those.
    For example, a bednight with start date 2020-01-20 00:00 and end date 2020-01-21 00:00 counts as only a single bednight. This is in line with the ActivityUnitCount calculated by MOH.

Total # bednights (sometimes seen as beddays) is the distinct count of bednights between the reporting start and end date. Where a single bednight activity crosses multiple reporting periods, only the individual bednights within the reporting period are counted.
For example, a bednight activity with start date of 2019-12-20 18:00 and end date of 2020-01-13 09:00 would have 11 bednights counted in the Oct-Dec19 quarter and 13 bednights counted in the Jan-Mar20 quarter.

Population = sourced from StatsNZ, Te Whatu Ora division projection for the reporting period

Seclusion activities are sometimes recorded differently due to varying business processes. To accommodate this, we create seclusion events or seclusion episodes in line with Ministry of Health recommendations.

Where there are fewer than 60 minutes between seclusion activities (for the same person and the same referral), these activities are rolled up into a single seclusion event.

A seclusion event combines overlapping or adjacent seclusion activities for an individual tangata whai ora within a single referral.

Indicator definitions

Total # seclusion events = distinct count of seclusion events where the seclusion event start date falls within the reporting period.

# tāngata whai ora secluded = distinct count of tāngata whai ora with a seclusion event where the seclusion event start date falls within the reporting period.

Seclusion events per 1000 bednights = Total # seclusion events / (Total # bednights / 1,000)

Seclusion events per 100k population = Total # seclusion events / (Population / 100,000)

Seclusion tāngata whai ora per 100k population = # tāngata whai ora secluded / (Population / 100,000)

Hours seclusion = sum of the hours of T33 seclusion activity that occur within the reporting period. When a single seclusion event crosses multiple reporting periods, only the hours within the reporting period are counted.
For example, a seclusion event with start date of 2019-12-28 18:00 and end date of 2020-01-02 09:00 would have 78 hours counted in the Oct-Dec19 quarter and 33 hours counted in the Jan-Mar20 quarter.

Additional notes

Seclusion events are allocated to a shift based on the seclusion event start time:

  • 00:00 – 06:59 night
  • 07:00 – 14:59 morning
  • 15:00 – 23:59 afternoon

Likewise, seclusion events are allocated to a weekday based on the seclusion event start date.

Note the fundamental distinction between seclusion events, which are allocated to a reporting period based on their start date, and seclusion hours and bednights, which are broken apart when they cross reporting boundaries and then apportioned pro rata. There are variations in these definition across the sector, so if you observe seclusion numbers that are close but not identical, this date logic may be a good place to check first.

A note about forensic data

Forensic seclusion events are summarised in a separate report. All forensic seclusion events and bednights are excluded from the national and Te Whatu Ora division overviews.

Hide this info

Wait times

Data source

PRIMHD

Technical notes

The process of service episode reconstruction is as follows:

1. Exclude any out of scope referrals, which are referrals where contact is not necessarily expected. These are any referrals that meet any of these criteria:

  • ReferralEndCode is RI, RO or DZ – referral was declined or discharged with no direct contact required
  • TeamType is 24 or 26 – Integrated Primary Access and Choice and Intellectual Disability teams
  • ReferralEndCode is in (DM, DG, DD, ID) AND there does not exist an in-scope activity on the referral – referral ended in a way that indicates contact may not have been expected, and there was no activity

2. Within each unique combination of tangata whai ora and organisation, combine all overlapping in scope referrals into service episodes. This can be achieved in various ways, but the KPI programme uses a ‘gaps and islands’ approach:

  • Sort referrals by Referral Start Date, Referral End Date, Referral ID
  • Calculate the Ranked Order for each referral (sorting by Referral Start Date, Referral End Date, and then Referral ID to tiebreak)
  • For each referral calculate the Previous End Date, which is the maximum Referral End Date of any referral with a lower Ranked Order (effectively the latest end date of any referral that started before this referral for this tangata whai ora at this organisation)
  • For each referral compare Referral Start Date to Previous End Date to determine whether this is an index referral or overlap
    • if referral Start Date <= Previous End Date this referral overlaps a previous referral and should roll into that service episode; set Index Referral?=0
    • if referral Start Date> Previous End Date this referral starts a new service episode; set Index Referral? =1
  • For each referral count the number of index referrals that have previously occurred for this tangata whai ora at this organisation, and append that value to the Organisation ID and HCU to form a globally unique service episode ID. For example the service episode for client ABC1234 at organisation G-0000 would be named G-0000_ABC1234_0; G-oo0o_ABC1234_1; etc.
  • Service Episode Start Date = Referral Start Date of the index referral in each service episode.

3. Within each service episode identify the earliest in scope activity on any referral in the service episode

a. Exclude all activities where either of these criteria are met:

    • ActivitySetting is one of these
      • WR Written correspondence
      • SM SMS text messaging
      • PH Telephone
      • OM Other social media, e-therapy
    •  ActivityType is one of these:
      • T08 Care/liaison coordination contacts
      • T24 Work opportunity/Employment/Vocational
      • T33 Seclusion
      • T35 Did not attend
      • T37 On leave
      • T43 Community support contacts
      • T44 Advocacy
      • T45 Peer support
      • T52 Health coaching contact
      • TCR MOH internal reference

b. Rank all remaining in scope activities by Activity Start Datetime, Referral ID, Activity ID

c. Calculate the Ranked Order for each in-scope activity (sorting by Activity Start Datetime, and then Referral ID and Activity ID to tiebreak)

d. Where Ranked Order = 1, this is the first in scope activity on the service episode

e. First In Scope Activity Start Datetime = Activity Start Datetime of this first in scope activity

f. Where Ranked Order = 3, this is the third in scope activity on the service episode

g. Third In Scope Activity Start Datetime = Activity Start Datetime of this first in scope activity

4. Calculate the wait time for each service episode:

a. Wait time to first in scope activity = difference in days between Service Episode Start Date and First In Scope Activity Start Datetime

b. Wait time to third in scope activity = difference in days between Service Episode Start Date and Third In Scope Activity Start Datetime

5. Calculate additional service episode metadata for use in analysis:

a. Service Episode End Date = maximum Referral End Date of all referrals within the service episode

b. Service Episode End Code = Referral End Code associated with the referral that has the maximum Referral End Date of all referrals within the service episode

i. Where multiple referrals share the maximum Referral End Date, if a DR exist then this is chosen. Otherwise the first alphabetical Referral End Code is chosen

c. Count Referrals = count referrals included in this service episode

d. Count Team Types = count of team types included in this service episode

e. Initial Team Type = Team Type of the index referral

f. Age at Service Episode Start = age in years on the Service Episode Start Date

g. In Scope Activity 365 Days Prior – Same Org? = if there exists an in-scope activity for this tangata whai ora at this organisation in the 365 days before Service Episode Start Date, them 1 else 0

h. In Scope Activity 365 Day Prior – Any Org? = if there exist an in-scope activity for this tangata wha ora at any organisation in the 365 days before Service Episode Start Date, then 1 else 0

i. Client Type =

i. If  In Scope Activity 365 Days Prior – Same org? = then “Recurring – same organisation” else

ii. If In Scope Activity 365 Days Prior – Any org? = 1 then “Recurring – another organisation” else

iii. “New”

6. Calculate additional activity flags for analysis:

a. Details of the first in scope activity based on its ActivityType

i. FirstInScopeAcitivityIsInpatient? = 1 if ActivityType in (T02, T03, T04)

ii. FirstInScopeActivityIsCommunityCrisis? = 1 if ActivityType in (T01, To5)

iii. FirstInScopeActivityIsCommunityNonCrisis? = 1 if ActivityType not in (T01, T02, T03, To4, T05)

iv. FirstInScopeActivityIsCommunityResidentail? = 1 if ActivityType in (T25, T26, T27, T28, T29, T30, T48)

v. FirstInScopeActivityIsCrisisorInpatient? = 1 if ActivityType in (T01, T02, T03, T04, T05)

b. Count out of scope activities before first in scope activity = count of all activities on a service episode where Activity Start Datetime < First In-Scope Activity Start Datetime and where the activity is not in scope (per the standard ActivityType definition)

Additional notes

An overlap is established by comparing referral start and end dates, not datetimes. For example, if Referral A ends at 09:30 on 01/01/2020 and Referral B begins at 23:30 on 01/01/2020, this is an overlap and these referrals will be combined. On the other hand, if Referral A ends at 23:30 on 01/01/2020 and Referral B begins at 00:30 on 02/01/2020, this is not an overlap and these referrals will be in separate service episodes (assuming there are no other referrals for this tangata whai ora).

When combining referrals, be careful not to simply sort referrals by start date and check for an overlap with the previous referral; tangata whai ora may have a single ongoing referral that overlaps multiple other brief referrals and forms a single service episode even though none of the brief referrals overlap one another.

Open referrals (Referral End Date = null) are included at step (1) regardless of whether an in-scope activity exists on that referral. This is to ensure that we reconstruct accurate service episodes, rather than discard pieces of a service journey.

The wait time is calculated using calendar days, not 24-hour periods. For example, a service episode starting at 09:00 on 01/01/2020 with the first in-scope activity at 23:30 on 01/01/2020 would have a wait-time of 0 days. A service episode starting at 23:30 on 01/01/2020 with first in-scope activity at 00:30 on 02/01/2020 would have a wait-time of 1 day.

There is no mechanism to categorise a service episode as urgent.

When identifying the Third in-scope activity, there is no correction for duplicate activities, and activities that occur on the same day are all considered individually. This is how a single service episode can have 0 days to both first and third in-scope activity, when three separate activities occur on the day of service episode start.

When allocating a service episode to a reporting period, we use the Service Episode Start Date.

Hide this info

7-day follow-up

Data source

PRIMHD

Description

Percentage of overnight discharges from the mental health and addiction service organisation’s inpatient unit(s) where a community service contact was recorded in the seven days immediately following that discharge.

This KPI calculates an overall follow up rate, which is the percentage of all acute inpatient discharges that were followed up, regardless of where that follow up occurred (Te Whatu Ora division, NGO or both)

Indictor rationale

A responsive community support system for persons who have experienced an acute psychiatric episode requiring hospitalisation is essential to maintain clinical and functional stability and to minimise the need for hospital readmission.

Service users leaving hospital after a psychiatric admission with a formal discharge plan, involving linkages with community services and supports, are less likely to need early readmission. Research indicates that service users have increased vulnerability immediately following discharge, including higher risk for suicide.

Denominator

Count of acute inpatient discharges

Numerator

Count of acute inpatient discharges where a follow up community contact (for the same person) exists where:

Community follow-up activity start date is between 1 and 7 days after acute inpatient discharge date

  • ActivityStartDate >= dateadd(1, day, InpatientDischargeDate)
  • ActivityStartDate < dateadd(8, day, InpatientDischargeDate)

Note: as of November 2020 terminology has changed from ReferralClosureDate to InpatientDischargeDate to eliminate confusion.

Technical notes

This denominator is shared with the other members of the acute inpatient KPI suite: 28-day readmission, length of stay, and pre-admission community contact.

General terminology

An acute inpatient discharge is any referral record where:

  1. ReferralEndDate is not null — ended referral
  2. TeamType is Inpatient — into an inpatient team
  3. ReferralEndCode is DR, DW, DT or DY — ended in a way where we expect follow-up
  4. ReferralTo is not PI, AE or NP — was not moving on to another hospital setting
  5. Exists at least one activity where — there was at least one acute inpatient bednight
    1. ActivityTypeCode is T02 or T03 — acute inpatient bednight codes
    2. ActivityUnitCount > 0 — for more than 0 days

An inpatient discharge date is calculated as the:

  1. Maximum ActivityEndDate for a referral record where: — end of last activity
    1. ActivityType is T02, T03, T04 or T37 — inpatient activity only

A community contact is any activity record where:

  1. TeamType is not Inpatient — not inpatient follow up
  2. ActivityUnitType is contact — not a bednight, seclusion or leave
  3. ActivitySetting is not WR, OM or SM — includes service user participation
  4. ActivityType is not T08, T35 or T32 — includes service user participation

Hide this info

28-day readmission

Data Source

PRIMHD

Description

Percentage of overnight discharges from the mental health and addiction service organisation’s acute inpatient unit(s) that result in readmission within 28 days of discharge.

This KPI calculates an overall readmission rate, which is the percentage of all acute inpatient discharges that were readmitted, regardless of where that readmission occurred (same Te Whatu Ora division or different division)

Indictor rationale

Psychiatric inpatient services aim to provide treatment that enables individuals to return to the community as soon as possible. Unplanned admissions to a psychiatric facility following a recent discharge may indicate that inpatient treatment was either incomplete or ineffective, or that follow-up care was inadequate to maintain the person out of hospital.

Denominator

Count of acute inpatient discharges

Numerator

Count of acute inpatient discharges where a readmission occurs within 28 days; that is where an activity exists (for the same person), where:

  • Referral team type is Inpatient — into an inpatient team
  • Activity type is T02 or T03 — acute inpatient bednight codes
  • Activity unit count > 0 — for more than 0 days
  • Activity start date is between 0 and 28 days after inpatient discharge date
    • ReadmissionActivityStartDate >= dateadd(0, day, InpatientDischargeDate)
    • ReadmissionActivityStartDate < dateadd(29, day, InpatientDischargeDate)

Technical notes

This denominator is shared with the other members of the acute inpatient KPI suite: 7-day follow-up, length of stay, and pre-admission community contact.

General terminology

An acute inpatient discharge is any referral record where:

  1. ReferralEndDate is not null — ended referral
  2. TeamType is Inpatient — into an inpatient team
  3. ReferralEndCode is DR, DW, DT or DY — ended in a way where we expect follow-up
  4. ReferralTo is not PI, AE or NP — was not moving on to another hospital setting
  5. Exists at least one activity where — there was at least one acute inpatient bednight
    1. ActivityTypeCode is T02 or T03 — acute inpatient bednight codes
    2. ActivityUnitCount > 0 — for more than 0 days

Hide this info

Continuity of Care

Data source

PRIMHD

Description

‘Continuity of care’ as indicated through NGO engagement before, during and after acute inpatient Te Whatu Ora division stays.

The indicator is a measure to support services understand continuity of care between Te Whatu Ora division and NGO mental health services so they can identify where improvement is required to contribute to better wellbeing outcomes for tāngata whai ora who access acute inpatient Te Whatu Ora division services.

The indicator includes data for tāngata whai ora with recorded NGO activity in the 28 days before an acute inpatient stay.

Indicator rationale

Tāngata whai ora report inconsistencies in the connectedness, or continuity of care they receive between Te Whatu Ora division and NGO mental health services.

Denominator

Open NGO referral at time of admission to inpatient unit.

Referral Start Date at least one day before, Referral End Date at least one day after an inpatient stay.

Activity pre inpatient – 1 to 28 days before inpatient admission.

Numerator

Activity within inpatient stay – inpatient admission to discharge date.

Activity post community care – 1 to 8 days after inpatient discharge.

Inpatient criteria

Meet the criteria for post discharge community care (7-day follow up).

‘Referral from’ is not psychiatric inpatient (PI).

Activity data selected for indicator inclusion

  1. ActivityTypeCode not in T08, T35, T32
  2. ActivitySettingCode not in WR, OM, SM
  3. ActivityUnitType=CONTACT
  4. TeamType=Community Team
  5. NGOs.

Current data exclusions / known limitations

  • Services focused on housing and employment currently excluded.
  • Phone based services currently excluded from the data.
  • Residential services currently excluded.
  • The current ‘within’ category includes the day of admission and discharge. Inclusion could infer that NGO engagement happens on the day of admission or discharge.
  • If there are more than two NGOs with a referral open at inpatient admission and with activity in the 28-days before admission, they can be counted more than once.

Hide this info

Whānau engagement

Data source

PRIMHD

Technical notes

Percentage of tāngata whai ora with whānau engagement

  1. Activity start date is >= start date of selected quarter and <= end date of selected
  2. Tāngata whai ora with an activity
    • Activity type is not
      • T35 Did not attend
      • T08 Care/liaison co-ordination contacts
    • Activity setting is not
      • WR – Written correspondence
      • SM – SMS text messaging
      • OM – Other social media/E therapy
    • Activity Unit Type is CONTACT
    • Team setting is not I Inpatient
    • Team type is not
      • 01 Inpatient team
      • 05 Forensic team
      • 08 Residential/accommodation team
      • 26 Intellectual disability
  1. Tāngata whai ora with whānau activity
    • Activity Type is one of:
      • T32 – Contact with family/whānau, consumer not present
      • T36 – Contact with family/whānau, tāngata whai ora/consumer present
    • and/or Family/whānau involvement = 1 Yes
      • Activity type is not T35 did not attend
    • Activity setting is not
      • WR – Written correspondence
      • SM – SMS text messaging
      • OM – Other social media/E therapy
  1. % of tāngata whai ora with whānau engagement in a quarter = Tāngata whai ora with whānau activity/ tāngata whai ora with an activity

First whānau engagement activity in episode of care

The process of service episode reconstruction is as follows:

1. Exclude any out of scope referrals, which are referrals where contact is not necessarily expected. These are any referrals that meet any of these criteria:

  • ReferralEndCode is RI, RO or DZ – referral was declined or discharged with no direct contact required
  • TeamType is 24 or 26 – Integrated Primary Access and Choice and Intellectual Disability teams
  • ReferralEndCode is in (DM, DG, DD, ID) AND there does not exist an in-scope activity on the referral – referral ended in a way that indicates contact may not have been expected, and there was no activity.

2. Within each unique combination of tangata whai ora and organisation, combine all overlapping in scope referrals into service episodes. This can be achieved in various ways, but the KPI programme uses a ‘gaps and islands’ approach:

  • Sort referrals by Referral Start Date, Referral End Date, Referral ID
  • Calculate the Ranked Order for each referral (sorting by Referral Start Date, Referral End Date, and then Referral ID to tiebreak)
  • For each referral calculate the Previous End Date, which is the maximum Referral End Date of any referral with a lower Ranked Order (effectively the latest end date of any referral that started before this referral for this tangata whai ora at this organisation)
  • For each referral compare Referral Start Date to Previous End Date to determine whether this is an index referral or overlap
    • if referral Start Date <= Previous End Date this referral overlaps a previous referral and should roll into that service episode; set Index Referral?=0
    • if referral Start Date> Previous End Date this referral starts a new service episode; set Index Referral?=1
  • For each referral count the number of index referrals that have previously occurred for this tangata whai ora at this organisation, and append that value to the Organisation ID and HCU to form a globally unique service episode ID. For example the service episode for client ABC1234 at organisation G-0000 would be named G-0000_ABC1234_0; G-oo0o_ABC1234_1; etc.
  • Service Episode Start Date= Referral Start Date of the index referral in each service episode.

3. Within each service episode identify the earliest in scope activity on any referral in the service episode

  • Exclude all activities where either of these criteria are met:
    • ActivitySetting is one of these
      • WR – Written correspondence
      • SM – SMS text messaging
      • OM – Other social media, e-therapy
    • ActivityType is one of these:
      • T08 Care/liaison coordination contacts
      • T24 Work opportunity/Employment/Vocational
      • T33 Seclusion
      • T35 Did not attend
      • T37 On leave
      • T43 Community support contacts
      • T44 Advocacy
      • T45 Peer support
      • T52 Health coaching contact
      • TCR MOH internal reference.
  • Rank all remaining in scope activities by Activity Start Datetime, Referral ID, Activity ID
  • Calculate the Ranked Order for each in-scope activity (sorting by Activity Start Datetime, and then Referral ID and Activity ID to tiebreak)
  • Where Ranked Order = 1, this is the first in scope activity on the service episode
  • First In Scope Activity Start Datetime = Activity Start Datetime of this first in scope activity This determines the time period for analysis.

4. Find the first whānau engagement datetime activity

  • Activity Type is one of:
    • T32 – Contact with family/whānau, consumer not present
    • T36 – Contact with family/whānau, tāngata whai ora/consumer present
  • and/or Family/whānau involvement = 1 Yes
    • Activity type is not T35 did not attend
  • Activity setting is not
    • WR – Written correspondence
    • SM – SMS text messaging
    • OM – Other social media/E-therapy

5. Calculate additional service episode metadata for use in analysis:

  • Initial Team Type = Team Type of the index referral
  • Age at Service Episode Start = age in years on the Referral Start Date

6. Calculate the difference between the First in Scope Activity Start Datetime and first whānau engagement datetime. For each episode group as follows

  • 14 days in under (Potential for a few negative number not an in scope activity)
  • 15-28 days
  • 29+ days
  • No whānau contact

7. Exclude initial team type

  • 01 Inpatient team
  • 05 Forensic team
  • 08 Residential/accommodation team
  • 26 Intellectual disability

8. Only include episodes with a third in scope activity.

Hide this info