Introduction to Data

INTRODUCTION TO DATA COLLECTION

The KPI Programme sources data from the Ministry of Health-held national database of mental health activity and outcomes, known as Programme for the Integration of Mental Health Data (PRIMHD), and directly from district health boards and non-government organisations that participate in the programme.

The KPI Programme is divided into a number of streams with each stream representing a specific aspect of mental health and addiction services, such as adult mental health services, child and youth mental health services, and adult forensic mental health services.  Participants of each stream nominate the indicators to be included in that stream's workbook, shape the definitions of the indicators, set target and alert levels, and specify relevant caveats.

Indicators are currently produced on an annual cycle aligned to the July-June financial year. Data collection takes place between July and September, covering the previous financial year.  Workbooks containing comparative data and charts for all participating organisations are produced and made available to participants shortly thereafter.  Workbooks of some selected streams are made available to the public early in the new calendar year. 

CAVEATS

  • The accuracy and completeness of the data is the responsibility of relevant participating organisations.
  • Indicators should be considered alongside additional contextual information and in clusters or themes.
  • Data quality may vary between participants.
  • Numbers are comparatively low in some districts, particularly for some ethnic groups, and this should be considered when interpreting data. 
  • There are considerable variations in resourcing and models of service delivery across the sector. Comparisons between like organisations are of the greatest value.

INDICATOR DEFINITIONS

Indicators are calculated by dividing a numerator by a denominator.  For example, average length of acute inpatient stay is defined as the total number of bed-nights relating to discharges during a given period divided by the number of discharges during the given period.  To facilitate collection of the relevant data, the numerator and denominator are also defined.  

SUPPLEMENTARY INDICATORS

The supplementary indicators provide contextual information alongside the core KPIs to help services better understand why their performance is what it is. Supplementary indicators are particularly helpful to understand whether variations in performance are due to different models of care, resourcing, population characteristics etc.

TARGETS AND ALERTS

Targets and alerts have been set by the participants of the New Zealand KPI Benchmarking Programme (adult stream, and child and youth stream). Targets provide an indication of good practice for each KPI. An organisation’s performance should be considered across a range of indicators rather than any one indicator in isolation.
Target ranges are depicted by a green band on charts (where a target has been set). Achievement of a target is depicted in tables by green highlighting of cells.
Alerts represent performance levels that should trigger investigation by the affected organisation. Alerts may be set above, below, or in the absence of target ranges and are depicted by red bands on charts (where alerts have been set).

" Liked the idea of developing my own benchmarking in my own service and have taken that away and talked with my team leaders about it and we are looking at what data we might want to collect next year to tell us whether we are doing a good job or not. " Adult Forum Participant


"I thought the discussion on the quantitative vs qualitative aspects of what KPIs meant was useful. The use of KPIs as vehicles of deeper understanding rather than as an answer in isolation was appreciated."  Adult Forum Participant

Benchmarking Forums