Ministry of Justice: Modernising court services

Presented to the House of Representatives under section 20 of the Public Audit Act 2001.

December 2017

ISBN 978-0-478-44281-6

B.29[17t]

Contents

Overview

Our recommendations

Part 1 – Introduction

Why we did the audit

What we looked at

What we did not look at

How we did our audit

The structure of this report

Part 2 – Improving court services

Why improving court services is important

The complex environment the Ministry works in

Development of the Ministry’s approach to improving court services

Intended improvements of the three projects

Part 3 – Measuring the improvements made

Why measuring achievement of improvements is important

Improvements have not been measured

Limited information indicates some improvement

What improvements people say have been made

Part 4 – Engaging with affected people and organisations

Why engaging with people and organisations is important

Effective engagement supported implementation of projects

A lack of engagement contributed to difficulties

Collaboration between organisations has been limited

Part 5 – Governance and accountability for the three projects

Governance and accountability is important

Limited evidence of formal approvals

Inconsistent reporting was not always addressed

Lack of engagement with affected people and organisations was not adequately addressed

Part 6 – Improving for the future

Changes to the Ministry’s governance of projects

Improving how feedback is collected

Establishing effective performance measures

Figures

1 – Difference between appearing in court in person and using Audio-Visual Links

2 – The intended improvements of centralising dealing with applications to dispute a fine

3 – The intended improvements of centralising dealing with civil claims

4 – Comparing two different calculations of the usage rate of Audio-Visual Links in Christchurch District Court, September 2014 to May 2017

5 – The Ministry of Justice’s project governance structure since 2014

Overview

To ensure that justice is not compromised, the courts must be, and must be seen to be, separate and independent from executive government. An important function of the courts is the public administration of people’s right to open justice according to the law. The courts operate on a rules-based system, and processes must meet legislative requirements.

A wide variety of people and organisations participate in the delivery of justice. These include judges, lawyers, the defendant and accused for criminal cases, and government organisations such as the Ministry of Justice, the New Zealand Police, and the Department of Corrections.

Two important roles of the Ministry of Justice are providing support to the judiciary and administering court systems for both criminal and civil cases. The Ministry provides a range of services, including criminal history checks, collecting fines, and public defence.

Court services have largely been provided in courthouses and been paper-based. However, in recent years, the Ministry has introduced a wide range of initiatives to improve the efficiency of court services and people’s experience of them.

When making changes to court processes, the Ministry needs to engage with a lot of people and organisations, which can be challenging. The Ministry’s aim is to provide a modern, effective, and trusted justice system that contributes to a safe and just New Zealand.

We carried out a performance audit to assess the Ministry’s investment in improving court services by looking at three projects between 2012 and 2016 that were part of this approach.

Some people’s experience of using court services has improved after the Ministry’s investment. However, the Ministry has not effectively tracked or measured what improvements have been made.

Although there have been some indications of improvements to court services, the lack of accurate information means that the Ministry does not have a reliable or complete understanding of how effective its investment has been or what the challenges to delivering further improvements are. This means that the Ministry is not likely to have achieved the return on investment that it could have.

People who we spoke to had different views of what improvements have been made. For example, there was general agreement that improvements had been made for people disputing infringement fines and that remote court appearances using Audio-Visual Links had improved court security and the welfare of people in custody awaiting trial.

However, there was also some disagreement about what had been achieved. For example, there was disagreement between the Ministry and some organisations about whether services for dealing with civil claims had improved.

The effectiveness of project management varied between the three projects we looked at. For example, when the Ministry consulted and communicated effectively with affected people and organisations, implementation of the project went well. On the other hand, when there was not enough consultation and communication with affected people and organisations, there were some difficulties during and after implementation.

Weaknesses in project governance and accountability meant that it was unlikely decision-makers had enough information to oversee the projects effectively.

The Ministry had limited processes for receiving feedback from people and organisations affected by the projects. The Ministry largely relied on one-to-one relationships and local meetings run by its staff in District Courts. These were useful for addressing localised and individual issues, but they did not give the Ministry good oversight of significant issues created by changes to processes that affect more than one court, so that the Ministry could effectively address these issues.

For the Ministry to achieve greater improvements from its investments in improving court services, we recommend that it set up a structured approach to measuring and reporting on achievement of improvements. This should provide the Ministry with accurate information to determine the extent to which improvements have been achieved and whether further changes to improve services are needed.

To help maintain good oversight of emerging issues, the Ministry should ensure that it has effective processes for collecting, collating, analysing, and reporting on feedback from organisations and people affected by changes to court services resulting from projects.

The Ministry told us that, since 2014, it has made substantial changes to internal governance and processes for investment and that further efforts to improve are ongoing. We did not audit the changes the Ministry introduced to improve project governance and accountability because they were put in place after the three projects we looked at were implemented.

In our view, it is important that the Ministry makes sure that its changes to governance and project management practices ensure adequate levels of accountability and governance over projects.

I thank the Ministry’s staff for their help and co-operation during our audit. I also thank people in the justice sector, including people from the Department of Corrections, the New Zealand Police, private and public defence lawyers, the judiciary, and other organisations for their co-operation.

Greg Schollum
Deputy Controller and Auditor-General

13 December 2017

Our recommendations

We recommend that the Ministry of Justice:

ensure that it has effective processes to collect, collate, analyse, and report on feedback from people and organisations that are affected by changes to court services resulting from projects; and

prepare a performance measurement and reporting framework, including a timetable, to track whether the intended improvements from its investments in projects to improve court services are being achieved.

Part 1 – Introduction

1.1
In this Part, we discuss:

why we did the audit;

what we looked at;

what we did not look at;

how we did our audit; and

the structure of the report.

Why we did the audit

1.2
Courts have a distinct role that requires them to be kept separate from the executive arm of government (for example, the Ministry of Justice). Maintaining the independence of the courts is important for ensuring that justice is not compromised.¹ The courts must be, and must be seen to be, separate and independent.

1.3
Two important roles of the Ministry of Justice (the Ministry) are supporting the judiciary and administering the court systems for both criminal and civil cases. The Ministry provides a range of services that are aimed at contributing to a safe and just New Zealand. These services include:

criminal history checks;

collecting fines;

legal aid;

public defence;

transcription; and

security in courts.

1.4
The Ministry is also the lead justice sector organisation.² This means that the Ministry is responsible for co-ordinating and encouraging collaboration between different justice sector organisations.

1.5
We did this audit because the Ministry’s work to improve court systems has involved a significant range of changes to court systems in recent years. Given the importance of the courts, we wanted to provide assurance to Parliament and the public about how these changes were implemented and what improvements have been achieved.

What we looked at

1.6
We carried out a performance audit to assess the Ministry’s investment in modernising courts to improve efficiency and people’s experience of the courts.

1.7
We examined how effectively the Ministry managed its investment in three projects between 2012 and 2016. The three projects we looked at were Audio-Visual Links (AVL), centralising dealing with applications to dispute a fine under section 78B of the Summary Proceedings Act 1957 (applications to dispute a fine), and centralising dealing with Notices of Civil Proceedings (civil claims, which involve claims to recover money).

1.8
In this report, we refer to these as the three projects. A focus of the three projects has been improving customer service and efficiency by streamlining the services the Ministry provides.

1.9
For each project, we looked at:

the project management practices used to implement each project; and

whether the intended improvements had been met or were likely to be met.

1.10
We wanted to estimate the return on investment each project has made, which would enable us to assess the Ministry’s management of its investment.

Why we chose the three projects

1.11
AVL enables people in custody awaiting trial to attend short court appearances by video from their Corrections facility, instead of appearing in court in person.

1.12
We chose to look at AVL because it:

required the Ministry and the Department of Corrections (Corrections) to work together;

affected many people and justice sector organisations (for example, people in custody awaiting trial, the judiciary, lawyers, and New Zealand Police);

involved an investment in physical assets, particularly audio-visual equipment;

involved changes to business processes; and

was considered an important step in modernising the courtroom.

1.13
We also looked at the Ministry’s projects to centralise two functions dealing with applications to dispute a fine and dealing with civil claims. Centralising involves moving some court functions from the 58 District Courts throughout the country to a single location – the Central Registry.

1.14
We looked at centralising these court functions into the Central Registry because they:

were an important part of changing the way the Ministry interacts with people who use court services;

were expected to improve the efficiency of court processes, including providing consistent customer experiences throughout the country; and

involved standardising some court systems and processes.

1.15
We chose to look at centralising dealing with applications to dispute a fine, and dealing with civil claims because they had:

the highest volume of tasks that have been centralised;

been carried out by 58 District Courts; and

been centralised between 2012 and 2014, which provided enough time for improvements to be made and measured.

1.16
We provide more information about each of the projects we audited in Part 2.

What we did not look at

1.17
We did not audit the performance of the judiciary, lawyers, or any organisations in the justice sector other than the Ministry. We did not assess the effectiveness of the administrative support provided to courts or tribunals.

1.18
Although we discuss the Ministry’s approach to modernising courts in Part 2, we did not audit its overall effectiveness.

How we did our audit

1.19
To carry out our audit, we reviewed project management documents for the three projects. These included:

business cases;

project initiation documents;

benefit realisation plans;

project status reports;

minutes from meetings;

implementation plans; and

estimated costs.

1.20
We interviewed staff from the Ministry’s head office involved in the three projects. We wanted to understand the internal processes used to project manage the three projects and how the Ministry tracked whether the intended improvements were achieved.

1.21
We spoke to a range of staff who worked at the Central Registry, Auckland District Court, Manukau District Court, North Shore District Court, Waitakere District Court, Rotorua District Court, and Tauranga District Court.

1.22
We interviewed the judiciary and a range of people and organisations working in the justice sector that are affected by changes to how the courts operate. They included District Court judges, private lawyers, public defence lawyers, the New Zealand Police (the Police), and Corrections.

1.23
We talked to a range of organisations that were affected by centralising dealing with applications to dispute a fine and dealing with civil claims, such as two prosecuting agencies (Auckland Transport and the Police), the Inland Revenue Department, and credit and finance companies.³ We collated their experiences of these changes to the process to develop an understanding of some organisations’ experiences of the changes.

1.24
We observed the use of AVL in Tauranga District Court, Rotorua District Court, Auckland District Court, and Manukau District Court. We also met with Correction’s staff at Mt Eden Corrections Facility and Waikeria Prison.

1.25
We looked at the Ministry’s financial information to calculate the return on investment for each project and track the value of the improvements made.

1.26
We examined the Ministry’s analysis of the data it had collected for each project. This included determining how effective the Ministry’s measures were and whether the Ministry used the data it collected to highlight areas where further improvements might be possible.

The structure of this report

1.27
In Part 2, we describe the environment the Ministry operates in and its efforts to improve court services. We also describe how the three projects have made some changes in the delivery of court services.

1.28
In Part 3, we examine whether the Ministry has tracked and measured the improvements made. We look at what information the Ministry has recorded and what improvements people say have been made.

1.29
In Part 4, we examine how the Ministry engaged with organisations and people affected by its investment to improve court services.

1.30
In Part 5, we examine the Ministry’s governance and accountability for the three projects.

1.31
In Part 6, we look at changes the Ministry told us it has made to its project governance structure and processes. We also recommend improvements to how feedback is collected and how improvements are measured.

1 For an overview of the role of courts and the judiciary, see courtsofnz.govt.nz.

2 Organisations in the justice sector include the Ministry, the Department of Corrections, the New Zealand Police, the Crown Law Office, the Serious Fraud Office, and the Ministry of Social Development (for youth justice).

3 Throughout this report, when referring to people who are involved in applications to dispute a fine or civil claims, we are specifically referring to these organisations unless we state otherwise. We acknowledge that the Ministry works in a complex environment and that these people do not reflect the experience of everyone involved. For example, it does not include members of the public.

Part 2 – Improving court services

2.1
In this Part, we discuss:

why improving court services is important;

the complex environment the Ministry works in;

the development of the Ministry’s approach to improving court services; and

the intended improvements of the three projects.

Why improving court services is important

2.2
Typically, people using court services can feel vulnerable, stressed, or angry. Courts can be confusing places. An important function of the courts is the public administration of open justice according to the law.

2.3
Courts operate on a rules-based system, and processes must meet legislative requirements. This is the case whether paper-based or electronic systems are used. Most court services in New Zealand are paper-based. However, technology can transform how services are delivered and could improve the delivery of court services.

2.4
The Ministry wants to provide modern, accessible, and people-centred justice services. This has included looking to improve the quality and speed with which services are delivered. The Ministry considers modernising courts to be part of a wider goal of improving people’s trust and confidence in the justice system.

The complex environment the Ministry works in

2.5
The courts uphold the law, and the disputes resolved by courts touch virtually all aspects of life in New Zealand. Important constitutional safeguards that enable people to have confidence in the courts must be protected when any change to the court system is made. These safeguards protect:

the rule of law;

the importance of judicial independence and impartiality;

open justice; and

everyone being equal before the law.

2.6
Courts rely on a range of people and organisations to operate. For criminal cases, this includes such court participants as judges, defence lawyers, and prosecutors. The Ministry has a role in supporting the judiciary to co-ordinate between the different parties to ensure that events in court can proceed as planned. A case is unlikely to go ahead until all the people involved are ready to proceed.

2.7
There needs to be a degree of operational independence between the different people and organisations. The prosecution, defence, and judiciary must not be able to influence each other. The independence of the judiciary and of the courts is a crucial component of a fair and impartial justice system and must be protected.

2.8
When making changes to court processes, the Ministry needs to work with a lot of different people and organisations. Modernising courts often also involves changes to legislation. Activities or changes in one part of the criminal justice system can have significant effects in other parts.

2.9
The introduction of AVL has affected every part of the criminal justice system. To take advantage of the improvements, the Ministry and Corrections needed to work together. This included introducing new technology and aligning business processes between them. The appropriateness of using AVL is for judges to decide. There are times when judges might not consider the use of AVL to be appropriate.

2.10
Moving some functions to the Central Registry is intended to make it easier for people and organisations, including small to medium-sized businesses, to use some court services. This includes dealing with applications to dispute a fine
and dealing with civil claims.

Development of the Ministry’s approach to improving court services

2.11
In 2012, the Ministry wanted to provide modern, accessible, and people-centred justice services. To achieve this, the Ministry focused on three main changes between 2012 and 2015. The Ministry said that these were from:

being “location focused” to “customer focused”;

having “complex, paper-based processes” to “simpler, standardised processes”; and

having “money in buildings and staff” to “investment where it’s needed”.⁴

2.12
To make these changes, the Ministry set out initiatives that covered:

enabling legislation, which included the Judicature Modernisation Bill;

process improvement, which included centralising some tasks;

infrastructure and technology, which included technology for AVL; and

modern services, which included making some processes digital.

2.13
In 2013, the Ministry started the modernising courts programme, which was a large transformation programme. Between 2014 and 2015, the Ministry narrowed the scope of its work to primarily focus on centralising some court functions and standardising the delivery of its services throughout New Zealand.

2.14
Performance improvement framework reviews in 2012 and 2014, and our annual audit in 2014/15, raised concerns about the Ministry’s performance in modernising courts. These concerns included:

needing to develop a picture of the desired result and clarity about the sequence of events necessary to achieve it;

needing to work with the judiciary;

needing to differentiate between the different organisations and people who use court services and what their needs are; and

the frequency with which the scope of the work the Ministry was doing to modernise courts changed.

2.15
In 2015, the Ministry changed its approach to modernising courts from a programme to a portfolio. The Ministry decided to treat modernising courts as a series of initiatives, recognising that the work was fluid and susceptible to change.

2.16
The series of initiatives were split into four areas that were each tied to an investment goal. The Ministry said the purpose of the four areas was to give decision-makers oversight of whether adequate progress was being made. These areas were:

reduce the length of time it takes to hear and resolve matters through the courts – for example, improving rostering and scheduling, and tribunals’ case management;

improve the user experience – for example, enabling a text messaging service for defendants and constructing the Christchurch Justice Emergency Service Precinct;

simplify and standardise to improve productivity and efficiency – for example, centralising tasks and standardising services; and

reduce dependency on physical location – for example, video capability and developing a customer-friendly website.

2.17
The Ministry believed that taking a portfolio approach better reflected its view that modernising courts involved continuous improvement. The Ministry told us that this approach also recognised that the Ministry does not have direct influence over, or control of, all aspects of court systems.

2.18
Although our audit focused on the implementation of the three projects between 2012 and 2016, we observed that the development of the Ministry’s approach, efforts, and activities to modernise courts changed a lot between 2012 and 2017.

2.19
In our 2014/15 annual audit opinion, we said that the period of prolonged uncertainty about scope had risked time and money being invested without a clear outcome. In our view, the Ministry’s frequent changes in approach to modernising courts may have created uncertainty for organisations about the value of the Ministry’s investment.

2.20
Although the approach to modernising courts was changing, our 2014/15 annual audit also said that the Ministry had displayed an inability to scope, manage, and monitor some significant projects. Given the broader concerns about the Ministry’s performance in modernising courts, we wanted to have a closer look at how effectively the Ministry had managed its investment by examining three projects between 2012 and 2016.

Intended improvements of the three projects

Audio-Visual Links

2.21
After a successful trial in October 2012, the Ministry wanted to expand the use of AVL by implementing it in 14 more District Courts. This expansion project, a joint venture between the Ministry and Corrections, was part of a broader investment in infrastructure and technology.

2.22
The business case established that these courts complemented Corrections’ investment in developing its network of Corrections facilities with AVL. The selected courts were considered to have the greatest level of improvements for Corrections, the Police, and the Public Defence Service.

2.23
At the start of the AVL project, the business case set out a range of intended improvements. These improvements were based on the Ministry’s benefits realisation plan for justice sector organisations, which the Ministry prepared in November 2011 with input from Corrections and the Police. The business case expected that the benefits realisation plan would be updated to include the additional courts with AVL.

2.24
The intended improvements from AVL for justice sector organisations, lawyers, and judges were:

increasing safety for the public and people in custody;

minimised risk of people in custody escaping;

reducing the risk of contraband entering Corrections facilities;

cost savings for Corrections, primarily with transport costs;

reducing court time and hearings; and

reducing the amount of time lawyers spent travelling to consult with people in custody.

2.25
Court hearings using AVL typically take less than 15 minutes. Figure 1 describes the differences between appearing in court in person and using AVL. The types of court hearings include taking a plea, bail hearings, and setting the trial date.

2.26
The Ministry carried out a literature review and found that countries such as Australia, the United Kingdom, Canada, and the United States of America used AVL for similar purposes.⁵

Figure 1
Difference between appearing in court in person and using Audio-Visual Links

For a person in custody, appearing at court in person involves getting up early and being subjected to a rub-down and metal detector body search before leaving the Corrections facility. To ensure the lowest number of transfer journeys, people in custody are transported together in vans in the morning, often regardless of what time of day the court hearing is scheduled. Travelling to court could be a short journey, or it could take several hours.

Once people in custody arrive at court, they are held in holding cells while they wait for their court hearing. At the end of the day, depending on the outcome of their court hearing, defendants return to the Corrections facility, where they are subject to strip-searching on re-entry, as required by legislation, and may go to a new cell in a different unit. This leaves people in custody having to find their social place in a unit again. If someone is entitled to be released from custody – for example, because they have been granted bail – they may be free to leave from the court.

For a person in custody, using AVL means they can remain in their unit and follow their usual routine, including attending any activities, such as literacy, numeracy, or chess. A Corrections officer will escort the person in custody to the area of the Corrections facility that provides AVL, and they will wait in a holding cell.

The person in custody needs to be ready 30 minutes before their scheduled remote court appearance. After their remote court appearance, they are escorted back to their unit, where they resume their normal daily routine. If someone is entitled to be released from custody – for example, because they have been granted bail – they are released once the necessary paperwork has been received from the relevant District Court, which can take a while.

Centralising tasks

2.27
To centralise some tasks previously carried out in District Courts throughout the country, a single process for these tasks needed to be agreed. This included developing decision-making trees that support consistent decision-making by deputy registrars at the Central Registry.

2.28
This can be challenging when each District Court previously had its own interpretations of, and ways to carry out, the rules that determine how tasks should be processed. District Courts also varied in how they resourced these tasks.

Centralising dealing with applications to dispute a fine

2.29
Centralising dealing with applications to dispute a fine was part of a bigger programme – the Courts and Criminal Matters Bill. In this programme, the Ministry saw centralising dealing with applications to dispute a fine as a business enabler.

2.30
In a detailed design document for the project, the Ministry said the project was to:

provide a better service to prosecuting agencies and people with fines that have been sent to the District Court; and

transfer responsibility for processing applications to dispute a fine from the District Courts to the Central Registry.

2.31
Changes to legislation were required to achieve the intended improvements. These changes included addressing loopholes that prosecuting agencies had complained about and introducing simpler, more efficient, and user-friendly processes.

2.32
Figure 2 describes the intended improvements of centralising dealing with applications to dispute a fine. At the start of the project, the Ministry said that the intended improvements were:

a more efficient and transparent system that would lead to better outcomes for people and improve staff productivity; and

a more efficient use of staff time because of closing a legislative loop hole.

Figure 2
The intended improvements of centralising dealing with applications to dispute a fine

Before centralisation, people who wanted to dispute a fine had to post or personally deliver their application to their District Court. Sometimes, this meant that individuals had to take time off work to make the application. Applications could involve a great deal of paper once all the necessary documentation and evidence had been provided.

At the District Court, a deputy registrar would consider whether the application met any of the grounds set out in law for removing the fine from court and returning it to the prosecuting agency to settle the matter. As part of the decision-making process, the deputy registrar would typically send the application to the relevant prosecuting agency for its response. Some prosecuting agencies dealt with several District Courts, and we were told that each court had its own process.

How long it took to deal with applications in each court varied, with applicants sometimes waiting up to six weeks before they received a response. Prosecuting agencies could wait for two to three weeks before they heard whether the District Court had decided to uphold the application or not.

By centralising dealing with applications to dispute a fine, the Ministry wanted to simplify forms and the process. People would be able to complete the form and email their application directly to the Central Registry.

The Central Registry, which discharges some District Court responsibilities, would become the single point of contact for prosecuting agencies, instead of the 58 District Courts throughout New Zealand.

Civil claims

2.33
Dealing with civil claims was one of five functions that the Ministry decided to centralise. The Ministry established three high-level investment aims for centralising these five functions in its business case, which were:

developing an operating approach that supports increased transparency of the Ministry’s performance, performance improvement, and quality;

improving consistency of administrative processes to maximise the similarities (people, processes, and technology) between services; and

putting in place a performance culture that enables the Ministry to continuously improve performance.

2.34
The Ministry said that the intended improvements from centralising the five functions were:

consistent customer service;

improved internal consistency;

a culture of continuous improvement;

freeing up the time of Ministry staff in the District Courts; and

reducing the number of full-time equivalent District Court staff through attrition.

2.35
The Ministry also expected that centralising these functions would contribute to its broader goal of reducing service delivery time by 50%. No specific intended improvements for centralising dealing with civil claims were established.

2.36
Centralising dealing with civil claims involved centralising dealing with Statements of Claim and applications for Judgment by Default.

2.37
A Statement of Claim is typically the document that a person or organisation uses to start civil action to settle a dispute. These civil disputes are typically about money. We were told that large creditors, including credit and finance companies and the Inland Revenue Department, make most of these claims. These creditors may represent clients or claim that money is owed to them. For example, they may pursue debts on behalf of small businesses or claim unpaid motor vehicle finance.

2.38
An application for a Judgment by Default can be made when there has been no response or action by a defendant after a specified time following the filing of a Statement of Claim and the issuing of a notice of proceedings. The Central Registry also deals with applications for Judgment by Default. However, if deputy registrars at the Central Registry are unsure about what judgment to make, the claim is returned to the relevant District Court for a judge to decide.

2.39
Figure 3 describes the intended improvements of centralising dealing with civil claims.

Figure 3
The intended improvements of centralising dealing with civil claims

Normally, District Court staff deal with Statements of Claim and applications for Judgment by Default. We were told that dealing with these claims was time-consuming and involved a lot of paperwork. There was also no consistency between the various District Court deputy registrars in the decision-making process, the outcomes of decisions, and the length of time it took to deal with the claims.

By centralising dealing with Statements of Claims and applications for Judgment by Default, the Ministry wanted to increase the speed of dealing with these claims, provide greater consistency in decision-making by deputy registrars, and make it easier for people recovering money to know who to contact if they have a query about their application.

4 Ministry of Justice, Modernising Courts and Tribunals 2012-2015.

5 Ministry of Justice (2016), Optimising Audio-Visual Links between court and custody: A literature review.

Part 3 – Measuring the improvements made

3.1
In this Part, we discuss:

why measuring achievement of improvements is important;

how improvements have not been measured;

how limited information indicates some improvement; and

what improvements people say have been made.

Summary of our findings

3.2
The Ministry has not taken a structured approach to measuring and reporting improvements from the three projects. Therefore, the Ministry cannot accurately determine how effective its investment has been or what the challenges to making further improvements are.

3.3
Limited information about the Ministry’s overall performance and the views of some people using these services indicate some improvements to these court services. However, the Ministry and some organisations disagreed about what improvements have been made. In our view, the lack of accurate information about the improvements made is likely to have contributed to this.

Why measuring achievement of improvements is important

3.4
We expected the Ministry to have set out the improvements expected from each project and to track and report on achieving those improvements over time.

3.5
Measuring improvements is important because the Ministry can use this information to see whether the projects are achieving the desired results. The Ministry can also use this information to make informed decisions about whether it needs to make further changes.

3.6
Measuring improvements also means that the Ministry can demonstrate to others what the projects are achieving. This can help increase organisations’ and the public’s confidence in the Ministry’s ability to improve court services and in its leadership of the justice sector.

3.7
The Ministry can also use this information to provide Parliament, people and organisations involved in the justice sector, and members of the public with an account of how effective the Ministry’s investment in modernising courts has been.

Improvements have not been measured

3.8
The Ministry did not take a structured approach to measuring and reporting on the achievement of improvements for the three projects. This means that the Ministry cannot accurately determine whether its investments have been effective and what the challenges to making further improvements are.

3.9
The Ministry identified the improvements it expected for each of the three projects. It could have used this information to determine how successful each project has been in achieving its goals over time.

3.10
However, the Ministry did not put in place performance measurement frameworks to see whether improvements had been made. Therefore, it is not possible for the Ministry to accurately determine how effective its investment in these three projects has been.

3.11
The lack of accurate information about whether the three projects made improvements also meant that we were unable to calculate a return on investment for any of them.

Limited information indicates some improvement

3.12
Because the Ministry did not measure and report on improvements from the three projects, we looked at other information that could provide some insight.

Information about the Ministry’s overall performance in its annual report

3.13
The Ministry has outcome and performance measures in its annual report that provide a high-level account of its performance during the past year. We looked at two areas in the report: “Increased trust in the justice system” outcome measures and “Vote Courts” performance measures. These measures indicated that there were varied results. In some areas, court services had improved during the past year, but these measures do not provide information about how much modernising courts has improved court services.

Information for activity levels for the three projects

3.14
The Ministry has collected activities data on the three projects. This data included the number of applications or claims made, how long it took to deal with applications or claims, and information about the use of AVL.

3.15
Activities data can sometimes provide useful indicators of effectiveness. The Ministry said in a literature review that increased usage rates were important for realising financial benefits.⁶ This was because maximising cost savings relied on high levels of use.

3.16
However, there were weaknesses in the Ministry’s activities data for the three projects. For example, to calculate the usage rate for each District Court that provides AVL, the Ministry divided the number of AVL appearances that have occurred (actual events) by all court appearances that could have used AVL but did not (potential events).

3.17
This calculation to determine AVL usage rates is flawed because the Ministry overestimates the number of potential events. The Ministry includes all courtrooms in a District Court, including courtrooms that do not provide AVL, in its calculation.

3.18
Calculating usage rates in this way has resulted in the Ministry underestimating the usage rate of AVL, particularly for larger courts. For example, for Christchurch District Court, when potential events were limited to the courtroom with AVL, the usage rate changed from about 55% to more than 90% (see Figure 4).

Figure 4
Comparing two different calculations of the usage rate of Audio-Visual Links in Christchurch District Court, September 2014 to May 2017

3.19
There were also weaknesses in the activities data for applications to dispute a fine and civil claims. This is because the information was too high-level and did not give a sense of the effect of changes to processes on improving services. The Ministry has recognised this and is continuing to work on it.

Operational information the Central Registry collects

3.20
We were told that the Central Registry collects data that includes the number of applications received, how fast it deals with applications, and the decisions made. The Central Registry has a target of dealing with applications within 24 hours if all the information received is correct.

3.21
Ministry staff indicated that 90% of applications to dispute a fine were dealt with within 24 hours. The Central Registry uses this type of information to allocate its staff to the various major tasks, such as dealing with applications to dispute a fine or dealing with civil claims, based on where there are pressures on the service.

3.22
Although collecting data to support operational targets is important, it does not accurately measure whether customer service has improved.

What improvements people say have been made

3.23
To understand what has been achieved, we spoke to people from the Ministry, the judiciary, lawyers, the Police, Corrections, the Inland Revenue Department, and some other prosecuting agencies. They held a range of views.

3.24
In some instances, there was disagreement between the Ministry, a sector partner, and some organisations about what improvements had been made. In our view, the lack of accurate information about improvements is likely to have contributed to this.

3.25
For centralising dealing with applications to dispute a fine, the people we talked to generally agreed that improvements to customer service have largely been made. For example, two prosecuting agencies told us that centralising dealing with applications to dispute a fine has created a more user-friendly and faster service.

3.26
However, for dealing with civil claims, Ministry staff and some affected organisations did not agree on what improvements had been made. For example, Ministry staff believed that people and organisations using this service receive more consistent decisions and processing times, have a good understanding of the process, and benefit from having a central point of contact.

3.27
Although some organisations using the service agreed in principle with centralising the process, they believed that the service they receive has not improved. We were told that they felt the processing time for some claims has slowed down or is variable and that the Central Registry is not always able to track the progress of a claim. Sometimes, claims are lost and need to be resubmitted, or claims need reworking because of changes to the process that have not been communicated to the organisation.

3.28
For AVL, people we spoke to agreed that some improvements have been made:

Improved welfare of people in custody awaiting trial. AVL reduces the stress levels many people in custody feel when attending court. Other improvements include reduced disruption to people in custody attending their programmes and improved participation at each stage of the process.

Improved safety and security for people in custody awaiting trial and the public. AVL reduces the risk of people in custody carrying contraband. People in custody are considered to be safer because the risks associated with travelling to court are avoided. There is also less risk of people in custody escaping, which improves public safety.

Improved court security. AVL provides improvements in situations involving high-risk defendants. This increases safety for people involved in the court process, including members of the public. However, some people told us that these improvements need to be balanced against the fair delivery of justice.

Efficiency gains for lawyers from using instruction suites. Instruction suites allow lawyers to consult with their clients through AVL before a court appearance. Lawyers told us that using instruction suites means that they do not have to travel to Corrections facilities to see their clients. The instruction suites also give lawyers more access to their clients, allowing them to build a relationship with them.

3.29
There was disagreement about the achievement of some improvements:

Whether District Court days ran more smoothly. Some people believed that the AVL booking system, which allocates court appearances to fixed 15-minute time slots, reduces the need for lawyers and family members to wait in court because they know what time the court appearance is planned for. However, other people told us that the fixed 15-minute time slots could mean fewer court hearings in a day because not all hearings need 15 minutes.

Whether there were financial benefits. The Ministry told us that it believed that Corrections has achieved some financial benefits by not having to transport people in custody to court. However, Corrections told us that they had not gained significant financial benefits. Although fewer people in custody are making in-person court appearances, this does not necessarily mean that fewer vans are going out. Each van holds up to eight prisoners, so vans carrying fewer people in custody still go to court. Corrections said that, to achieve significant financial benefits, the scheduling of AVL appearances would need to change.

6 Ministry of Justice (2016), Optimising Audio-Visual Links between court and custody: A literature review.

Part 4 – Engaging with affected people and organisations

4.1
In this Part, we discuss:

why engaging with people and organisations is important;

how effective engagement supported implementation of projects;

how a lack of engagement contributed to difficulties; and

how collaboration between organisations has been limited.

Summary of our findings

4.2
Engaging with people and organisations affected by the three projects was variable. We saw some effective engagement planning that included identifying affected people and organisations, and planning to engage with them. There was also some effective consultation and communication by the Ministry with people and organisations. This supported a smooth implementation of the projects.

4.3
In other instances, not enough consultation and communication with people and organisations contributed to a range of difficulties during and after implementation of a project. Collaboration between organisations was limited, which is likely to have affected the delivery of service improvements. It is also likely to have risked people and organisations losing confidence in new services and the Ministry.

Why engaging with people and organisations is important

4.4
We expected the Ministry to have identified the people and organisations likely to be affected by the implementation of the three projects.

4.5
We also expected the Ministry to reach a common understanding of each project’s objectives and intended improvements with these people and organisations. Keeping people informed of the progress of a project and any important decisions is important for making the intended improvements.

Effective engagement supported implementation of projects

4.6
For centralising dealing with applications to dispute a fine, the Ministry engaged well with the affected organisations. The Ministry identified and put in place engagement plans at the start of the project and during preparation for its implementation. This led to some good engagement with these organisations, which contributed to a smooth implementation of the project.

4.7
At the start of the project, the Ministry prepared detailed plans for engaging with affected organisations. The plans included who was responsible for communicating with the organisations and when it should be done. For example, the Ministry identified Wellington City Council, the Police, and the New Zealand Transport Agency as three organisations that it needed to engage with.

4.8
This planning led to the Ministry engaging well with a range of organisations. This included:

consulting with the Chief District Court Research Council on whether judges needed training and concluding that training was not needed; and

establishing relationships and working with affected organisations. This supported a request from the Ministry of Transport, the New Zealand Transport Agency, and the Police to change the implementation date from May 2012 to August 2012.

4.9
Some prosecuting agencies told us that effective engagement from the Ministry resulted in a smooth transition. For example, one prosecuting agency told us that it attended a half-day seminar in Wellington where the Ministry explained how centralising dealing with applications to dispute a fine worked and how it would affect them. The prosecuting agency believed that the project successfully addressed the existing problems.

A lack of engagement contributed to difficulties

4.10
For civil claims and AVL, the Ministry did not engage enough with the judiciary or organisations affected by the changes to processes. This contributed to a range of issues during and after implementation.

4.11
At the start of the projects, the Ministry did not plan to engage with the judiciary, and we found no evidence of it consulting with them. Given the importance of preserving the function of the judiciary when making changes to how courts operate, we expected the Ministry to have consulted with them about the planned changes.

4.12
For example, the lack of consultation contributed to the judiciary reviewing the new process for dealing with civil claims shortly after it was centralised. This review led to a clarification of some of the rules that govern quasi-judicial decision-making.

4.13
Deputy registrars are now expected to refer a claim to the relevant District Court judge if they are unsure about what judgment should be made. In our view, the need to review the new process and provide some clarification could have been avoided if the Ministry had consulted more with the judiciary earlier in the project.

4.14
For civil claims, the Ministry identified in its business case that the District Courts would be affected by changes to the process. Although we found no evidence of the Ministry planning to engage with District Court staff, some engagement occurred. For example, some District Court staff participated in weekly teleconference project meetings, attended a series of workshops organised by the Ministry, and received consultation documents for comment.

4.15
To prepare for implementing the project, the Ministry outlined the new process in its internal magazine, Straight Up, which it sent to all District Courts. This included setting out what District Court staff’s involvement would be once civil claims were dealt with centrally.

4.16
The Ministry did not identify or plan to engage with people or organisations likely to be affected by centralising dealing with civil claims. For example, the Ministry did not identify major users of the process, such as finance and credit companies or the Inland Revenue Department, as organisations to engage with.

4.17
During the projects, the lack of planning contributed to the Ministry not engaging enough with affected people and organisations on the progress of centralising dealing with civil claims and on important decisions. For example, the Ministry’s engagement with organisations was largely by writing to them, and it was not clear which organisations or people the Ministry had shared this information with.

4.18
As a result, the Ministry would not have been able to develop an understanding of the potential effect of the changes to affected organisations’ business processes and how effective these changes were likely to be in improving court services.

4.19
This contributed to some difficulties during and after implementation of the project, which have undermined the extent to which the intended improvements have been achieved.

4.20
For example, we were told that, because some organisations did not understand the new requirements, they had not been able to align their business processes for dealing with civil claims. This lack of alignment contributed to a high rate of claim forms being rejected and to organisations needing to rework these claims. Six months after the project was implemented, the Ministry was helping the affected organisations prepare the required templates for the new process.

Collaboration between organisations has been limited

4.21
Collaboration between important people and organisations is important for effectively addressing complex problems that cannot be solved by a single organisation. People and organisations need to work together to develop a shared vision and understanding of what the justice sector is trying to achieve.

4.22
Collaboration includes reaching early agreement on intended improvements, who is likely to benefit from the intended improvements, and the best approach for addressing issues that need people or organisations to work together.

4.23
The business case for AVL stated that the Ministry and Corrections needed to share information during the project. Sharing information between the two organisations was important because the project involved developing an AVL network between the courts and Corrections facilities. Good technology and an alignment of business processes was needed to achieve the intended improvements.

4.24
During the project, information was not adequately shared between the Ministry and Corrections. For example, business processes were not developed to ensure that the two organisations were aware when standard system software tests were carried out. The lack of agreed process created the risk that seemingly small changes to systems software made by either organisation could cause system misalignment and lead to a range of problems for operating the AVL network.

4.25
We also expected the Ministry to lead collaboration with lawyers and other justice sector organisations affected by implementing AVL. We expected this collaboration to include information sharing, planning to align business processes, and, as appropriate, a shared reporting framework.

4.26
The Ministry’s collaboration with justice sector organisations and lawyers has been limited. This has contributed to a lack of alignment of business processes throughout the justice sector, which is likely to have prevented the project making all the intended improvements.

4.27
We were told that most operating issues since the implementation of AVL were caused by a lack of alignment of business processes and inadequate communication between the Ministry and Corrections. For example, people in custody sometimes appeared in District Court in person when they were supposed to have used AVL. There have also been instances where the wrong person in custody appeared before a judge using AVL.

4.28
There has been a range of local initiatives to improve the use of AVL and get more alignment of business processes between justice sector organisations. For example, the Auckland Optimisation project involved the Ministry working closely with Corrections and the Police to identify challenges that were preventing increased use of AVL in Auckland. We were told more recently that the Ministry has started to engage with some lawyers and members of the judiciary in Auckland.

4.29
The Auckland Optimisation project has been trialling three important changes to the processes for using AVL. These are:

better scheduling of AVL appearances by increasing the alignment of business processes between the courts and Corrections in Auckland, which included increasing communication between courts and Corrections facilities from once a day to several times a day;

exploring the use of instant messaging between the courtroom and the Corrections facility to improve communication when using AVL; and

Corrections organising the bookings for instruction suites through a central phone number. This was expected to improve the use of instruction suites by lawyers and lead to more court hearings proceeding as planned.

4.30
Although this is likely to bring improvements for District Courts and related Corrections facilities, these improvements are likely to be limited to Auckland. We discuss the importance of a more comprehensive view of feedback about services in Part 6.

Part 5 – Governance and accountability for the three projects

5.1
In this Part, we discuss:

why governance and accountability is important;

the limited evidence of formal approvals;

how inconsistent reporting was not always addressed; and

how a lack of engagement with affected people and organisations was not adequately addressed.

Summary of our findings

5.2
The adequacy of governance and accountability varied between the projects. Although governance and accountability arrangements for centralising dealing with applications to dispute a fine were better than for the other projects, there were weaknesses in how the Ministry maintained standards of governance and accountability.

5.3
In each of the three projects, there were documents that did not appear to have been formally approved. During two projects, there was inconsistent reporting on the progress of projects. There were also significant weaknesses in records management, and a lack of engagement with affected people and organisations was not adequately addressed.

5.4
In our view, these weaknesses indicated that there was insufficient governance and accountability over the three projects. It would have been difficult for governance groups to maintain effective oversight of the relevant Ministry investment in modernising courts, and it is not clear that appropriate decision-making processes were consistently followed.

5.5
The Ministry told us about substantial changes it made to its internal governance and processes around investment since 2014. We describe these changes in Part 6.

Governance and accountability is important

5.6
Our report on governance and accountability states that good governance helps ensure that the intended improvements from projects are achieved on time and to budget.⁷ We expected the Ministry to use good practice that supported good governance and accountability of its projects.

Limited evidence of formal approvals

5.7
Governance groups need to approve important reports and decisions. There was limited evidence of the Ministry consistently recording formal approvals. Even for a project that was run as “business as usual”, such as centralising dealing with civil claims, we expected evidence of governance groups giving formal approval and the Ministry recording it where appropriate.

5.8
In each of the three projects, there were documents, such as business cases, project initiation documents, and benefit realisation plans, that did not appear to have been formally approved. For example, there is no evidence that the Ministry formally approved the business case that proposed centralising five functions, including dealing with civil claims.

5.9
There was also a lack of evidence that formal approvals were given for requests to change the scope of projects, although it was clear that the proposed changes had been acted on.

5.10
For example, when centralising dealing with applications to dispute a fine, the project management team wanted approval to delay the implementation date because of feedback from three organisations. Although there was no evidence that the Ministry gave formal approval, the implementation date was changed from May 2012 to August 2012.

5.11
The lack of formal approvals could indicate not enough governance and accountability or that appropriate decision-making processes were not followed. Formal approvals are important because they help ensure effective decision-making on issues and proposed changes to projects. It also ensures that project managers are held to account for the delivery of the project.

No formal reporting for dealing with civil claims

5.12
Although centralising dealing with civil claims was not managed as a formal project, we still expected information about costs, time, and risks to be reported to an appropriate governance group. This information was needed for the Ministry to know that the project was progressing as expected.

5.13
During the project, there was no evidence of formal tracking and reporting of costs, risks, issues, and progress against timetables to any governance group.⁸ We also saw no evidence that the relevant governance group needed formal reporting about the project. This meant that the relevant governance group did not have the information it needed to ensure that the project for centralising dealing with civil claims was progressing appropriately or to decide whether it needed to intervene.

Inconsistent reporting was not always addressed

5.14
There was inconsistent reporting of the three projects’ progress. This included tracking the progress of projects in status reports against different information than was identified in their business case. There were also instances of project dependencies, risks, and issues not being consistently identified and tracked.

5.15
For example, information in the AVL business case did not align with information in the project status reports that were available. This included:

the budget for the project changed from $7.6 million capital funding in the business case to $8.99 million in the project status report without explanation; and

the risks identified in the business case largely did not align with the risks in the project status reports. For example, some main operational and reputational risks in the business case did not appear in the project status reports. Several new risks were also added to the project status reports without explanation. These new risks included duration of project, changing scope, technology procurement, and performance of vendors.

5.16
For centralising dealing with applications to dispute a fine, the Ministry did not identify at the start or during the project what the risks were to successfully achieving the intended improvements. This meant that it was unlikely that the Ministry adequately considered risks that could have prevented the successful delivery of the project, the likelihood of their occurrence, or what mitigations, if any, should be put in place.

5.17
During the project, there was minimal reporting on issues. For example, in the programme management team meeting minutes in November 2011, the Ministry noted that the project manager would consider the risks and issues from a potential delay to implementing the project in response to internal feedback.

5.18
However, it is not clear whether the project manager communicated this information to the Courts and Criminal Matters Bill steering committee, the group that was governing the project. This is important because identifying, understanding, and managing risks and issues is a fundamental part of effective governance.⁹

5.19
We expected the relevant governance groups to provide project oversight and hold project managers to account for maintaining good project management practices. This includes requiring project managers to give them the appropriate information so they can do their job effectively. There was no evidence that governors asked for improved reporting for these projects.

Weaknesses in records management for the AVL project

5.20
There was significant weaknesses in records management. For example, for AVL, the Ministry lost 16 monthly project status reports between September 2013 and March 2015. Ministry staff attributed the loss of reports to the lack of an effective document management system and staff not handing over a full set of documents to the Programme Management Office.

5.21
Project status reports are for members of the relevant governance groups. Significant gaps in records meant that we could not determine whether governors had all the information they needed to effectively govern the projects. We also could not determine how effectively the Ministry tracked costs, time, risks, project dependencies, and preparing for implementation of projects over time.

Lack of engagement with affected people and organisations was not adequately addressed

5.22
Because of the number of organisations involved in AVL and centralising dealing with civil claims, we expected the relevant governance groups to ensure that there was proper oversight of engagement planning and actual engagement.

5.23
However, for both projects, we found no evidence that this occurred. As we discussed in Part 4, a lack of consultation and communication contributed to a range of difficulties during and after implementation.

7 Office of the Auditor-General (2016), Reflections from our audits: Governance and accountability.

8 The business case for centralising some administrative functions did not identify the governance body or owner for the work that dealing with civil claims was a part of.

9 We talk about the importance of managing risks effectively in our 2016 report, Reflections from our audits: Governance and accountability.

Part 6 – Improving for the future

6.1
In this Part, we discuss:

changes to the Ministry’s governance of projects;

improving how feedback is collected; and

establishing effective performance measures.

Changes to the Ministry’s governance of projects

6.2
The Ministry told us that it has made substantial changes to its internal governance and processes around investment since 2014. The aim was to improve the Ministry’s ability to drive investment “from the top down” based on a coherent strategy, prioritisation, and asset management.

6.3
These changes included creating two new governance committees in 2014 to support the Ministry’s senior management in its governance role. Figure 5 shows the new governance arrangement that established the Investment Committee, which is responsible for overseeing new investment projects, and the Planning and Resources Committee, which is responsible for overseeing the Ministry’s strategic and business planning processes. The Programme Management Office was shifted to provide support to the Investment Committee.

Figure 5
The Ministry of Justice’s project governance structure since 2014

6.4
The Ministry also wanted to put in place stronger budget controls and centralise funding by having the Investment Committee allocate and oversee funding for programmes and projects.

6.5
The Ministry told us that they established two sub-portfolios to focus on project delivery in 2015/16. The aim is to encourage the Investment Committee to focus more on strategic issues such as longer-term investment planning and achievement of improvements. In our annual audit in 2016/17, we said that it was pleasing to note that the Ministry’s governance structures continue to strengthen and improve.

6.6
During the same period, the Ministry also made changes to its project management practices. For example, we were told that the Programme Management Office implemented an Investment Delivery Framework in 2015. This framework provides checkpoints for the different stages a project goes through. The Ministry considers that this represents a significant improvement in its project and programme management methodology.

6.7
In 2017, the Ministry reshaped its Operations and Service Delivery group. To support the Ministry to achieve the intended changes, the Programme Management Office was moved into Strategy Development. However, to ensure that good project management practice is applied to projects, it is important that the Ministry makes sure that project managers have access to the Programme Management Office. This includes developing capability for members of staff by providing the advice and support they need.

6.8
We did not audit the changes the Ministry has made to improve governance and accountability. However, there have been two recent reviews of the Ministry’s project management arrangements. These were:

a Treasury examination of the maturity of the Ministry’s Portfolio, Programme, and Project Management as part of its Investor Confidence Rating assessment in 2016; and

an Internal Audit Review of Investment Delivery in 2016/17.

6.9
Both reviews concluded that the Ministry has made some gains in recent years. The Investor Confidence Rating assessment concluded that, in “the delivery of initiatives aligned with the strategic direction of the organisation”,¹⁰ the Ministry was performing relatively well. The Investor Confidence Rating for the Ministry is a B. However, both assessments noted that the Ministry needed to improve its portfolio management in such areas as risk management and stakeholder management.

6.10
The Ministry told us that it plans to improve its portfolio management and the support available to senior decision-makers to make decisions through the application of enterprise portfolio management disciplines. An enterprise portfolio management approach is about selecting the right investments to achieve the organisations’ strategic objectives.

6.11
Having selected a portfolio of investments, portfolio management focuses on whether programmes and projects are being delivered in the right way and making sure that the intended improvements are realised. Before the end of 2017, senior managers expect to be presented with options that will further align the Ministry’s governance with the principles of portfolio management.

6.12
For AVL, the Ministry has established an internal Audio Visual Services governance group to help the Ministry improve the delivery of existing remote audio-visual services and to change the way these services are delivered in Auckland and Christchurch.

6.13
In our view, it is important that the Ministry makes sure that its changes to governance and project management practices ensure adequate levels of accountability and governance over projects.

Improving how feedback is collected

6.14
The Ministry needs to have good oversight of emerging issues when changing processes or putting in place new processes to know how successfully they are embedded. Information about emerging issues needs to be collected over time. It should be easy for people to provide feedback about their experiences.

6.15
To help the Ministry maintain good oversight of emerging issues and enhance courts services, it needs to improve its current processes for collecting and analysing feedback.

Feedback processes have limited effectiveness

6.16
Processes were put in place for organisations to give feedback on the three projects. These processes had some effectiveness. For applications to dispute a fine and civil claims, prosecuting agencies had a single point of contact to give feedback – the relevant Central Registry operating manager.

6.17
We were told that having a good one-to-one relationship with an operating manager made it easy to communicate when small issues arose in the system and have them addressed. For example, one prosecuting agency told us that, when minor issues with the system arise, it contacts the Central Registry operating manager to solve the issue.

6.18
For AVL, the Ministry gathers feedback from justice sector organisations, lawyers, and judges. For example, we were told that District Courts in Auckland, Manukau, North Shore, Hawkes Bay, and Dunedin held meetings for people and organisations working in the justice sector.

6.19
The purpose of these meetings was to get feedback from people on their experiences working with courts. People attending these meetings could include local police officers, representatives from the law society, representatives from Corrections, and, in some instances, local judges.

6.20
We were told that people at these meetings discussed their experience of, and concerns with, using AVL. It was not clear how effective these meetings were in addressing these concerns.

6.21
Although one-to-one working relationships and local feedback can be appropriate for addressing individual and local issues, they do not provide the Ministry with good oversight of systemic issues that the three projects created by changing processes.

Formal feedback mechanisms are needed to identify systemic issues

6.22
We were told that the Ministry was about to introduce a way for people to give feedback for part of the work done by the Central Registry. This will allow the Ministry to collate all of the feedback and complaints information it receives for applications to dispute a fine in one place and to identify emerging issues. This is a positive development.

6.23
The Ministry needs to have formal mechanisms to collect, collate, analyse, and report on feedback from organisations and people affected by changes to court services. This will give the Ministry good oversight of emerging issues and the main challenges to optimising the achievement of improvements from projects. The Ministry then needs to use this information to identify systemic improvements and feed into continuous improvement processes.

Recommendation 1

We recommend that the Ministry of Justice ensure that it has effective processes to collect, collate, analyse, and report on feedback from people and organisations that are affected by changes to court services resulting from projects.

Establishing effective performance measures

6.24
As discussed in Part 3, the Ministry’s lack of an effective framework for tracking and measuring improvements achieved through projects has meant that it does not have good information on what improvements have been made or challenges to further improvements.

6.25
The Ministry needs to establish performance measurement and reporting frameworks so that it can determine the extent to which projects have made intended improvements. The Ministry can use this information to inform discussions about how to increase the effectiveness of intended improvements and to provide certainty to all affected people and organisations about the extent to which intended improvements have been made.

Recommendation 2

We recommend that the Ministry of Justice prepare a performance measurement and reporting framework, including a timetable, to track whether the intended improvements from its investments in projects to improve court services are being achieved.

6.26
The Ministry has acknowledged the need for improvement and told us about progress made to improve benefits realisation. The Ministry is implementing a new mechanism for tracking intended benefits. In addition, the Ministry is working towards making sure that cross-sector projects have benefit owners assigned and that benefits are reported through individual organisations’ benefit-reporting frameworks.

10 Axelos Ltd (2015), Introduction to P3M3.