B.29[17p]

Auckland Council: Working to provide customer-centred services online

Presented to the House of Representatives under section 20 of the Public Audit Act 2001.

November 2017

ISBN 978-0-478-44277-9

Contents

Overview

Our recommendations

Part 1 – Introduction

The role of Auckland Council

The Customer-centric Transformation programme

What we reviewed

How we carried out our review

Structure of this report

Part 2 – The online services programme and the organisational strategy

Auckland Council’s organisational strategy

The purpose of the programme is consistently understood

Governance and accountability arrangements have become confused

Part 3 – Delivering projects in the online services programme

Choosing and approving the right projects

Elements of good practice in delivering projects

Reporting to governance groups about projects needs to improve

Effectively moving projects to the responsible business units

Part 4 – Achieving the expected benefits and learning from projects

What has been achieved so far?

Improving the tracking and monitoring of benefits

Lessons are being applied

Figures

1 – What we looked at, in the context of Auckland Council’s wider organisational strategy and projects

2 – Relationship between the organisational strategy and Auckland Council’s planning documents

3 – The Council’s strategic outcomes and examples of the associated performance measures

4 – Objectives of the online services programme

5 – Single individual with multiple leadership roles

6 – Sequence for reviewing and approving projects

7 – Summary of the planned and actual dates for making online services available to the public

8 – Summary of the planned and actual cost for the projects, as at May 2017

9 – Estimated net financial benefits to Auckland Council from approved projects under the customer-friendly services focus area, 2017-19

10 – Percentage of smart form applications completed online each month, from December 2016 to July 2017

Overview

The services provided by Auckland Council and its council-controlled organisations affect every Aucklander. These services include, for example, issuing resource and building consents, collecting rubbish and recycling, providing drinking water, and providing public transport.

In 2015, the Council began preparing an organisational strategy for 2017-19 to improve the way it delivers its services to Aucklanders. The organisational strategy focuses on internal culture and processes. It provides the Council with a “roadmap” of 34 initiatives to improve the Council’s performance, which are organised into four focus areas.

The Council intends to improve the services it provides to Aucklanders and make it easier for people to use those services. As part of the “customer-friendly services” focus area, many services are being put online and being simplified. The Council called this programme of work the “Customer-centric Transformation programme”. For simplicity, we call it the online services programme.

As part of our regular reviews of the Council’s service performance, we looked at how the Council was managing the online services programme, and we looked closely at two projects in that programme.

The Council delivered the projects we looked at largely on time and to budget. We saw some good practice, including:

using an iterative approach to deliver projects;

getting feedback from customers and staff before and during the project; and

appropriate management of project risks and issues.

By June 2017, the online services programme had digitised 22% of the Council’s most common transactions, towards its target of 70% by the end of 2019. It is now easier for people to access services online. The Council has also achieved additional benefits and learned lessons from the programme.

The two projects we looked at help to improve customer interactions with the Council. Aucklanders can use a single log-in to access all of the Council’s online services, and the process for accessing particular services has been simplified – for example, making registering a dog or applying for a licence to run a business quicker and easier for the Council’s customers.

However, there are risks to the online services programme’s continued progress.

We found that the governance and accountability arrangements for the online services programme and the customer-friendly services focus area that it sits under had become confused. In our view, a lack of distinction between the two has caused this.

We have recommended that the Council review these arrangements to ensure that:

it maintains effective governance and oversight of the customer-friendly services focus area and its associated programmes and projects;

projects that are intended to deliver customer-friendly services are managed in the most effective and efficient way; and

it keeps the right balance between the programme’s focus on putting services online and the other objectives of the customer-friendly services focus area such as improving customer experiences.

In our view, the Council needs to improve its reporting on the progress of projects to the relevant governance groups by making that reporting more accurate and including more information about whether projects are progressing as expected. Governance groups need regular, clear, and objective reporting of a project’s performance. The risks of poor reporting are that the governance groups do not get the information they need to make good decisions in a timely manner.

During our audit, the Council was working to improve how it could demonstrate that the benefits from projects were being achieved. We agree that there is a need for the Council to improve its reporting and monitoring of benefits, particularly non-financial benefits. This would give the Council a more complete view of the difference the online services programme and individual projects were making .

The Council has acknowledged the areas for improvement that we identified and is making changes in response. Once the Council has addressed these matters, it will be in a strong position to achieve the objectives of its organisational strategy, including making its services more customer-friendly.

We encourage other agencies to consider this report when planning to invest in information and technology systems to improve how they deliver services.

I thank the staff of Auckland Council for their help and co-operation.

Greg Schollum
Deputy Auditor-General

24 November 2017

Our recommendations

We recommend that Auckland Council:

1. review its framework for the governance and accountability of programmes and projects that contribute to achieving customer-friendly services;

2. provide all governance groups with reports that are accurate, enable progress to be tracked over time, and enable the people who govern projects to maintain effective oversight of risks and issues; and

3. review its monitoring and reporting on progress towards achieving the financial and non-financial benefits from projects and how projects contribute towards achieving all of the objectives of the customer-friendly services focus area.

Part 1 – Introduction

1.1
In this Part, we discuss:

the role of Auckland Council;

the Customer-centric Transformation programme;

what we reviewed;

how we carried out our review; and

the structure of this report.

The role of Auckland Council

1.2
Auckland Council (the Council) delivers a wide range of services to about 1.6 million people. Those services include, for example, issuing resource and building consents, collecting kerbside rubbish and recycling, managing community and recreation centres and parks, and supporting and funding local events. The Council also provides some services (such as public transport and drinking water) through council-controlled organisations.

1.3
The Council was formed in November 2010, when a law change meant that eight separate councils became one council for the whole region.

1.4
In 2016/17, the Council spent about $3.8 billion on providing services to Aucklanders.

The Customer-centric Transformation programme

1.5
In 2015, the Council began preparing its organisational strategy for 2017-19. Because the Council had previously focused on bringing together the eight councils, this strategy is the first to look at how the Council could make improvements and plan for the future. The organisational strategy sets out four focus areas:

engaging and enabling communities;

providing customer-friendly services;

making its size work; and

having a high-performance culture.

1.6
Using these focus areas, the Council developed a three-year “roadmap” of 34 initiatives. These initiatives are a range of programmes and projects to improve its performance and help to achieve the goals in its statutory planning documents (such as the Auckland Plan, the Unitary Plan, and the long-term plan). As part of the organisational strategy, the Council also developed various performance measures to monitor its progress. The Council aims to achieve its strategic outcomes by the end of 2019.

1.7
The Customer-centric Transformation programme is one of the initiatives in the customer-friendly services focus area. For simplicity, we refer to it as the online services programme. The programme aims to deliver better services to the Council’s customers, communities, and citizens, with a focus on “digital solutions” and improved processes. To do this, it aims to deliver a set of projects by 2019. Much of the online services programme is about putting services online (including setting up the back-end systems and processes to support those services).

1.8
The online services programme’s business case sets out the following reasons for the programme:

Customers need easy, simple, and fast interactions with the Council.

Customers expect to be able to access and use services online.

Putting services online can create a better experience for staff and customers.

The Council has significant fiscal pressures, but taking a customer-centred approach can help meet these pressures and the expectations of customers.

1.9
We reviewed the online services programme because it:

affects how the Council delivers its services and its interactions with Aucklanders; and

provided insight into the Council’s implementation of the organisational strategy.

What we reviewed

1.10
Section 104 of the Local Government (Auckland Council) Act 2009 requires the Auditor-General to review the service performance of the Council and its council-controlled organisations from time to time. We have completed four reviews so far.¹

1.11
In this review, we looked at how effectively and efficiently the Council managed the online services programme. We looked at the:

outcomes sought from the online services programme and how these align with the Council’s organisational strategy;

delivery of two projects within the online services programme; and

benefits achieved from the online services programme.

1.12
We did not review:

how the Council prepared or implemented the organisational strategy; or

any online initiatives in the Council’s council-controlled organisations.

How we carried out our review

1.13
We assessed the Council’s effectiveness and efficiency in managing the online services programme by looking at:

the overall online services programme; and

two projects within the online services programme in more detail – the Identity Management project (including phases 1 and 2 of the project) and the Licensing and Compliance Services Smart Forms project (the Smart Forms project). See Figure 1.

Figure 1
What we looked at, in the context of Auckland Council’s wider organisational strategy and projects

Source: Office of the Auditor-General, using information from Auckland Council.

1.14
The purpose of the Identity Management project was to make it possible for Aucklanders to sign in to all Council services using a single log-in account (a combination of username and password). Historically, each service was set up so that customers had different accounts to carry out different tasks. The Identity Management project provides a person with access to all of the Council’s services through a single log-in account. People can set up a Council account or use their RealMe, Facebook, or Google accounts.

1.15
The aim of the Smart Forms project was to put 21 of the Council’s Licensing and Compliance Services forms and payment options online, including dog, alcohol, food, and street-trading licenses, and health and hygiene certificates. The project also aimed to make the forms easier to understand and use.

1.16
We chose these projects because of their size, when they were launched, and their potential to improve how people access the Council’s services.

1.17
To carry out our review, we:

interviewed staff from the Council;

reviewed Council documents, including management and governance reports, customer research, and project documents; and

analysed information on the benefits achieved from the online services programme.

1.18
We carried out our fieldwork and analysis from February to July 2017.

Structure of this report

1.19
In Part 2, we examine Council staff’s understanding of the online services programme. We discuss the purpose and scope of the online services programme and the programme’s alignment with the Council’s overall strategy.

1.20
In Part 3, we discuss how the Council chose and approved the Identity Management and Smart Forms projects, the delivery of the projects, and how it moved the projects from the online services programme to the business units that became responsible for them once they were complete.

1.21
In Part 4, we discuss:

what the online services programme and projects have achieved so far;

how the Council monitors benefits; and

the lessons from the online services programme.

1 A list of all of our Auckland-related work, including the previous four reviews, is on our website (www.oag.govt.nz/reports/auckland). We have also completed a follow-up audit on the progress of Watercare Services Limited in addressing the recommendations in our report Watercare Services Limited: Review of service performance.

Part 2 – The online services programme and the organisational strategy

2.1
In this Part, we discuss:

the Council’s organisational strategy;

Council staff’s understanding of the online services programme and its aims; and

a lack of clarity between the customer-friendly services focus area and the online services programme.

Summary of our findings

2.2
The online services programme aimed to put Council services online and improve processes to create a better experience for Aucklanders.

2.3
The online services programme is well aligned with the Council’s organisational strategy, especially in helping to make Council’s services more customer-friendly. In our view, the programme should support the Council in achieving the outcomes it has set out in its organisational strategy. Council staff had a consistent understanding of the programme as being about putting services online.

2.4
However, the governance and accountability arrangements for the online services programme and the customer-friendly services focus area had become confused. This creates risks for the Council in:

maintaining effective governance and oversight;

managing projects within the customer-friendly services focus area in the most effective and efficient way; and

appropriately balancing the programme’s emphasis on putting services online and the other objectives of the customer-friendly services focus area.

Auckland Council’s organisational strategy

2.5
In August 2015, the Council began work on an organisational strategy. It identified areas for improvement, including increasing the number of Council services available online. The Council adopted the organisational strategy in February 2016, describing it as a three-year “roadmap” to a higher-performing Council. The organisational strategy also supports the Council in achieving its goals under its statutory planning documents (see Figure 2).

2.6
The organisational strategy includes six strategic outcomes, each of which has associated performance measures. Figure 3 shows the Council’s strategic outcomes and examples of the performance measures.

Figure 2
Relationship between the organisational strategy and Auckland Council’s planning documents

Auckland Council’s vision

Create the world’s most liveable city and deliver value for money to Aucklanders.

Auckland Plan

A 30-year plan for Auckland to deliver the Council’s vision. The plan is led by the Mayor, and delivered by the Council, central government, and external stakeholders.

Long-term plan A 10-year budget for Auckland Council that is set by the Mayor and Councillors. It is delivered by the Council and council-controlled organisations. | Local board plans 3-year plans for each of the Council’s 21 local boards.

Organisational strategy

A 3-year plan for the Council outlining its priorities and goals to meet its objectives of being a high-performing council that can deliver the Auckland Plan.

Source: Auckland Council.

Figure 3
The Council’s strategic outcomes and examples of the associated performance measures

Strategic outcome | Main performance measure | Other performance measures include …
We get the job done faster, more conveniently, and at a lower cost than before for customer, community, and citizens. | Customer experience | Digitisation of most common transactions
Elected members are better supported to make high quality decisions. | Members’ overall satisfaction with Council staff advice and support | NZIER advice quality rating
Citizens have a strong voice and are key in shaping Auckland. | Trust in Council | Awareness of Council’s services
Our high-performing, safe, and inclusive workforce serves a diverse and changing Auckland. | Employee engagement | Overall staff well-being
General rates burden decreases as non-rates revenue share of total revenue grows. | Total non-rates revenue as percentage of total revenue | Percentage growth in non-rates revenue
We do more with less, without compromising service and the customer experience. | Maintain Standard & Poor’s AA rating Core operating costs as percentage of revenue | Number of FTE per 1000 residents

Source: Office of the Auditor-General, using information from Auckland Council.

2.7
To support the organisational strategy and its outcomes, the Council set out four focus areas:

engaging and enabling communities, with the aim of giving Auckland communities the tools and knowledge to influence decisions and take an active role in shaping their future;

customer-friendly services, with the aim of meeting people’s changing expectations about how they access services;

making our size work, using the Council’s size to be innovative and using its resources more effectively to deliver better outcomes for Auckland; and

high-performance culture, supporting and developing Council staff and focusing resources where they are needed.

2.8
The online services programme was a prioritised initiative under the customer-friendly services focus area. Other customer-friendly services initiatives included improving the Council’s website, improving the Council’s management of consents, and providing online booking for Council-managed venues.

2.9
The Council prepared the business case for the online services programme at the same time as it prepared its organisational strategy. The business case recommended setting up the online services programme because the Council:

needed to improve people’s experience of using Council services;

needed more effective ways to deliver services for customers, including through online solutions;

had cost pressures and a need to deliver services more efficiently without compromising customer service; and

needed to have clear accountability for core processes and improved understanding of the different parts of the process.

2.10
The online services programme has objectives for the Council, its staff, and its customers (see Figure 4). To achieve these outcomes, the programme aimed to deliver a series of projects that focused on putting services online and improving processes and services.

Figure 4
Objectives of the online services programme

For the public | For the Council | For staff
Customers and communities can serve themselves | Reduced cost to serve customers | Have tools to deliver great service and have better conversations
Easier for customers to deal with Council | Maximised efficiency through low cost channel | Feel part of a successful, customer-aligned organisation
Trust in Council increased | Automate low value work | Can lead change on behalf of customers where appropriate
Customer satisfaction increased | Eliminate workarounds | Focused on assisting the more complex tasks

Source: Auckland Council.

2.11
The online services programme focused on putting services online. The customer-friendly focus area had a broader view on how to improve the customer experience and make it easier for people to interact with the Council (such as friendly and efficient call centres).

2.12
The online services programme and the projects in it contribute to achieving the performance measures of the customer-friendly services focus area of:

increasing the “customer experience” score to 80%;

improving the Council’s “customer effort” score;

putting 70% of the most common transactions online;

processing consents within the statutory number of days allowed; and

decreasing the cost to the Council.

The purpose of the programme is consistently understood

2.13
Council staff had a consistent understanding of what the online services programme was trying to achieve.

2.14
Council staff said that the online services programme’s purpose was to achieve the performance measure of “digitising” 70% of the Council’s most common transactions by the end of 2019. More broadly, they said that the online services programme would help the Council catch up with Aucklanders’ expectation that they should be able to interact with the Council online. One interviewee told us:

There’s the target of 70% of transactions online by 2019. We want to move the customer trust measure, and perception of cost-effectiveness. We’re always told by customers, why can’t we do this online. It would be cheaper and more efficient.

The online services programme is well aligned with the organisational strategy

2.15
The online services programme’s focus on putting services online and making them better and easier for customers to use supports the Council’s organisational strategy.

2.16
The business case for the online services programme includes analysis about how the programme would support the Council to achieve the organisational strategy’s outcomes. The online services programme’s objectives are similar to the organisational strategy’s outcomes and associated performance measures (see Figures 3 and 4). Some of the common elements between the two include reducing the cost of serving customers, increasing trust in the Council, and making it easier for people to interact with the Council.

2.17
Council staff described the online services programme as well aligned with the organisational strategy. One interviewee said:

I think [the online services programme] is well-aligned because it closely aligns with the Customer Friendly Services [focus area]. From the amount of research we have at Council, we know that customers want it to be easier to deal with us, they want to do it at a time that suits them, they don’t want to necessarily talk with us, and they want to access information on their own. The whole digital-led proposition, they strongly lead in.

Governance and accountability arrangements have become confused

2.18
Although the purpose of the online services programme is consistently understood and well aligned with the Council’s organisational strategy, the governance and accountability arrangements for the programme and the customer-friendly services focus area have become confused.

2.19
To support the implementation of the organisational strategy, the Council approved new governance arrangements in April 2016. As part of this, the Council formed an Investment Group to assess and approve business cases, prioritise projects, and monitor the achievement of benefits. It also formed four subgroups connected to each of the focus areas.

2.20
These subgroups had responsibility for providing overall governance for the programmes and projects within their focus area. At the time of our audit, there was also a Customer-centric transformation steering committee (the steering committee) under the Customer-friendly services subgroup (the subgroup). The steering committee provided guidance to the overall online services programme.

2.21
The person in charge of the online services programme is also the subgroup lead for the subgroup. At the time of our audit, they were also the chairperson of the steering committee (see Figure 5). The overlap between these governance and management roles is not consistent with good practice.

Figure 5
Single individual with multiple leadership roles

Source: Office of the Auditor-General, using information from Auckland Council.

2.22
The overlapping roles has also created confusion for staff. Some Council staff were unclear about the difference between the online services programme and the customer-friendly services focus area. One interviewee said:

We tied CCT [online services programme] to Customer-friendly services and put the same person in charge. [Their] passion and [their] strength is CCT. No disrespect to [name] at all. [They have] been asked to do two things, with no clarity. [It] is really [a] question about whether CCT and CFS [Customer-friendly services] are both needed.

2.23
We saw other examples of the lack of clarity about the distinction between the online services programme and the customer-friendly services focus area. The Council appears to use the terms “customer-friendly services pillar” (the Council refers to the focus areas as pillars) and “Customer-centric Transformation programme” interchangeably. For example, we received a document that the Council said was a comprehensive list of projects to be completed under the “customer-friendly services pillar” from 2017 to 2019. However, the document was entitled CCT Indicative Pipeline.

2.24
We also saw a lack of clarity about which projects are part of the online services programme in other documents. For example, projects in the Consenting Made Easy programme are identified as being part of the online services programme and separate from it.

2.25
Project reporting to governance groups reflects the lack of distinction between the customer-friendly services focus area and the online services programme. We compared the project reporting to the steering committee with the project reporting that goes to the subgroup in May 2017. We found that:

the steering committee received reports on projects identified as not belonging to the online services programme;

the steering committee did not receive reports on some projects that belong to the online services programme;

neither the steering committee nor the subgroup received reports on all of the projects in the online services programme; and

two projects are not reporting to the subgroup, even though (according to the work plan) they are under way.

2.26
In our review of reports to the subgroup, we did not see any reporting that provides an overall view of the online services programme. Instead, reporting is provided to various governance groups on a project-by-project basis.

2.27
In our view, the lack of distinction between the online services programme and the customer-friendly services focus area has created confusion about governance and accountability arrangements. This creates risks for the Council in:

maintaining effective governance and oversight;

managing projects within the customer-friendly services focus area in the most effective and efficient way; and

appropriately balancing the programme’s focus on putting services online and the other objectives of the customer-friendly services focus area.

2.28
The risk to effective governance and oversight over the customer-friendly services focus area and associated projects is heightened by inaccurate project reporting. We discuss the issues with reporting in Part 3.

Recommendation 1

We recommend that Auckland Council review its framework for the governance and accountability of programmes and projects that contribute to achieving customer-friendly services.

2.29
The Council has acknowledged the areas for improvement and is addressing them by:

being clearer about the distinction between the “customer-friendly services pillar” and the Customer-centric Transformation programme (since renamed the Digital and Transformation programme);

identifying other projects in 2017/18 that contribute to the other non-digital objectives of the customer-friendly services focus area; and

incorporating the identified issues with the reporting into future improvements.

Part 3 – Delivering projects in the online services programme

3.1
In this Part, we discuss:

choosing and approving the right projects;

elements of good practice in delivering projects;

improving reporting to governance groups; and

effectively moving the completed projects to the responsible business units.

Summary of our findings

3.2
We did not see any evidence of a formal prioritisation framework for choosing and approving projects in the online services programme. Without such a framework, the Council risks prioritising projects that do not provide the greatest benefit in achieving its strategic outcomes, including making it easier for people to access and use the Council’s services. The Council has identified that prioritising projects is an area for improvement.

3.3
We identified elements of good practice for delivering projects in the online services programme. The Council completed the Identity Management and Smart Forms projects largely on time and within budget.

3.4
The progress of individual projects against timelines and project costs, risks, and dependencies were reported to the steering committee and are reported to the subgroup. However, some of the reports were inaccurate, and they were presented as “a snapshot in time”, without any trend information. Poor reporting means that the governance groups might not get the information they need to make good decisions in a timely manner, which could affect their ability to maintain effective oversight.

3.5
Council staff in the responsible business units and those working on the projects considered that projects have been moved effectively to the responsible business units. Aucklanders can now use a single log-in to access all of the Council’s online services and can go online to register their dog or apply for the licences they need to run their businesses.

Choosing and approving the right projects

Improving prioritisation of projects

3.6
The process for choosing and approving projects in the online services programme consists of prioritising proposed projects, preparing business cases, and approving business cases.

3.7
Proposed projects are prioritised:

within the online services programme;

by the Customer-friendly services subgroup; and

by the Investment Group.

3.8
Each of these has a slightly different focus. However, we did not see any evidence of a formal prioritisation framework. Without a framework, the Council risks making poor decisions about how it prioritises projects. A prioritisation framework would help ensure a common understanding between the three levels of the prioritisation process. It would also provide a consolidated view of factors that influence the prioritisation of projects at the different levels.

3.9
The business case for the online services programme recommended setting up a performance management framework with standard criteria to help prioritise projects. However, when the governance arrangements changed and the Investment Group was formed, the Council did not put this framework in place.

3.10
The Council has identified prioritisation as an area for improvement. Council staff have described the prioritisation process for 2017/18 as more rigorous than for the previous year. The Council told us that, for 2017/18, the Investment Group asked for more information to support requests for funding. All subgroups had to complete a more rigorous prioritisation process to meet the Investment Group’s criteria, including testing the ability to deliver the projects based on the available resources.

3.11
The Council is also doing further work, including:

developing a formal prioritisation framework; and

planning to use information about the progress of projects in achieving the expected benefits to guide future prioritisation.

3.12
We encourage the Council to continue its work in improving prioritisation of projects. Limited funding makes it even more important that the Council has an effective prioritisation process to help it choose the projects that will provide the greatest benefit in achieving its strategic outcomes, including making it easier for people to access and use the Council’s services.

Preparing business cases for projects

3.13
The way that the online services programme develops business cases follows elements of good practice. In our view, these will help to ensure that projects meet the needs of customers and the responsible business unit.

3.14
Each project in the online services programme needs to have its business case approved by the Investment Group before the project can begin. Each business case provides information on matters including:

the rationale for the project;

scope and deliverables;

expected costs, timelines, and benefits; and

how the project supports the Council’s strategic outcomes.

3.15
Lessons from earlier projects were used to develop the Identity Management and Smart Forms projects. As part of the Identity Management pilot, the team talked with experts in identity management to understand what was needed and how to achieve it. The project manager for the Smart Forms project looked at what could be learnt from other projects that had used the chosen software.

3.16
The Council developed principles for the Smart Forms and Identity Management projects so it was clear what the projects needed to provide. The Council also checked whether the chosen technology would meet the requirements of each project. The Identity Management project did this through a pilot, while the Smart Forms project completed a proof of concept.

3.17
The online services programme used a co-design methodology, which meant that customers and business unit staff were involved in the design process. For example, as part of the Smart Forms project, researchers talked to café managers about how compliance activities and licensing could be more user-friendly. Council staff told us that this was a change from the Council’s usual process.

3.18
Information from business units was gathered in several ways, including through workshops with staff and someone from the project sitting with the business unit to learn about what they do. Council staff told us that project teams within the online services programme formed a good understanding of the business before preparing the business case.

3.19
The project team then identified what needed to be delivered and what potential benefits might result. For example, the Smart Forms project team identified that improving the applications and putting them online could reduce the time Council staff spent on each application by an average of 14 minutes. In consultation with staff, the project team then decided how they would deliver the project, including what technology they would use, and calculated the project cost and time frames.

3.20
The Investment Group expects project teams to consult with the relevant parts of the Council before submitting business cases. This helps to ensure that the right people have considered and commented on the business case. Staff told us this was an improvement on their previous experiences of Council projects.

A more rigorous approval process for business cases

3.21
The process for approving business cases appears to be working well. The Council has a multi-layered approval process for projects supporting the organisational strategy and for information technology projects (see Figure 6). Business cases are reviewed by the head of the online services programme and then by the subgroup before being submitted to the Investment Group.

Figure 6
Sequence for reviewing and approving projects

Source: Office of the Auditor-General, using information from Auckland Council.

3.22
Members of the subgroup told us they had declined to endorse business cases that were not yet good enough to go to the Investment Group.

3.23
The Strategic Portfolio Office and Corporate Strategy teams jointly review business cases submitted to the Investment Group. They provide the Investment Group with an assessment of each business case, any concerns, and a recommendation about whether the Investment Group should approve it. These assessments also score business cases against the criteria of desirability, feasibility, and viability.

3.24
At the start of each Investment Group meeting, members are briefed on the overall programme of projects under the organisational strategy and the effect of approving funding for the submitted business cases. The Investment Group then considers each business case and questions the project manager, business owner, and head of the online services programme. For example, for the Identity Management project, members expressed concerns about how benefits from the project would be monitored and asked for targets based on use.

3.25
In the minutes of Investment Group meetings, we saw examples of the Investment Group deferring business cases and allocating resources to help project teams revise their business cases before re-submitting them. Members of the Investment Group told us that they also sometimes decline business cases. In one instance, this was because the business case was not clear about where ongoing operating expenditure would come from.

3.26
Council staff described the current approval process as “maturing”. However, they also said that, despite this, it is more rigorous, provides better visibility of projects throughout the Council, and is generally an improvement on previous approval processes. This should help ensure that approved projects are co-ordinated and contribute towards achieving the Council’s strategic outcomes, including making its services more customer-friendly.

3.27
We suggest that the Council explore opportunities to improve the process for developing and approving business cases to avoid some of the issues identified at the Investment Group stage.

Elements of good practice in delivering projects

3.28
The online services programme has elements of good practice for project delivery, as identified in our 2012 report Realising benefits from six public sector technology projects. This good practice helped the Council to deliver projects that were largely on time, that had justifiable increases in costs, and that meet the needs of the Council and its customers.

3.29
The first part of the Identity Management project was delayed (see Figure 7), but this had a limited effect when the overall system was made available to the public. The Smart Forms project was delivered earlier than planned.

Figure 7
Summary of the planned and actual dates for making online services available to the public

Project | Planned date (in the business case) | Actual date
Smart Forms | 28 December 2016 | 12 December 2016
Identity Management (phase 1) | 17 October 2016 | 14 November 2016
Identity Management (phase 2) | 30 November 2016 | 12 December 2016

Source: Office of the Auditor-General, using information from Auckland Council.

3.30
The Council delivered both projects largely to budget, with justified increases in cost. The Council delivered the phases of the Identity Management project under budget (see Figure 8). The Smart Forms project needed an increase in budget that went through the appropriate approval process.

Figure 8
Summary of the planned and actual cost for the projects, as at May 2017

Project | Planned | Actual
Smart Forms | $500,000 (Revised to $595,000) | $595,000 (100% of revised budget)
Identity Management (phase 1) | $257,400 | $249,678 (97% of budget)
Identity Management (phase 2) | $152,000 | $109,440 (72% of budget)

Source: Office of the Auditor-General, using information from Auckland Council.

3.31
We identified elements of good practice in how the online services programme delivers projects. These elements include:

using a business-led, flexible, and “agile” approach;

working effectively with the right people, including those who will use the online service; and

using the right technology.

3.32
The online services programme uses a “minimum viable product” approach to deliver projects.² An example of this approach was the Council’s decision to have a “soft launch” of the Smart Forms project, with limited publicity to minimise the initial number of users. This meant that the Council could test the resilience and readiness of the system and get feedback from staff and customers without affecting many people if there were significant problems.

3.33
The projects in the online services programme consider the needs of the Council and its customers by involving the business unit throughout the project and using a co-design methodology during the development of business cases. Relevant business units were involved in developing and delivering a project by business unit staff working with project teams to identify benefits, having some business unit staff on the project team, and business unit staff providing advice as subject matter experts.

3.34
When preparing the Smart Forms and Identity Management projects’ business cases, both project teams took steps to ensure that the technology would be fit for purpose. Both projects used external software suppliers, selected through appropriate procurement processes. The Identity Management project team provided information to their suppliers on the design of webpages.

3.35
We heard positive feedback from Council staff about the online services programme. Staff from the business units said that they had a good relationship with project teams and that projects delivered what they wanted. Other interviewees said that Council staff used to try to avoid transformation. Now, the rest of the Council wants to work with the online services programme because of what the programme has delivered.

Managing project risks and issues

3.36
The Council appropriately managed risks and problems with the projects. Risks were identified within the projects’ business cases, and both projects had regularly updated risk registers. Risks were also identified at specific points within the projects, such as when procuring services or during testing.

3.37
The head of the online services programme has promoted a culture among the project managers of being open about any problems or challenges. All staff working on projects in the online services programme gather for quick daily meetings. This is a chance for participants to provide updates on their projects, discuss any issues, and informally share what they have learned.

3.38
We heard several examples of governance groups intervening to manage problems in projects. We also observed governance groups intervening in projects to minimise delays or ensure that any increase in cost was justified.

3.39
An escalation process for change requests supports intervention by governance groups. The effect of the requested changes on cost, scope, quality, and timelines decides who needs to approve it. For example, the subgroup needs to approve increases in the cost of a project of 5% to 20%. An appropriate authority approved the change requests for the projects we looked at, and status reports to the governance groups included information on the change requests.

3.40
However, the escalation process applies only to individual increases. The reporting we saw did not record the cumulative effects of the changes on a project’s cost, timelines, or scope. This creates a risk that a series of small increases with a significant cumulative effect may not be visible or fully appreciated when small, individual changes are approved. We suggest that the Council consider ways to provide greater visibility of the cumulative effect of approved changes to a project.

3.41
The Council appropriately managed issues that arose within the projects. The cost of the Smart Forms project increased by $95,000 (about 19%) because a planning oversight meant that the business case did not include the cost of setting up the Smart Forms platform. The budget increase went through a change request process, and the subgroup approved it after confirming that they had made the right decision about which technology to use. We were told that the Council had learned from this incident and now does more due diligence of costs when preparing business cases.

3.42
One of the identified issues for the Identity Management project was using RealMe³ as a way for users to log in to access the Council’s services. This was because the Council needed to integrate RealMe into the new platforms and to migrate existing RealMe users on to these. The project had two main problems with integrating RealMe. These were:

getting enough information from the Department of Internal Affairs about the requirements for using RealMe; and

complying with RealMe requirements for error messages.

3.43
Working with an external contractor and the Department of Internal Affairs, the project team overcame these problems, with some delays to the project. The project team told the relevant people about these problems and used the change request process to get decisions made to resolve them.

Reporting to governance groups about projects needs to improve

3.44
The Council needs to improve the accuracy of reporting and find ways to track the performance of a project over time. Poor reporting risks the governance groups not getting the information they need to make good decisions in a timely manner and can affect their ability to effectively oversee projects.

3.45
Under the online services programme’s previous governance arrangements, both the steering committee and the subgroup were responsible for monitoring the delivery of projects. Projects in the online services programme provided dashboard-style reporting to the steering committee, the subgroup, or both. Reporting covers risks, issues, timelines, cost, and dependencies.

3.46
We asked members of the governance groups about the usefulness of the reporting. Most members said that the reporting does its job of raising issues with projects. However, a senior manager also told us that one of the limitations of the current reports was that they did not provide any trend data on the project’s progress.

3.47
We found numerous inaccuracies in reports from several projects, including the Identity Management project. These inaccuracies included a project being 80% complete in one month but then 50% complete in the next, and incorrect budget figures. When we asked about these inaccuracies, we were told that they were mistakes. Having trend data would have helped reduce inaccuracy, because some inaccuracies became clear only when we tracked reporting over time.

3.48
Reporting without this wider context or trend data can provide only “a snapshot in time”. For example, a project team would report that they had spent 45% of their budget but not provide any information on whether this is what they expected to have spent at that stage.

3.49
All projects approved by the Investment Group now need to use the same reporting tool as the rest of the Council, including projects that are part of the online services programme. The Council expects that this will help mitigate the issues we found with the accuracy of the reporting.

Recommendation 2

We recommend that Auckland Council provide all governance groups with reports that are accurate, enable progress to be tracked over time, and enable the people who govern projects to maintain effective oversight of risks and issues.

Effectively moving projects to the responsible business units

3.50
Once a project in the online services programme is completed, it is transferred to the responsible business unit in the Council. This handover needs to be thorough and effective so that staff are confident in delivering services in a new way and the full benefits are achieved for Aucklanders and the Council.

3.51
We found that the transition of the projects, once launched, to “business as usual” was generally effective, with no significant issues.

3.52
In preparation for moving the projects to the responsible business units, both project teams did testing, prepared handover documentation, trained staff, and provided troubleshooting guides. The Council’s reviews of the projects identified aspects that went well during this stage, including:

There was a smooth transition to the new Identity Management systems.

Staff could see the results of all the testing activities.

Quick daily meetings about defects kept the project teams’ focus on problems and priorities.

3.53
In the project teams’ reviews, they identified things that they could have done better. Both project teams identified the need for a stable system for project testing and staff training. Time constraints also meant that different types of testing activities were running in parallel.

3.54
Both projects had issues before the online services were made available to the public. The Identity Management project’s main issue was that it needed to train staff at the contact centre at the same time as the staff had training scheduled for another project. Instead of delaying the “go live” date, the Council trained a different group of staff to provide support until the contact centre staff could take over.

3.55
The launch of the Smart Forms project was delayed because staff had not seen how the forms would work in practice. They identified some problems that needed to be fixed before they were comfortable with people using the forms. We were also told that staff were used to an environment in which change is uncommon and slow, in contrast to an “agile” environment where continuous improvements are made after a project has been delivered.

3.56
Neither project experienced significant problems after online services were made available to the public. The few problems that did occur were quickly resolved. Council staff from the business units, as well as the project teams, considered the transition to be effective.

2 A minimum viable product has just enough features to satisfy early customers and to provide feedback for future improvements.

3 RealMe is a collaboration between New Zealand Post and the Department of Internal Affairs that provides a secure way to verify a person’s identity online.

Part 4 – Achieving the expected benefits and learning from projects

4.1
In this Part, we discuss:

what has been achieved so far;

tracking and monitoring benefits; and

learning and applying lessons.

Summary of findings

4.2
There are indications that customers and the Council are receiving benefits from the online services programme. However, not all of the outcomes in the customer-friendly services focus area are being tracked and reported to the subgroup. This means that the subgroup does not have a complete view of how the online services programme and other customer-focused initiatives are helping the Council to provide customer-friendly services.

4.3
The Council has not made as much progress as it had planned in achieving benefits from the Smart Forms project. There is also limited information about whether the Council is getting the expected non-financial benefits from the Identity Management project – for example, whether the project has made it easier for customers to access the Council’s online services.

4.4
The Council should improve its tracking of the expected non-financial benefits from projects to give it a better view of whether projects are achieving all of their expected benefits, including for customers.

4.5
Although lessons from projects have been identified and applied to other projects, there is a risk that this will not happen consistently for every project.

What has been achieved so far?

The online services programme is delivering improvements for customers

4.6
By the end of June 2017, the Council had “digitised” 22% of its most common transactions. This was below its goal of 27% of transactions by this date because the Council was focused on the launch of another Council-wide project, New Core. However, although the Council has not made as much progress as planned, it is now easier for people to access services online. The Council expects that, with the completion of New Core, more progress can be made, especially with back-end processes.

4.7
Customer feedback about smart forms has been positive. The Licensing and Compliance Services unit surveys people at three different points during the online application process. Staff from this unit told us that they receive positive feedback about the smart forms through the surveys and feedback given to frontline staff.

4.8
From 10 April 2017 to 30 July 2017, there have been over 30,000 log-ins using Identity Management to access the Council’s online services. However, based on the reporting that goes to the subgroup, it is not clear whether the expected non-financial benefits from the Identity Management project are being achieved (see paragraph 4.25).

Financial benefits from projects

4.9
Overall, projects are delivering financial benefits to the Council. Figure 9 shows the estimated net financial benefits from approved projects that are part of the customer-friendly services focus area. From 2017 to 2019, the Council estimates a total benefit of about $9.3 million.

Figure 9
Estimated net financial benefits to Auckland Council from approved projects under the customer-friendly services focus area, 2017-19

Type of financial benefit | 2017 $ | 2018 $ | 2019 $ | Total $
Cost reduction | 144,308 | 1,101,209 | 1,455,090 | 2,700,607
Increase in revenue | 291,878 | 718,256 | 1,273,964 | 2,284,098
Cost avoidance | 955,895 | 1,628,086 | 1,754,346 | 4,338,327
Total | 1,392,081 | 3,447,551 | 4,483,400 | 9,323,032

Source: Office of the Auditor-General, using information from Auckland Council.

4.10
More than 70% of applications need to be completed online for the Council to get financial benefits from the Smart Forms project. The Council planned to reach this target by July 2017. It has not done so.

4.11
The percentage of applications completed online initially increased from December 2016 to February 2017 (see Figure 10). Since then, an average of 25% of applications have been completed online. Different types of applications have significantly different online application rates. For July 2017, the highest rate was 30% and the lowest was 11%.

Figure 10
Percentage of smart form applications completed online each month, from December 2016 to July 2017

Source: Office of the Auditor-General, using information from Auckland Council.

4.12
Representatives from the business unit and the online services programme meet regularly to identify how they can increase the percentage of applications completed online.

4.13
The Identity Management project does not have any direct financial benefits (see paragraph 4.22) but instead enables Council services to be accessed online.

Other benefits from the online services programme

4.14
Council staff said that the online services programme was delivering benefits that were not in its business case, such as:

better information on the activities of business units;

helping to change the Council’s culture so that staff can help the people who need it most;

better capability to extract and analyse data to support the advice given to customers, third-party stakeholders, and elected officials;

improved reporting to better identify issues in business units; and

having an overview of all the Council’s services.

The Council has made changes to help get the expected benefits

4.15
The Council has made changes to help it understand whether the expected benefits from projects are being achieved. For example, the Council has a new review process to ensure that it is achieving the benefits outlined in project business cases. This review looks at benefits gained from a completed project and identifies any opportunities to achieve further benefits. The first review looked at an online booking system for Council-managed venues, including seeking feedback from users. The Council plans further reviews, including on the Smart Forms project.

4.16
Council staff said that achieving and measuring benefits was an area of focus for the Council. We encourage the Council to continue to improve its understanding of whether the expected benefits are being achieved.

Improving the tracking and monitoring of benefits

Reporting progress on performance measures

4.17
The online services programme’s business case did not specify any performance measures that were separate from the performance measures of the customer-friendly services focus area. Instead, the business case linked the programme’s achievements to the organisational strategy outcomes and associated performance measures. For this reason, we looked at the reporting of the performance measures under the customer-friendly services focus area.

4.18
The Council has set up systems to track its progress towards putting 70% of the Council’s most common transactions online. This is the only performance measure for the customer-friendly services focus area that is regularly tracked and reported to the subgroup.

4.19
Two of the other performance measures (increasing the Council’s customer experience score and decreasing the processing time for consents) are reported in quarterly performance reviews to the Council’s Executive Leadership Team. We did not see any reporting for the last two performance measures (improving the Council’s customer effort score and decreasing the cost to serve).

4.20
In our view, the Council needs to report progress towards achieving all of the performance measures of the customer-friendly services focus area. This would provide a more complete view of the effects of the online services programme and other initiatives in achieving the Council’s aims under the focus area, including making it easier for customers to access and use services.

Reporting of non-financial benefits can be improved

4.21
Expected benefits for projects in the online services programme are identified in their business cases. The expected financial benefit for the Smart Forms project was $135,000 in gross savings if 70% of applications were submitted online. Expected non-financial benefits included a better customer experience and allowing staff to focus on higher-value work.

4.22
The Identity Management project business case did not specify any financial benefits. Instead, financial benefits were expected to be realised by other Council services being made possible by a log-in service and by making online services easier for customers to access. However, as part of approving the business case, the Investment Group asked about setting targets related to use.

4.23
After the projects were launched, metrics were set up to track benefits. For the Smart Forms project, the main metric is the percentage of applications filed online. Realised benefits are then reported to the subgroup and the Investment Group, which holds overall responsibility. The online services programme also provides informal reporting on benefits – for example, through email updates to members of the subgroup.

4.24
Some Council staff thought that the reporting on project benefits was good overall. However, we found that there have been some problems. For example, it was discovered that reporting on the use of Identity Management was inaccurate, and reliable reporting began only in April 2017 – four months after the project was launched.

4.25
We did not see any reporting to the subgroup on the expected non-financial benefits from the Identity Management or Smart Forms projects, such as reporting on whether people find it easier to access the Council’s online services now they can use a single log-in account.

4.26
Some Council staff said that benefits for people using the Council’s services need to be better defined and measured. For example, it would be useful to assess whether the Council has made the process of applying for services easier and better. The need to measure benefits to customers, and non-financial benefits generally, is a challenge for many organisations.

4.27
The Council told us that it is working to measure the non-financial benefits of projects. Reporting on the non-financial benefits of projects would give the subgroup and other governance groups a better picture of whether projects are realising all of their expected benefits for the Council, its staff, and the public.

Recommendation 3

We recommend that Auckland Council review its monitoring and reporting on progress towards achieving the financial and the non-financial benefits from projects and how projects contribute towards achieving all of the objectives of the customer-friendly services focus area.

Lessons are being applied

4.28
Throughout its work on the online services programme, the Council has identified and applied lessons from the Identity Management and Smart Forms projects. Lessons are informally shared through the online services programme’s quick daily meetings, informal project manager lunches, and technical meetings with other parts of the Council. One of the project managers said that there is not a day where there is not an “Aha!” moment from the daily meetings.

4.29
There is also a process for formally recording lessons. At the end of the Identity Management and Smart Forms projects, there were formal post-project reviews and workshops on what went well and what could been done better. Some of the lessons identified from these included:

the importance of regular communications and updates;

working with the responsible business unit during the early development stages;

including customer testing earlier; and

needing a clearer minimum viable product definition so that the project team knew what to prioritise.

4.30
The Council is applying these lessons to other projects. The Smart Forms project’s business case identified the opportunity to use its lessons on technical delivery and customer insights to reduce the delivery risk of the Consenting Made Easy programme. Council staff told us that this is now happening.

4.31
The Council intends to use lessons from the Identity Management project for the My Account project.

4.32
However, Council staff told us that sharing and applying lessons can depend on who is involved. This could lead to the lessons being applied inconsistently. We suggest that the Council look at ways to ensure that lessons are applied to relevant projects, regardless of who is involved.