Part 5: Auckland Transport could better understand its passengers' experiences
5.1
Performance reporting is the main way that public organisations explain how well they have used public money to deliver services and achieve outcomes.
5.2
Analysing performance information can help organisations and stakeholders deepen their understanding of how well services are being delivered and what progress is being made towards goals.
5.3
We expected Auckland Transport to have enough information to understand and monitor people's perceptions, expectations, and experiences of safety. We expected to see that Auckland Transport regularly analyses this information to identify where it can make improvements and that it acts on that information where appropriate.
5.4
Although Auckland Transport collects a significant amount of timeliness and location information about its bus, train, and ferry services, it does not report this information in a way that corresponds with how people use these services.
5.5
Although Auckland Transport collects a range of information about personal safety incidents, that information is not routinely brought together for analysis. In our view, this is important to ensure that Auckland Transport has a clear understanding of how it is making public transport services safe. Auckland Transport is aware of these issues and has made recent improvements to build its data and insights capability.
Auckland Transport measures punctuality and reliability
5.6
Auckland Transport commissions external research and surveys people about their public transport experiences, including through an on-board customer satisfaction survey. Some of the surveys include questions about reliability, punctuality, and safety.
5.7
In addition, Auckland Transport can track the movement of buses, trains, and ferries in real time, including their arrival and departure from stops, stations, and wharves. Automatic Vehicle Location devices attached to buses collect data and transmit it to Auckland Transport's Command Centre system.
5.8
Train data is reported through the COMPASS performance software maintained by Auckland One Rail. Auckland Transport can monitor the time each train service takes, compare the service to the schedule, and analyse schedule variances.
5.9
Auckland Transport uses GPS information from Automatic Identification System Transponders on each ferry, which is also transmitted to Command Centre. We were told that Auckland Transport tracks all ferries in real time, but the data is not always accurate.
5.10
Tracking can be inaccurate because skippers bypass a wharf if no-one is waiting, operators can misallocate a vessel to a route, and vessels pass through other wharf GPS locations on the way to berth. Auckland Transport told us it is working to improve the accuracy of the data.
5.11
Auckland Transport uses the tracking information it collects to measure services' reliability and punctuality for all types of public transport. Figure 2 sets out Auckland Transport's definitions of reliable and punctual services.
Figure 2
Auckland Transport's reliability and punctuality measure definitions for all services
| Measure | Definition |
|---|---|
| Reliable | A service that operates and leaves within -.59 to +9.59 minutes of schedule |
| Punctual | A service that both leaves its first stop between -0.59 minutes and +4.59 minutes of the scheduled start time and arrives at the final stop no later than +4.59 minutes of the scheduled arrival time |
Auckland Transport has made improvements to its performance measures
5.12
Our 2024 guidance Local government planning and reporting on performance explains that, when defining measures, people should consider:
- the aspects of service and performance that are most important to the community;
- whether the measures are relevant, understandable, and verifiable;
- whether the measures and targets will enable readers to assess the effectiveness of a council's main policy and investment decisions;
- whether the measures and targets reflect the financial significance of the activity; and
- the most important dimensions of performance in delivering services, such as quantity, responsiveness, quality, reliability, timeliness, and accessibility.
5.13
Auckland Transport has improved its performance measures over time. For example, until 2022/23, Auckland Transport measured punctuality at only the first stop. In 2022/23, Auckland Transport revised the measure to include the last stop to provide a better picture of punctuality for the whole trip.
5.14
In 2023/24, Auckland Transport introduced a reliability measure that considers services that were cancelled or did not start their route within a reasonable time (see Figure 2).
5.15
Auckland Transport staff told us that it has increased the performance reporting team's resourcing and is focused on ensuring that its measures are meaningful and that its definitions for punctuality and reliability are widely understood.
5.16
Planned improvements include new measures to assess the timeliness of communications during service disruptions. Currently, when disruptions are communicated, Auckland Transport takes about 22 minutes to inform the public about a disruption. It wants to improve this to an average of 10 minutes. It is also considering measuring the time people lose on a trip because of a disruption.
5.17
These improvements are promising because they are more focused on people's experiences, but we think there are further opportunities for improvement.
There are opportunities for performance information to better reflect passengers' experiences
5.18
Auckland Transport publicly reports its performance at a high level. For example, its 2023/24 annual report set out the results for reliability and punctuality measures for buses, ferries, and trains combined as 12-month rolling averages, weighted by patronage.
5.19
The reliability measure is new, and the punctuality measure has been revised, so information is not able to be presented over time and there is no way for a reader to understand whether performance has improved or declined compared with previous years.
5.20
The combined result gives an overall impression that public transport in Auckland is reliable and punctual. For example, for 2023/24, 94% of services were reported as reliable and 88% as punctual. This is accurate according to the definitions that Auckland Transport uses for these measures (see Figure 2).
5.21
However, in our view, these measures could more accurately reflect the way people typically use services. For example, people might not use public transport from the first stop of a route to the last. A measure that looked at or factored in the punctuality of services at interim stops might better align with a passenger's actual experience.
5.22
Monitoring information in this way could also lead to better performance analysis and, if Auckland Transport used it to make improvements, better outcomes for Aucklanders.
5.23
Auckland Transport told us that it is moving towards a "turn up and go" network on key bus (and some train) routes where people do not need to consider the timetable. In that context, it told us that adherence to schedule is not a good way to measure performance because its rapid and frequent networks have high-frequency services (which means that although any one bus or train might not be punctual, the frequency arriving minimises the impact).
5.24
For example, Auckland's link bus services do not run to a timetable. Auckland Transport's website says that these can be expected at stops every seven to 15 minutes (although actual frequency might vary), depending on the specific service.
5.25
This makes sense. However, there are still routes that are not part of the frequent network corridor or do not yet have high-frequency services in place. In these circumstances, timetables are the practical commitment Auckland Transport currently makes about when people can expect public transport services to arrive. In our view, they should be as accurate as possible, and public transport should run to these times as much as practicable.
5.26
Auckland Transport told us that it plans to improve its reporting and intends to report reliability by type of transport in its 2024/25 annual report.
Analysing long-term trends is important to understanding performance over time
5.27
We analysed information that Auckland Transport provided to us to understand the reliability of scheduled trips for services that actually ran (that is, were not cancelled) between 2018 and 2024.10
5.28
Figure 3 shows how reliable each type of transport was according to Auckland Transport's definition (see Figure 2) from 2018 to 2024.
Figure 3
Auckland Transport's monthly reliability scores, July 2018 to June 2024
5.29
Information that Auckland Transport publicly reports indicates a 94% reliability score for buses, trains, and ferries for 2023/24. Our analysis shows the reliability for each type of transport over six years. In our view, showing long-term performance information better allows people to compare their perceptions of performance and provides transparency of whether performance has improved over time.
5.30
Figure 3 shows that bus reliability was best in late 2021, dropped during 2022 and 2023, and returned to 2021 levels in 2024. Ferry reliability was best in late 2018 and worst during September and October 2021. Train reliability has been decreasing slightly over time since 2018 but was at its worst in February 2023.
5.31
We also analysed the data to see whether the services that ran between 2018 and 2024 were punctual. Figure 4 shows how punctual each type of transport was according to Auckland Transport's definition (see Figure 2) from 2018 to 2024.
Figure 4
Auckland Transport's monthly punctuality scores, July 2018 to June 2024
5.32
Figure 4 shows that punctuality decreased slightly between 2018 and 2024 for all types of transport.
5.33
Our analysis shows bus punctuality was best in early 2022 and declined during 2022 and 2023. Ferry punctuality was best between 2020 and 2022, but its performance has decreased since late 2022. Train punctuality is more volatile than bus and ferry punctuality. It has decreased since 2018 and early 2019, except for late 2021/early 2022.
5.34
In our view, it is important that Auckland Transport monitors the long-term trend so it can understand whether its planned initiatives are making a difference.
5.35
Auckland Transport publicly reports punctuality only as a combined score for train, bus, and ferry services. In our view, there is value in disaggregating Auckland Transport's high-level performance information to allow people to understand the performance of the type of transport that they use.
Measures should reflect the way passengers use services
5.36
We also analysed all bus trips from 1 July 2022 to 30 June 2024 to understand how well buses adhered to their entire schedules (rather than just the first stop and the last stop). This is because passengers might not travel from the first stop to the last.
5.37
Figure 5 shows the percentage of stops for all bus routes that met punctuality targets and whether the others were early or late.
Figure 5
Adherence to schedule for all stops for Auckland Transport buses, 2022/23 to 2023/24
5.38
Most buses (about 70%) were punctual. Slightly more than 30% were either early or late to any given stop in both 2022/23 and 2023/24, with more late buses in 2023/24.
5.39
We also looked at how buses from 1 July 2022 to 30 June 2024 have adhered to schedule (between one minute early and five minutes late according to the timetable) at the first stop, an interim stop, and the last stop. We found that, on average, 10% of trips that were punctual at the first and last stop were not punctual at the stop in between.
5.40
Although including the last stop in Auckland Transport's punctuality measure has been an improvement, considering intermediate stops – or middle stops at key interchanges – might better reflect a passenger's experience of bus punctuality.
| Recommendation 4 |
|---|
| We recommend that Auckland Transport review the information it holds about passenger experience to help inform service improvements and improve performance reporting (to allow the public, Auckland Council, and others to better understand how reliable public transport is in Auckland). |
Schedule adherence data shows differences across Auckland
5.41
Auckland is a large city and many people who use public transport only travel to parts of it. We wanted to understand differences in how well buses adhere to their schedules in different areas of Auckland. We did not see evidence of Auckland Transport carrying out this kind of analysis.
5.42
Figure 6 shows the percentage of buses that met punctuality targets and how early or late others were in different areas of Auckland.
Figure 6
Adherence to schedule for all stops for Auckland Transport buses, 2022/23 to 2023/24, by area
| Area | Buses more than 5 mins early | Buses 1 to 5 mins early | Buses 1 min early to 5 mins late (Punctual) | Buses 5 to 10 mins late | Buses more than 10 mins late |
|---|---|---|---|---|---|
| Central | 6.1% | 17.4% | 62.9% | 13.4% | 0.0% |
| East | 3.7% | 19.1% | 68.7% | 8.4% | 0.0% |
| North | 2.2% | 12.8% | 76.3% | 8.5% | 0.0% |
| South | 3.3% | 13.7% | 70.7% | 12.2% | 0.0% |
| Waiheke | 0.1% | 4.6% | 79.8% | 15.3% | 0.1% |
| West | 5.4% | 14.4% | 66.8% | 13.3% | 0.0% |
5.43
Figure 6 shows that buses on routes in central Auckland were the least likely to remain on their schedule. Buses on Waiheke Island were the most likely to remain on their schedule. However, buses on Waiheke Island that did not remain on their schedule were most likely to be the latest.
5.44
Some of the results from our analysis are unsurprising. For example, traffic congestion is likely to be higher in central Auckland than other areas at certain times, and this can affect bus punctuality.
5.45
In our view, making information available by route or area would allow people to better understand the performance of services they use.
Figure 7
Number of bus routes in Auckland, by area
There are opportunities to better understand personal safety
5.46
As we describe in paragraph 3.34, personal safety is about people feeling able to use public transport without threat, fear, or harm. To understand how safe passengers are when they use public transport in Auckland, and how safe they feel, Auckland Transport needs to collect information about personal safety. It also needs to understand any shifts or trends in that information.
5.47
Auckland Transport collects information about personal safety from a range of sources. The customer satisfaction monitor run by its Customer Insights team asks respondents about how safe they feel on public transport.
5.48
In Auckland Transport's Brand and Reputation Tracker (an externally run survey that asks Aucklanders about their perceptions of Auckland Transport and its work), respondents are asked about their perception of safety and whether this has changed, if they feel worried using public transport, and whether safety concerns affect how they use public transport. Auckland Transport also carries out a quantitative survey of passengers to test attitudes towards solutions to help people feel safer on public transport.
5.49
Other key sources of information include incident reporting and end-of-shift reports from Transport Officers, operator staff, and security. People can also make complaints about personal safety matters to Auckland Transport.
5.50
Auckland Transport collates complaints in a customer relationship management database. Auckland Transport's health and safety system, Synergi, and its incident recording system, RiskShield, also contain information about personal safety incidents.
5.51
A survey that Auckland Transport ran between July 2022 and June 2024 indicated that people generally felt safe waiting for or travelling on a service.
5.52
At the same time, information from the Brand and Reputation Tracker indicates that concerns about safety when using public transport are increasing and influence when people will use it (for instance, they might not use it at night or when travelling in certain areas).
5.53
Personal and public safety is also a focus for local boards and they have asked Auckland Transport to prioritise it. Their suggestions to Auckland Transport include increasing the number of CCTV cameras on buses and at stations, and increasing the number of Transport Officers on buses.
5.54
We consider that Auckland Transport could improve its data collection and analysis to gain more insights into personal safety.
5.55
Because data is collected from various sources, comes from various parties, and is in different formats, it is not easy to bring together to provide an accurate overall picture of personal safety on public transport. We were unable to easily access information about how many personal safety incidents have happened on public transport services, what type of incidents they were, or what type of transport they happened on. Trend information for personal safety is not readily available.
5.56
Most of the information we did see was health and safety reporting or operator reporting of personal safety incidents involving operator staff, drivers, and crew. Violence, threats, and aggression were the health and safety risks that employees and contractors reported the most. They made up 81% (266 of 330) of total reported events from August 2023 to July 2024 in the health and safety data we were provided.
5.57
Auckland Transport uses an Enterprise Data Warehouse to store data, including reliability, punctuality, and patronage information. Auckland Transport has plans to integrate Crimestoppers (a tool for passengers to report crime, fare evasion, and anti-social behaviour) into the AT Mobile app and link it to its Enterprise Data Warehouse.
5.58
This is positive. It is likely to help Auckland Transport gain more insights into personal safety matters and better categorise incidents. However, in our view Auckland Transport still has more work to do to better integrate other sources of information to generate insights in this area.
Auckland Transport has increased its data capability
5.59
We have seen evidence that Auckland Transport is improving its data and insights to enable more effective interventions. For example, Auckland Transport's Public Transport Advisory Group is made up of representatives from disability advocacy groups. Based on feedback from this group, Auckland Transport has introduced alerts on the AT Mobile app to let users know when station lifts are not working, and is developing a website that will show the status of lifts at all stations.
5.60
Auckland Transport has also strengthened its reliability and punctuality expectations in its contracts with operators in line with improvements in its ability to track their services, most recently with ferries. As discussed in Part 2, the monthly operational performance report helps Auckland Transport understand its operators' performance.
5.61
In our view, the better Auckland Transport's data and information is, the more effective its interventions will be.
5.62
In December 2023, Auckland Transport set up an insights-focused data team in the Public Transport and Active Modes directorate. The team now regularly reports on the performance of public transport services.
5.63
We were told that the team has a specific remit to understand what Auckland Transport could do to improve public transport's reliability. We understand the team has developed a framework to transform raw data into insights and enable data-driven decision-making, communication, and strategic planning. The data team also produces the monthly operational performance report.
A Customer Insights team gathers and analyses public feedback
5.64
Auckland Transport's Customer Insights team is responsible for bringing the public's voice into Auckland Transport. Previously it was its own division. It now reports to the Customer and Network Performance directorate. We were told this change was made to encourage the rest of Auckland Transport to better consider passengers' experiences in their work.
5.65
The Customer Insights team collates information from various sources to understand people's expectations of Auckland Transport. The team runs satisfaction surveys, develops case studies, gives input into business cases for improvement projects, and examines customer-related issues in Auckland Transport's work.
5.67
We were told that although the Customer Insights team has always worked closely with the Public Transport and Active Modes directorate, they are still establishing how they will work together with the new insights-focused data team.
5.68
In our view, the work of both these teams is important for Auckland Transport to gain a full understanding of overall performance. Although Auckland Transport has met its reliability and punctuality targets in 2023/24, it did not meet its customer perceptions targets. The Customer Insights team carried out research that indicates that Aucklanders would like public transport to be fast, frequent, and reliable, but some do not perceive that it is.
5.69
Auckland Transport needs to bring these aspects of performance together to understand the changes it needs to make to its public transport services and why.
10: Auckland Transport did not report reliability information until 2023/24. For our analysis, we sourced punctuality and reliability information from 2018-2022 so that we could understand any longer term trends in the data.