Objective

To determine whether New York City Transit’s (Transit) new metrics provide customers with better information about their trips and provide a more accurate picture of Transit’s performance. The audit covers the period between July 1, 2017 and November 30, 2018.

About the Program

The Metropolitan Transportation Authority (MTA) is a public benefit corporation chartered by the New York State Legislature in 1965. Transit is an agency of the MTA that operates New York City’s subways and the majority of its bus service.

Under the Public Authorities Law, the MTA is required to issue an annual report on its mission statement, measurements, and performance indicators. One of the goals the MTA cites in its annual report is to provide on-time and reliable service to customers.

In July 2017, the MTA prepared a Subway Action Plan (Plan) to improve service. One of the major changes in the plan was an agreement to implement new customer-focused performance measures with a goal of making these metrics more relevant and easier to understand by subway customers. In September 2017, Transit introduced the new metrics¹: Additional Platform Time (APT), the average time that customers wait at a station beyond their scheduled wait time; Additional Train Time (ATT), the average time customers spend on board a train beyond their scheduled travel time; and the sum of these, Additional Journey Time (AJT). AJT is the key component of Customer Journey Time Performance (CJTP), the percentage of customer trips completed within five minutes of the scheduled time. It was not reported as a performance measure on the MTA’s Subway Performance Dashboard until November 2019. Section 1276-f of the Public Authorities Law, enacted as part of the fiscal year 2019-20 State Budget, requires the MTA to publicly report APT, ATT, AJT, and CJTP.

Key Findings

• Transit’s new customer-focused measures do not appear to meet the Plan’s goals. For a metric to be relevant, it should be closely connected to the goals, easily understood, and straightforward.
• Transit does not disclose the significant assumptions made during its calculation of subway APT, ATT, and AJT or the limitations of the data used.
• APT and ATT depend on knowing where and when a customer entered and exited the system. However, Transit’s automated fare collection system does not require customers to swipe out of the system, so Transit does not know where and when each customer’s trip ends. Without this information, the new measures rely heavily on a series of assumptions, each of which introduces uncertainty and complexity to the model.
• APT, ATT, and AJT represent the average additional time for each leg of a trip. A customer who transfers multiple times would have to add the times for each leg of their trip together to obtain the total APT, ATT, or AJT of their trip. However, based on the published definition, this is not clear.

Key Recommendation

• Evaluate whether APT, ATT, and CJTP meet the goals of the Plan and disclose the assumptions and margin of error for each assumption.
   

¹ It is the MTA’s practice to refer to performance measures interchangeably in its documentation as measures, indicators, or metrics.