Purpose

To determine whether Empire State Development (ESD) has established adequate internal controls to oversee, monitor, and manage contracted marketing services, including the extent to which ESD employs appropriate performance measurement systems that provide management with information about program effectiveness and cost-efficiency. Our audit scope covered the period December 1, 2011 through November 6, 2014.

Background

ESD’s mission is to promote a vigorous and growing economy, prevent economic stagnation, encourage the creation of new job opportunities, increase revenues to the State and its municipalities, and achieve stable and diversified local economies. To this end, ESD plans and conducts programs to promote travel, tourism, and business investment. In support of these programs, ESD awarded a contract in December 2011 to BBDO USA LLC (BBDO) for an amount not to exceed $50 million, as its non-exclusive, full-service advertising, marketing, branding, media, and communications agency. Due to the value of the contract and the appropriated State funds involved, it was reviewed by the Office of the Attorney General to ensure that it was in proper form and included all provisions mandated by law, and by the Office of the State Comptroller to ensure, among other things, that fair, proper procurement procedures were followed and that the cost of the procured services was reasonable on its face. Neither of these Offices was responsible for determining whether the contract was the most appropriate method of accomplishing the intended purpose or whether it would successfully achieve the program goals ESD intended, which in this instance were not set forth in the contract. Instead, ESD management remains ultimately responsible for ensuring that the contract is necessary and accomplishes its intended purpose, and for establishing a system of internal controls to monitor, oversee, and manage the contract. These controls should include employing a performance measurement system that provides management with timely and accurate information about contract performance and outcomes.

By September 2014, ESD had executed four amendments to this contract, bringing the total contract amount to $211.5 million. Of this, $36.5 million is specifically targeted to promote tourism and business in the wake of Hurricane Sandy. The remaining $175 million is available to be spent at ESD’s discretion. As of October 2014, ESD had committed $182.9 million of the $211.5 million, including $33.2 million of the $36.5 million set aside for post-Hurricane Sandy campaigns.

Key Findings

• ESD has an appropriate system of internal controls in place to ensure that it receives the advertising services for which it paid, and that those services are appropriately priced in keeping with the terms of its contract with BBDO. However, these controls focus on the specific services that are provided (i.e., outputs) rather than on the results that are achieved (i.e., outcomes).
• ESD has not quantified what it expects to achieve from its advertising efforts, except in the broadest terms like increasing tourism or creating jobs. As a result, ESD does not have an appropriate system to monitor, measure, and evaluate the extent to which any accomplishments or outcomes resulting from these efforts compare to expectations.
• ESD is unable to evaluate the extent to which its $211.5 million planned investment has contributed toward achieving the purposes of the underlying programs or whether it has been cost-effective. In fact, ESD officials reject the idea that advertising programs should be measured against the results achieved by the underlying programs they aim to benefit, except in the broadest terms.
• ESD does track certain measures that officials believe are indicative of possible program impact, including website traffic, attendance at certain tourism attractions, and business leads developed. However, officials consider any reported improvement in these measures, as well as other factors such as the extent to which people perceive New York to be a good place for business development or to visit, as evidence that the advertising programs are a success. In doing so, ESD officials have not considered ways to account for any other factors that may influence these measures, or to assess whether the State has received an appropriate return on its investment in these marketing services.

 Key Recommendations

• Develop strategic plans that include performance measures for monitoring the extent to which marketing efforts have a positive impact on desired outcomes, such as ESD’s stated goal of improving the perceptions of New York as a good place to visit and for business development.
• Set specific targets, goals, and benchmarks for evaluating performance outcomes and use these measures to monitor program performance.
• Regularly evaluate the program outcomes associated with marketing efforts and use this information to periodically adjust program goals, strategies, and resource allocations.

Other Related Audit/Report of Interest

Empire State Development: Oversight of International Offices (2012-S-7)