Performance Based Technology Procurement

 Gerald H. Goldberg and Ralph Shoemaker

[tab name=”MEDIA COVERAGE”]Coming Soon[/tab]

[tab name=”VIDEO”]Coming Soon[/tab]

[tab name=”IMPACT”]Coming Soon[/tab]

[end_tabset]

The Problem

Any fundamental change in government must be founded on significant opportunity or need. In the 1990s, California faces both. Social, economic, and technology trends make it increasingly imperative for the state government to replace its outdated technology infrastructure in order to meet changing business needs. However, the long-term fiscal crisis has left the state with few funds to reinvest in infrastructure. In addition, large high-profile project failures have cost the state millions of dollars and made large systems a particularly “hard sell” for funding approval.

By early 1993, the California Franchise Tax Board (FTB) realized that the political environment for information technology (IT) projects was changing. State revenue streams were drying up as a result of the nationwide recession, and we predicted that little or no funding would be available for project work. At the same time, the state auditor was actively investigating a $155million automation project at the California Department of Motor Vehicles (DMV). Serious problems reported to the legislature engendered a loss of confidence that the state could successfully complete large automation efforts. Members of the legislature became unwilling to fund large projects and began to consider new and more stringent project controls.

In this environment, IT projects were viewed as costs to be reduced or eliminated rather than opportunities for efficiency and savings. FTB was in the planning stage for major project initiatives that were critical to the agency’s ongoing success. We clearly needed to undertake and fund technology projects in a way that would address the concerns of the state legislature and the control agencies regarding costs and risks. FTB became active in the debate and the search for solutions to resolve this critical government reform issue.

In September 1994, a report issued by California Governor Pete Wilson’s Task Force on Government Technology Policy and Procurement addressed the issue of technology procurement:

For decades, public trust has been based on the principle of checks, balances and controls. When failures occur, the typical response has been to add more layers of control. As layers of control are added, the processes instituted to protect the public trust become so cumbersome as to constrain the ability to effectively manage risk….The policies and procedures instituted to ensure that IT expenditures are appropriate have created an environment in which it takes too long to develop an IT solution from conception to implementation; problems or mistakes are not quickly surfaced; projects are subject to delays and cost overruns; more appropriate technologies are often bypassed in favor of an outdated solution; and an adversarial relationship between the State and its vendors prevails.1

The findings of the task force were based in large part on significant state IT project failures. Most notable was the widely publicized DMV automation project failure. Ultimately, the state paid $44 million for computer hardware that was unusable because the software programs could not be delivered. The DMV project had used the traditional procurement approach to purchase separate components the agency felt were needed. The state lost not only the investment in hardware and related costs but, more significantly, it lost the potential benefits of the project in improved government services to the public.

Unfortunately, this was not an isolated incident. This type of failure has been repeatedly identified as a significant risk of the traditional procurement process. Studies at the John F. Kennedy School of Government’s Taubman Center at Harvard University have documented similar findings:

The government IT procurement process typically focuses almost exclusively on pre-specified technical requirements. The vendor is responsible for supplying tools, not for producing organizational results.2

The Governor’s task force focused on the traditional procurement process as a major contributor to project failure. Its report documented the following significant findings concerning the traditional procurement process:

  • …the procurement process detracts from the vendor’s ability to propose creative or innovative approaches…and so limits the potential benefit…from innovative solutions (p. I8).
  • Successful partnerships based on teamwork and shared objectives are rare because the State’s current procurement process reinforces an adversarial relationship between the vendor and the State (p. I8).
  • …as many as 50 percent of large procurements end up in protest….The overall cost of the protest process, including delayed or lost project benefits, is substantial (p. V18).
  • By specifying the solution, the State bears 100 percent of the technological risks in IT projects (p. V17).
  • IT project vendors are rarely granted performance incentives, positive or negative (p. I7).

It was becoming very clear that procurement policies and practices that had served in the past were no longer suited to a dynamic technology market and a changing environment for government. The question was how to employ the best technology solutions to meet rising demands for service and governmental efficiency within the confines of severe funding shortfalls. The answer was a new procurement model for government—Performance Based Procurement.

The Solution: A New Procurement Model

The opportunity to develop and use a new procurement approach originated from a conference held by the California Department of General Services. Attended by government and private sector executives, the goal of the conference was to develop solutions to fix the burdensome procurement processes. One solution was a two-year pilot during which a new procurement model would be developed and tested. The main requirement was that the model process retain the qualities of competition. The FTB was more than willing to rethink government business processes and volunteered to help develop and test a new procurement model.

The timing for the pilot was perfect for the FTB. Having just completed 18 months of in-depth business and technology strategic planning, FTB management and staff had a shared vision of the future. The strategic planning helped bring a strong business focus to project planning and revealed a critical need to accelerate the deployment of appropriate technology within the FTB. However, the FTB did not have the in-house expertise to develop and implement the needed technology on its own. Also, the FTB did not have funding available to meet its technology needs through the normal budget process.

The alternative procurement model developed by the FTB is performance based procurement (PBP). It was approved by the Department of General Services and the Department of Finance as a pilot effort and was used by the FTB for a large, high-risk technology project—replacement of its bank and corporation tax systems. The PBP approach developed by the FTB is aimed at the adoption of “best of breed” technology and business practices through unique public-private partnerships that provide incentives for private partners to share in the risks of such joint ventures. PBP is built around four principles:

  1. Strategic Partnerships. PBP replaces the traditional adversarial vendor relationships with long-term, mutually beneficial business relationships based on trust, honest and open communication, and teamwork. The process establishes partnerships with qualified vendors who have knowledge and expertise in target business and technology arenas.
  2. Business-Driven Solution. PBP focuses on obtaining proven solutions to specific business problems and identifying new and innovative ways of achieving business goals. It is a top-down, iterative process of developing and evaluating ideas in an “egoless” environment with the goal of achieving business solutions that are “best of breed.”
  3. Best Value Evaluation. PBP results in solutions that provide best value to the state rather than only the lowest bid. This is achieved through an objective method that critically reviews and competitively scores solutions based on the following criteria:
    • quality of solution
    • risk of implementing the new technology and work processes
    • net benefit produced.
  4. Performance Based Payments. PBP provides payment to the business partner only if and when the benefits are realized after implementation of the solution. This principle solves the upfront project funding problems, increases vendor commitment to success through their assumption of upfront project costs, and limits the state’s financial liability for unworkable solutions.

The PBP process forgoes the methods typically used in IT procurement—beginning with exhaustive situation analysis and detailed specifications development. Instead, a government agency presents several competing business partners with its business problem and outlines broad objectives, performance criteria, and technology infrastructure requirements. The business partners are then invited to work with the agency toward a creative solution that achieves the best value. Detailed competitive proposals are then submitted for review, with evaluation based on quality of solution, risks, and benefits.

After selecting the winning proposal, the government agency negotiates a contract with the business partner that includes the proposed deliverables, schedules, project management structure, costs, and payments. The business partner funds technology infrastructure (hardware, software, and telecommunications) and system development up front. Payment is contingent on actual achievement of benefits, which are the source of funding for partner payments. Hence, if a business partner builds a system that does not perform as agreed, it does not get paid.

Case Study: Performance Based Procurement at the FTB

The three-member Franchise Tax Board consists of the state controller, the chair of the Board of Equalization, and the director of the Department of Finance. The board appoints an executive officer to carry out its policies and direct the day-to-day activities associated with administering California’s two major revenue programs—the personal income tax and the bank and corporation tax, which account for nearly $22 billion or 53 percent of California’s entire General Fund. In addition to processing nearly 15 million income tax returns, the FTB provides information and assistance to taxpayers and tax preparers, performs audit and collection activities, and administers a growing number of nontax programs (e.g., Political Reform Act audits, child support collections, and collection of delinquent motor vehicle registrations).

Several years ago the FTB identified a strategic objective to modernize its core operating systems. However, due to the severe fiscal crisis in California at that time the department could not secure funding for development of the project. Participation in a pilot to test a new procurement model gave FTB the opportunity to replace its bank and corporation tax systems. The Bank and Corporation (B&C) System Redesign Project is a large, highly complex technology project. The B&C Project replaces an antiquated processing and accounting system that had failed on numerous occasions; it also provides new functionality to the FTB in administering the bank and corporation tax. Since the state’s ability to generate revenue is directly related to the performance of the B&C automated system, the success of this project is critical.

Due to the vast scope of the approved B&C Project, the department proposed breaking the system into three separate projects. Each of the three components required different business and technology knowledge and experience. The FTB used the new procurement model in three distinct procurement phases with three different Project Steering Committees and Project Teams. This allowed the FTB the opportunity to develop a process, use, and improve it three times. This approach was particularly valuable in establishing baseline information for conducting the PBP process and improving the level of participation and interaction with the business partners for each solicitation process. Breaking the project into smaller, more manageable components also reduced the department’s risk in implementing a massive, highly complex technology solution. The three components are as follows:

Business Entities Tax System (BETS)

BETS includes modules for taxpayer identification, revenue accounting, automated correspondence, document and return processing, taxpayer accounting, and accounts receivable. BETS includes banks, all types of corporations, limited and general partnerships, real estate mortgage investment companies, and credit unions. BETS provides easier access to information, including multiple taxpayer identification numbers (e.g., state and federal employer identification numbers) and real-time information online.

Collection Accounts Processing System (CAPS)

CAPS is the FTB’s solution for replacing the old B&C collection system. CAPS has full collection functionality for business entity accounts. It is structured to work for the collection of other kinds of accounts, including delinquent child support, vehicle registration, court-ordered debt, and student loans. CAPS provides inventory tracking and case management, automated functions on liens, withholding orders, and notices.

Pass-through Entity Automated Screening & Support (PASS) System

PASS system objectives are to provide a comprehensive audit structure to support the FTB’s various audit activities. PASS will accommodate all pass-through entities, including general and limited partnerships, S corporations, regulated investment companies, real estate investment trusts, real estate mortgage investment conduits, and limited liability companies, as well as C corporations and banks.

To establish the partnership pool, the FTB published the following notice in the California State Contracts Register:

The Franchise Tax Board (FTB) is developing a new approach to procuring services. Contingent upon approval of the California Department of General Services and the Department of Finance, FTB will use the new “alternative procurement process” to replace its existing Bank and Corporation (B&C) Tax System with a modern Business Entities Tax System (BETS).

FTB will be establishing partnerships with firms who have demonstrated success in large tax system designs, Data Base Management System environment development, and who have experience in implementing new products and technologies. Establishing cooperative partnerships will give the FTB the opportunity to incorporate the expertise of partners in the BETS design and streamline the procurement process.

FTB expects partners to share in the risks of successfully developing and implementing BETS. FTB anticipates that the joint venture benefits will serve as the source for compensating its business partners. Payment will begin once the new system is operational and will be funded from benefits generated from the new systems. FTB anticipates that the initial system will be operational January 1995. With benefits for the BETS anticipated to be in the multimillion dollar range, the rewards will make the risks worthwhile.

In addition, the FTB did direct mailing to the vendors found in the initial survey and to over 400 vendors listed with the Department of General Services. The FTB repeated this process three times during the first year of the pilot. Twenty-one vendors responded to the solicitation. Eleven provided the requested information, and six were selected as qualified. Two additional partners were added to the partnership pool during subsequent screenings.

Preparing the FTB organization for the development and then the use of a new procurement model involved several steps. The most important one was to establish the teams that would actually do the work. The FTB had just completed the department-wide strategic systems planning effort, and the staff involved in that effort had experience in visioning, were able to shift paradigms, and work cooperatively with vendors. These and other staff members were pulled in to provide a mix of team members that took into account complex resource needs including the best available staff from program, technical, and administrative areas.

The FTB established an Executive Oversight Committee, a Project Steering Committee, and project teams for the three B&C Project components. The project management teams, normally comprised of FTB program and technical project lead staff, had to change as a result of the PBP model. With the implementation of the new model, it was necessary to extend project responsibilities to include the qualified business partners. Since the business partners share the project risks and the benefits, they must have some say in managing the project.

The FTB also established the “Performance Based Procurement” Steering Committee to oversee development of the new procurement process. This committee included members of the Executive Oversight Committee and the project managers. In addition, the FTB added a new team, the Vendor Partnership Evaluation Team, to develop and then implement some of the steps in the new procurement model, such as advertising and establishing criteria for vendor evaluation and selection. The final organizational change was to assign a fulltime manager responsibility for the model development and for successful implementation.

The open communication and team environment was the most significant change in the PBP process. The traditional procurement process greatly restricts the amount and content of interaction with vendors in the bid process. Both the FTB and our business partners immediately recognized the potential benefits the opportunity for open dialog presented. As stated in a letter from one of our business partners, “Probably the most important attribute of the alternative procurement process is that it fosters unconstrained, open communication. This communication increases trust since the vendor and State have common goals targeted at solving the identified business problem.”3

For the FTB the business problem was fairly well established in the prior attempts to obtain project approval and funding. The FTB, however, found a fundamental difference between defining a problem to obtain funding and defining a problem so that outside partners can develop effective solutions! Thus, the FTB had to have a new approach for defining the business problem—one FTB program executives found tremendously valuable. It was a process of self-searching, and of questioning the old business assumptions and processes. This penetrating evaluation provided the opportunity for identifying and implementing business improvements not related to the project, and it provided a powerful validation of the appropriateness of existing policy and practices.

The California Department of Finance, Office of Information Technology (OIT), and Department of General Services/Office of Procurement (OP) were invited to participate in the new process. OIT declined, while OP assigned an analyst to participate in the early development phases. In addition, several briefings were held with control agencies—Department of Finance budget staff, OIT, and OP—to provide information on the process. The business problem definition was used to develop the Conceptual Business Proposal, and to obtain approval to proceed from the FTB Executive Oversight Committee and the control agencies. Table 1 illustrates the procurement process as it unfolded, from submission of the Conceptual Business Proposal to the Contract Award.

During the conceptual proposal stage, FTB staff held confidential meetings with interested business partners to help them explore the agency problem in more depth. These were individual meetings that allowed the business partners to address concerns and ask questions about the Solicitation for Conceptual Proposal (SCP).

FTB project teams felt that this specific activity—working in face-to-face interactions—made the working relationship significantly positive and gave the best sense of a business partner’s ability to meet the needs of the FTB. This approach provided the fastest response to raising and solving issues, ensured that the business partners understood the problems and that FTB staff understood the solution, and gave each participant a better feel for the commitment of the other.

In addition, the participants felt that the communication established and used throughout this phase was a vital aspect to the development of effective solutions. The communication provided checkpoints to make sure that everyone was on the same track. It also helped establish excellent risk and quality assurance in each business partner’s proposal. It was during this process that business partners, fully understanding the FTB’s business problems, combined with their own knowledge and expertise in building complex tax administration systems, unveiled benefit streams that would have gone unrealized in the traditional procurement, i.e., “no communication allowed,” environment.

This was also the most important step for transfer of knowledge. FTB program staff shared their knowledge of their business programs and the existing problems to FTB technical staff and business partners. The FTB technical staff shared their knowledge of existing and planned technical standards and architecture to the business partners and FTB program staff. And the business partners shared their knowledge of new processes and technology, and potential solutions, to FTB program staff and technical staff.

Table 1. Timetable for First Use of Performance Based Procurement

Submit Conceptual Business Proposal
Approval to proceed July 26, 1993
Establish Partnerships
Letters of Invitation to participate July 28, 1993
Invitation published in State Contracts Register July 30, 1993
Finish selecting qualified firms September 15, 1993
Develop Conceptual Business Solutions
Start sending Solicitation for Conceptual Proposals to partners September 13, 1993
Receive Intent of Partners September 16, 1993
Partnership Conference held September 17, 1993
Individual conferences September 27 to October 8, 1993
Draft Proposals due October 12, 1993
Draft Proposal discussions October 8 to November 5, 1993
Select Final Proposal
Final Proposal due November 8, 1993
Evaluation November 8 to December 15, 1993
Intent to award December 16, 1993
Negotiate Contract
Negotiation December 17 to April 26, 1994
Award Contract
Award April 29, 1994

 
In evaluating the vendors’ final proposals, the FTB looked for the solution that would provide the “best value” rather than the lowest cost. Not only did this mean factoring anticipated benefits into the equation, but also the overall quality of the solution being proposed and the level of risk involved in implementing it.

Once a vendor was selected, FTB’s challenge was to negotiate a new type of contract—one that established the types of measurable benefits anticipated from implementation of the project and the percentage of benefits that would be provided to the vendor by way of payment (subject to a payment cap). As reflected in the table, it took the FTB approximately nine months from the time the Conceptual Business Proposal was approved to award a contract for development of the BETS portion of the B&C project—the first to be undertaken using the PBP approach. This included delays in obtaining control agency approvals resulting from the use of a new process.

Traditional Versus Performance Based Procurement

To understand the PBP model, it is helpful to see how it compares to the traditional procurement cycle. The new model completes the same work and has the same level of protection and approvals as the traditional model, yet it provides significant improvements in both quality and cycle time.

  • Instead of performing an alternatives analysis based on detailed business and technical specifications, the PBP model is business driven; i.e., the model obtains qualified business partners to help develop the “best of breed” solutions to solve the defined business problem.
  • Instead of performing analysis and documentation of requirements in isolation early in the process, the PBP model first develops a business case by defining a business problem and a high-level conceptual plan for developing solutions. The model provides for more analysis and documentation of requirements later in the process with help from outside experts.
  • Instead of limiting solutions to those that address the premature requirements, the new model gathers qualified vendors to act as partners in fully investigating the business problem, fully developing the requirements, and in developing the most responsive solution to the business problem.
  • Instead of seeking project and funding approval based on limited solutions, the model seeks project and funding approval after full investigation of the problem and requirements, and selection of the “best value” solution.
  • Instead of seeking to procure a solution based on low bid, the model seeks to optimize return while reducing risk to the state.

In terms of the amount of time required to implement a project, the traditional and PBP approaches can be similar in duration up to the point at which project development begins.4 Traditional procurement can take significantly longer during this period depending on the number and complexity of protests filed. The 1994 Task Force on Government Technology Policy and Procurement found “70 to 80% of IT projects over $1 million are protested.”5 Of these protests, only 5 percent are successful.6

Table 2. Comparison of Traditional and Performance Based Procurement Models

Traditional Procurement Model Performance Based Procurement Model
Focus on justifying a requirements study. Agency staff independently define detailed business and technology requirements. Focus on describing the business problem sufficiently to allow qualified business partners to develop effective solutions.
No parallel step. Create a pool of qualified business partners who have the experience and resources to help develop solutions to the defined business problem.
Agency staff develop solution alternatives based on the existing knowledge of technology opportunities. Agency staff work with qualified business partners to develop “best of breed” conceptual business solutions (solution alternatives).
Agency staff do a cost/benefit analysis (feasibility study) to identify lowest cost alternative. Agency staff competitively select the “best value” final proposal from the proposed conceptual business solutions.
No parallel step. Agency staff negotiate contract terms and conditions of solution delivery with the business partner who developed the “best value” proposal.
Agency staff write Feasibility Study Report (FSR) and Budget Change Proposal (BCP) for project approval. Agency staff write Project Plan (FSR) and Funding Plan (BCP) for project approval.
No parallel step. Agency staff obtain approval of the unique contract terms and conditions, and sign the contract.
Agency staff write detailed business and technology requirements for project proposal, complete bid process and award standard contract. No parallel step.
Agency staff work with successful vendor to develop and implement solution according to bid specifications. Project development is the same, except

  • business partner participates in project management
  • no payment until solution is in production
  • business partner has financial incentive to implement a working solution within budget
  • fewer contract amendments and delays should occur
  • “nice to have” changes may be limited to those that do not affect the project schedule and cost.

 
The PBP model can be expected to shorten the project development phase for two reasons. First, some of the project definition that traditionally occurs at the beginning of the project development phase will have already occurred during the development and evaluation of the business partner’s proposal. Second, the dependence of payment to the vendor on the actual delivery of system benefits provides an incentive to the vendor to get the system up and running as quickly as possible.

Costs and Benefits

The B&C Project is now well into the fourth year of its project lifecycle and has proven extremely successful. CAPS was fully implemented in May 1996. BETS, Phase 1, was implemented in February 1996, and Phase 2 was implemented January 1997. PASS, Phase 1, was implemented in October 1995, Phase 2 in July 1996, Phase 3 in December 1996, and Phase 4 in July 1997.

The project has already generated significant additional revenues.7 CAPS had generated $59.8 million by the end of 1996.8 As of July 1997, BETS had generated over $15 million in additional revenues,9 and PASS had generated $31.6 million.10 The project is expected to generate $259.5 million in additional revenue by June 30, 1999 ($47.1 million from BETS; $94.8 million from CAPS; $117.6 million from PASS).11 Taking into account total project costs of $133.7 million, the project will generate $125.8 million in net new revenues by the end of the 1998–99 fiscal year. The project will also provide other benefits in the form of personnel savings and existing system savings, which are not included in the figures above.

The performance based model focuses on defining and achieving positive business results. The process is designed to be dynamic and generate results. A critical part of the process is documenting, in specific contractual terms, the quantifiable business outcomes that are the basis for payments. This forces the identification and documentation of real measurable benefits. The benefits as defined in the contract must be achieved for payments to be made to the vendor partner.

Using the performance based model provides hard benefits in two basic areas:

1. Speed of the procurement and project delivery leverages business opportunity. The traditional procurement process has incentives for delays—delays that result in the average procurement taking 24 months to complete. Delays also mean business opportunities are lost. Extended delays due to protest or other disputes can make solutions obsolete due to evolving business needs. The PBP model has built-in incentives for bringing solutions into production as soon as possible and decreases overall project development time by accomplishing more of the project definition prior to the project development phase.

Using the performance based model allowed FTB to bring technology solutions to bear faster than ever before. We completed each of our three procurements within nine months. This allowed us, in the case of our collection system (CAPS), to procure and install the system and generate $13 million in new revenues in the same time we would normally take to complete the traditional procurement.

2. Better solutions yield increased benefits. The PBP model encourages vendors to provide the best solution possible to address a business problem. Because the process to select the winning solution is based on best net value—not low cost—vendors have an incentive to maximize the benefits generated.

Accurate evaluation of the benefits generated from the PBP approach was critical for FTB, the vendor partner, and our overseers in the administration and legislature. Comparison of expected versus actual results yield an impressive picture of the potential for performance based procurements in terms of improving business outcome.

In 1992, prior to undertaking the performance based approach, FTB conducted a feasibility study to evaluate the potential revenue return of a new bank and corporation tax system. FTB business experts projected that $8.4 million in new revenues could be produced annually with a total project cost of $48 million. These estimates were based on an internally developed system design using our traditional development approach. Given the marginal returns, we expected to undertake the project as a modernization effort to replace an aging and inadequate system.

The actual realized and projected ongoing benefits of the systems implemented under Performance Based Procurement are significantly higher than our earlier in-house projections. The costs of the project developed using PBP were $134 million or roughly three times the original estimate. However, new revenues are stabilizing at just over $70 million annually12 or more than eight times original projections.

Under the PBP model, the investment picture changed from a cost model to a revenue opportunity model. This striking difference in project outcomes is, we believe, a result of the cooperative solution development, which brought together the business talents of FTB with the specialized technical talents of quality business partners.

Obstacles to Implementation

Change in government, even positive change, is often viewed with alarm and met with resistance. Implementing change as significant as performance based procurement may present significant obstacles that must be overcome. The following issues posed significant barriers to successful implementation of FTB’s new procurement approach:

Cultural inertia. The structure of government encourages staff to follow process and boilerplate approaches. Staff are not trained to think creatively or encouraged to take risk. Central control agency staff feel threatened when they do not have expertise with a new process. These factors resulted in subtle resistance to change and less than optimal support. Ultimately this was partially overcome by coordinating and gaining support from the highest levels of agency and department executives who understood the need for change.

Staff resistance. Internal staff charged with actually undertaking the procurement process were uncomfortable with the lack of structure. Procurements had previously been done using a structured “cookbook” approach with which they were familiar. The tendency was to revert unconsciously to the old way of doing things. This forced department executives to provide active leadership in developing and clearly defining the “new way.” This required documenting key new concepts and even creating new procurement terminology. Education was also important in helping staff to see the “big picture” and understand the importance of the new approach.

Legislative concerns. Even our overseers, who were most aware of the overall need for change, repeatedly expressed concern over the possible effects that subjective evaluations and increased decision flexibility would have on perceived competition—particularly for small business. Legislators were concerned over the loss of appropriation authority that would occur in future years after contracts were signed and in place. Holding informational meetings and working closely with key legislators who carried technology agendas proved critical to maintaining support within the legislature. Sponsorship within the Governor’s Cabinet was key to maintaining support within the administration.

Vendor skepticism. Potential private sector partners expressed concern regarding FTB’s ability to implement the new processes successfully and sustain them through the end of a multiyear contract. They wondered whether government would follow through on a shared-risk contract. They questioned whether FTB could gather and maintain support from overseers. Through open communication, we actively demonstrated our commitment to seeing the process through to completion. Eventually, FTB and private sector vendors were able to work through these concerns and establish the needed partnerships.

Defining the process. Because this was a fundamentally new approach for government, there were no existing models to follow. We were required to conceptualize, define, and implement the procurement process at the same time we were actively conducting the procurement. Although this presented a significant challenge, we were able to gather and maintain executive support throughout the department to provide direction and hands-on leadership.

Contract negotiation. Historically, the state has not negotiated contract terms and conditions, but has relied instead on “boilerplate” language developed over years of trial and error. This lack of negotiating experience required us to develop skills quickly and rely on in-house attorney staff with the personal flexibility to do something different. Mechanically, we had to develop a means of converting business outcomes to measurable contract terms as the basis for payments. High-level managers intimately familiar with the business issues defined valid outcome-related payment terms.

Organizational issues. Organizational culture is an often overlooked but critical factor for successful change. Many government agencies are not ready to implement significant changes in approach. Government organizations in general are ill-equipped to function in a true shared-risk partnership with private industry. The culture of private industry tends toward more risk-taking and speed in implementing solutions. The government partner must be able to think quickly and creatively, make timely decisions, and respond to new challenges. FTB had recently undergone significant organizational change that provided a real opportunity to meet this challenge. Executive management had consciously committed itself to strategic technology implementation and its associated risk. Business managers and experts were brought into the program to ensure that key decision makers were available and committed to the success of the final project.

Assessment for Replication

  • The PBP model is most appropriate for 1) large technology projects requiring substantial investment in, and management of, resources; 2) new technology projects, such as those requiring new computer technology beyond the agency’s in-house expertise; and 3) high-risk projects that involve enough cost, new technology, or potential impact on key business functions to require a risk-sharing approach with external service providers. The success requirements for this model are broad, ranging from staff attitudes to the ability to measure program costs in detail. Specifically, the agency should
  • have a definable business problem and be dissatisfied with the standard approach to solving problems, specifically with the need13 to develop technology solutions in-house
  • be open to creative solutions and be able to trust that effective solutions can be developed through the interaction of agency staff and outside experts
  • be open to working with outside vendors to explore business problems and willing to shift from independent to dependent working relationships
  • be educated in the new process and accept the new process as a valuable business alternative
  • have completed strategic business and technology planning so that the vision of the future has already been developed and accepted by agency management and staff
  • be able to develop and use performance measures that accurately determine benefits that result from implementation of business solutions.

While the FTB and California’s taxpayers are the initial beneficiaries of this innovative approach to technology procurement, PBP is appropriate for all government agencies in need of external assistance in solving complex business problems. The PBP approach has the potential to foster profound improvement in public service and government efficiency throughout the United States. According to the Gartner Group, a leading industry analyst,

Government executives are advised to leverage the success of others and experiment with the process to deliver more value to their constituents without arduous funding requirements. By the year 2000, 50 percent of all large-scale U.S. government systems integration projects that use external service providers will be performance based procurements.14

Because of the FTB’s success with PBP, interest in its use by other governmental organizations has been widespread. The FTB’s PBP model and variations on it are being used by California’s Department of Consumer Affairs, Department of Housing and Community Development, and the Board of Equalization; the Hawaii Department of Taxation; the Virginia Department of Taxation; and the Utah State Tax Commission.

About the Authors

Gerald H. Goldberg is executive officer of the California Franchise Tax Board and a CPA. Prior to beginning his career in accounting, he served as a VISTA volunteer and then as director of an antipoverty manpower program in Kansas City, Missouri.

Ralph Shoemaker is assistant executive officer for the technology and resources branch of the California Franchise Tax Board and serves as advisor to Governor Wilson’s Task Force on Government Technology Policy and Procurement.

Appendix A

A Step-by-step Guide to Performance Based Procurement

The PBP model, just like the traditional procurement model, has several steps. Unlike those in the traditional model, the steps in the PBP process can be overlapped as needed to support agency procedures and culture.

Getting Ready

  1. Obtain Approval—At this time there are no standard procedures for obtaining approval for the use of the new model. It is important, however, to recognize that this model is a significant change from the old process and as such it is critical that executive management and control agencies understand and support the use of the model.
  2. Prepare the Organization—Successful use of the model requires that the organization have experience in strategic business and technology planning; be able to track and measure the work product at detail levels; be willing to undergo significant cultural changes to work in partnership with vendors; and to accept and implement potentially significant changes.
  3. Defining the Business Problem—This standard business step has a new twist: the problem must be defined sufficiently so that qualified business partners can understand enough about it and the existing environment to develop effective solutions. Detail specifications for a solution are not developed independently by the agency.
  4. Establishing Teams—The use of teams is critical to the PBP model because of the interaction expected from program and technical staff, and vendor staff. Careful consideration of team membership is critical to the success of the process. The assignment calls for the best mix of staff representing a cross section of the agency. The critical nature of the target business program will require people who have the best business knowledge. Because the teams are expected to work as equal partners with vendors, the best team players available should be assigned.

Establishing Partnerships

In this step, interested business vendors work with the agency to establish their credentials. The agency competitively reviews the vendor credentials and selects a limited pool of qualified vendors. These vendors become qualified business partners for the agency. The intent is to have a public selection process that provides a pool of vendor partners who have the resources, expertise, creativity, and financial ability to provide “best of breed” solutions for consideration and implementation. The selected partners sign a formal agreement, wherein they agree to adhere to the principles of the partnership.

It is important to recognize that the establishment of strategic partnerships is just the initial step in the development of a long-term cooperative relationship. The actual “partner relationship” is developed throughout the process and does not exist merely as result of a signed agreement. It is the process, not the form, that makes working relationships of value.

The establishment of partnerships is an ongoing process. New vendor partners may join at convenient junctures, such as the introduction of a new business problem. One value of the PBP model is that it allows potential partners to make early decisions whether or not to participate in the partnership, or on any project initiative.

The traditional procurement process allows and encourages vendors to bid on a multitude of projects with minimal investment and understanding of the business problem or agency needs. This results in a larger number of bids to a state procurement, and a lower overall quality of the solutions.

The PBP model requires a larger upfront investment and commitment, which helps to improve the quality of bid solutions provided to the state. The degree of upfront commitment requires that the partners 1) have the resources, knowledge, and experience to participate; 2) invest resources in in-depth research about the business problem, and 3) invest resources in the development and implementation of the solution.

Soliciting Conceptual Proposals

The Solicitation for Conceptual Proposal (SCP) is a formal, written document provided to the qualified business partners asking for proposals to design, develop, test, and implement a solution to the existing business problem as described in the Conceptual Business Proposal.

Interested business partners develop a draft proposal and submit it to the agency for consideration. This is an iterative, conversational mode of proposal development. Agency staff and each business partner work together in confidence to investigate further the existing business problem, and to assess and discuss the viability and effectiveness of the business partner’s proposed methods of solving the problem. At this time, the business partners will have a better understanding of the problems faced by the agency and can use this information to withdraw from further participation or to reaffirm the feasibility of their continuation on the project.

The submittal of a draft proposal by the business partners may be repeated if more than one draft proposal is needed. For example, the first draft may only address the business proposal, such as workflow reengineering, while a subsequent draft submittal contains the technical recommendations for solving the business problem.

Agency staff evaluates the submitted proposals on several criteria, including

  • ability to meet and/or exceed the agency business and technical requirements
  • reasonableness
  • ability to achieve the benefits/revenues projected by the business partner
  • overall impact, including compatibility with architecture standards, ease of use, security and integrity requirements, etc.

During the evaluation period, open and honest discussions continue between the agency staff and the business partners. The agency meets in confidence with individual business partners to provide in-depth feedback on the evaluation of the draft proposal.

It is important that throughout these events care is taken to ensure the confidentiality of the discussions. Confidentiality is important because the agency staff will be privy to “work products” that are the property of the participating business partners, and the business partners may be privy to agency information that is subject to confidentiality laws.

“Best Value” Evaluation

A key component in this final proposal phase is the “best value” evaluation of proposals. “Best value” is based on a number of components, from the impact of the solution on the business and technical programs, to the benefits expected. The “best value” approach assumes the agency will adopt a solution in its entirety. This model does NOT recommend that agencies attempt to select the best components from each solution, as it is highly unlikely that separate components from separate solutions will effectively work together.

The agency staff evaluate each final proposal for “best value.” Care must be taken to ensure fairness in this evaluation. The solutions are different, which makes it difficult to develop a net assessment of benefits for comparison between the solutions. The “Best Value” evaluation for the PBP model must be based on predefined evaluation criteria:

Quality

  • Partnership Quality. Evaluate the business partner’s willingness to engage in open and honest communication with the agency in developing an understanding of the business and technical requirements and in working together with the agency to develop a proposal addressing those requirements. The criteria for the evaluation should also include prior experience with similar systems including actual vs. projected time and costs; satisfaction of previous customers; degree of diligence used in investigating and learning about the agency’s business and problems; and the ability to commit the resources necessary to complete the project.
  • Meeting Business Requirements. Evaluate the business partner’s proposal for meeting the business requirements, i.e. fully automated, partially automated, or manual. Include the extent to which the proposal would meet the customer’s needs, the clarity of the proposal, and the functionality proposed.
  • System Management Features. Evaluate the business partner’s proposal for the processes, facilities, and displays involved with system management, and reporting or evaluation of the work product. This would include the extent to which the proposal meets customers needs, the clarity of the proposal, and the functionality proposed.
  • User Friendliness. Evaluate the business partner’s approach to the overall system, including the “look and feel” of the recommended solution processes. This includes the degree to which the system is “intuitive”; the ease with which the user can change processes, rules, models, work schedules, and data presentation; and the help functionality provided.
  • Business and Technical Training Approach. Evaluate the business partner’s approach to training, including environment, materials, classes, instructional aids, etc., to allow for an effective and efficient installation and ongoing operation of the new system. In addition, the evaluation should take into account how the system manuals, unit procedures, and training proposed would support the end-user community, the plan for transfer of knowledge, and the use of training tools.

Risk

  • Project Management. Evaluate the business partner’s project management experience, communication plan, issue resolution plan, and change control plan. The evaluation should include how the partner addresses and uses the system’s development life cycle, skills offered, quality of the work plan, types of project management tools proposed, and project change control and quality control procedures.
  • Costs. Evaluate the business partner’s understanding of the costs for the project. Is the cost information in the proposal complete, clear, realistic, and consistent with the solution proposed?
  • Data Center Impact/Production Platform. Evaluate the business partner’s proposed impact on the Data Center during development and implementation. Does the proposal maintain security and the use of productivity platforms such as databases? Does the proposal include the conversion time, backup, and recovery procedures? Does the proposal address equipment and/or system software changes, support staffing needs, and allowance for growth? Can the proposed solution simultaneously run batch and online processes, etc.?
  • Development/Implementation. Evaluate the business partner’s approach to development and transition from development to production. Evaluate the partner’s ability to find the best fit between the use of new technology and existing systems, and the partner’s plans to modify, or reengineer work flows and processes to take advantage of new technology. How would the proposed system impact existing systems? Are the post-implementation support plan and interfaces with other business programs acceptable?
  • Ease of Maintenance. Evaluate the business partner’s proposal for ease with which system changes and enhancements can be developed and tested, including modularity, data independence, and documentation. Does the proposal include appropriate upgrade paths with appropriate costs? What are the maintenance plans and costs and how well can the solution transition into maintenance?

Benefits

  • Benefits. Evaluate the business partner’s ability to bring in the expected benefits within the expected time frames. Does the proposal identify appropriate types of benefits streams? Are there duplicate savings? Are the anticipated benefits realistic and achievable?
  • Infrastructure Improvements.15 Evaluate the business partner’s proposal for taking advantage of new technologies including the degree of value that is added beyond the business requirements. How well will the proposed technical platform support future technology within the agency? Is the foundation consistent with the long-term architectural plans? Is the proposal compliant with standards and is the proposed solution portable to other platforms? “Credits” may be provided for proposals that include valued infrastructure improvements such as equipment or software. Valued equipment is defined as equipment that would be procured in the near future, with or without the project. As such, the credit may offset future anticipated procurements.
  • Other. Evaluation might include a “Royalty” benefit stream. Such royalty would occur if the solution was to be developed jointly and then marketed to others. The solution cost/benefit may take into account the ability to share potential royalties.

For each of the above components a score should be given dependent on the relative ranking of the proposal. Each component may also be weighted in value. It is recommended that individual reviews, evaluations, and scoring should be done first. Then the individual evaluators should meet to establish an overall consensus. While this evaluation may be subjective, it is in practice excellent because it is based on objective criteria and requires each evaluation team member to support his or her rating to reach consensus.

It is strongly suggested that the rating methodology and tools be developed in advance and be tested for validity and reliability. One of the recommended tests is to develop a variety of “test solutions” and use the methodology and tools to rate the solutions. The “test solutions” should include poor solutions as well as excellent solutions. This test can also provide an opportunity for staff to gain training and experience in using the rating methodology and tools.

As with the previous steps, this evaluation phase is marked by continuous communication between the business partners and agency staff. The business partners are invited to provide the Project Steering Committee presentations of the proposed solutions. These presentations are vital to the selection process, as it is this step that allows the Project Steering Committee executives to understand what the solutions will bring to the business. The strongest presentations are those that are based on a prototype solution as it gives the clearest picture of the solution.

Negotiating and Awarding the Contract

The model calls for development of a contract that does not rely on the standard contract models. This contract is significantly different:

  1. Acceptance occurs only when the solution is successful.
  2. Payments are provided only if and when the solution delivers sufficient benefits to cover the cost. The contract sets the terms and conditions for availability of benefits as payments including interest, payment schedule, and the percentage of benefits to be applied to payments. The contract also sets the payment cap.16
  3. The contract is for delivery of a working solution. This should significantly reduce the need for contract amendments to cover cost overruns and other changes needed to make the solution work correctly.

Awarding the contract is essentially the same as the traditional procurement process with some new twists. For example, it is at this time that the desired solution and real costs are known, so approvals are obtained now for the project plan (Feasibility Study Report or Special Project Report) and funding plan (Budget Change Proposal). In the traditional process these approvals would have been obtained much earlier and would have been based on premature estimates that would require new approvals of revised project and funding plans.

Developing the Project

This step is the same as the traditional process, but it is affected significantly due to the changes in the procurement model. For example, there will be a greater feeling of urgency by the business partner to implement the solution because payment will not occur until after the solution is implemented and providing benefits. In addition, change control must be sufficiently flexible to ensure that needed changes are made, while it must be restrictive enough to ensure that the project cost and schedule are not negatively affected by changes that are only “nice to have.” In addition, agency staff must be able to work at a fast pace, understand the sense of urgency and be able to work side-by-side with others in a team environment.

Endnotes
  1. Task Force on Government Technology Policy and Procurement, Report to Governor Pete Wilson, State of California, September 1994, p. I4.
  2. Jerry Mechling and Victoria Sweeney, Overcoming Budget Barriers: Funding Information Technology Projects in the Public Sector, A. Alfred Taubman Center for State and Local Government, John F. Kennedy School of Government, Harvard University, Cambridge, MA, May 1997, p. 23.
  3. Paul A. Brands, Chief Executive Officer, American Management Systems, Inc., letter dated September 7, 1994.
  4. The traditional approach generally takes an estimated 11 to 19 months if no protests are lodged; the PBP approach can be expected to take between 12 and 16 months.
  5. Task Force on Government Technology Policy and Procurement, Report to Governor Pete Wilson, Appendix B21.
  6. Ibid, p. V15.
  7. “Additional revenue” or “new revenue” represents the amount of revenue that exceeds the amount that FTB estimates would have been generated in the absence of these new systems.
  8. Additional CAPS revenues are calculated by comparing the pre-CAPS baseline collection percentage to the collection percentage obtained under CAPS. The collection percentage is defined as the amount collected during the period divided by the amount available to collect during the period. The amount available to collect is the dollar inventory at the beginning of the period, plus additions to inventory during the period, less cancellations during the period. Inputs are adjusted to reflect any non-CAPS related factors, in order to maintain the integrity of the model.
  9. Additional BETS revenues are calculated by measuring qualified revenue based on actual payments received by BETS. FTB has established criteria for the measurement of each of five benefit streams for definition of the increased revenues and is developing measurement processes for 10 additional benefit streams. The measurement process involves both computer and manually compiled information.
  10. These additional revenues are the result of two areas of non compliance that FTB is now able to address as a result of PASS implementation: Pass through Entity (PTE) Filing Enforcement System notifications and PTE Audits. The figure is based on return payments and miscellaneous payments that resulted from the issuance of filing enforcement notifications and on PTE audit net final assessments.
  11. In addition to PTE Filing Enforcement system notifications and PTE Audits, these additional PASS revenues also include revenues associated with Bank and Corporation (B&C) audit effectiveness and B&C audit efficiency. Revenues associated with increased audit effectiveness are calculated by comparing PASS B&C net final assessments in the measurement period to B&C net final assessments in the base period and adjusting for any changes in actual audit production hours. Revenues associated with B&C audit efficiency are calculated by determining the reduction in the number of hours per audit under PASS when compared to the base period and applying the resulting hours saved to additional revenue producing audit workloads.
  12. These new revenues are attributed as follows: $8 million from BETS, $17.5 million from CAPS, and $45 million from PASS.
  13. The term “need” refers to the belief that the only viable option is to develop technology systems in-house.
  14. L. Cohen, Case Studies, CSCON042, “Reengineering Revenue Collection, California Style,” Gartner Group, External Services Providers Government, October 16, 1996, p. 3.
  15. Infrastructure improvements should only include those improvements needed for implementation of the solution.
  16. If the benefit streams exceed projections, the agency should have the option to make payments in accordance with the payment schedule or to make prepayment on the unpaid principal balance, with no prepayment penalties. Increased benefits do not increase payments to vendors, but do give the opportunity to accelerate payoff and thereby reduce interest costs.
0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published.