TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
Annual Assessment of the Business
Systems Modernization Program
June 2006
Reference Number: 2006-20-102
This
report has cleared the Treasury Inspector General for Tax Administration
disclosure review process and information determined to be restricted from
public release has been redacted from this document.
Phone Number |
202-927-7037
Email Address | Bonnie.Heald@tigta.treas.gov
Web Site |
http://www.tigta.gov
June 30, 2006
MEMORANDUM FOR CHIEF INFORMATION OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Annual Assessment of the Business Systems Modernization Program (Audit # 200620008)
This report presents the results of our Annual Assessment of the Business Systems Modernization Program. The overall objective of this review was to assess the progress of the Business Systems Modernization (BSM) program for Fiscal Year (FY) 2006, as required by the Internal Revenue Service (IRS) Restructuring and Reform Act of 1998.[1]
The BSM program is a complex effort to modernize the IRS’ technology and related business processes. According to the IRS, this effort will involve integrating thousands of hardware and software components. All of this must be done while replacing outdated technology and maintaining the current tax system. The BSM program is in its eighth year and has received approximately $2.1 billion for contractor services. Additionally, the IRS had spent $170 million through FY 2005 and plans to spend an additional $50 million in FY 2006 to manage the BSM program.
Synopsis
Extensive
changes signal the beginning of a different phase and approach for the entire
modernization effort.
The BSM program’s
efforts to modernize IRS technology over the last 8 years have achieved mixed
results. During this period, the IRS
made changes to the program in response to many difficult challenges. This past year, the IRS began taking dramatic
actions by restructuring and redesigning significant areas within the BSM program. Some examples include the IRS taking over the
role of systems integrator[2] from the PRIME contractor, developing a new Modernization
Vision and Strategy, and changing its approach from completely replacing
current business systems to using current business systems to accomplish
modernization. We believe these
extensive changes signal the beginning of a different phase and approach for
the entire modernization effort.
Along with accomplishments made at the BSM program level, the IRS and its contractors have completed modernized projects that provide significant benefits to taxpayers. For example, the IRS reports the Modernized e-File project has resulted in over $18 million in savings to businesses, through reductions in tax preparation fees, postage, and storage, and within the IRS through efficiencies gained. Another example is the Customer Account Data Engine project. This system has generated over $1 billion in tax refunds and provides refunds to taxpayers 50 percent faster than the old system.
While the successes
achieved are commendable, the IRS and its contractors have struggled to develop
mature management capabilities and implement defined and repeatable processes
necessary for effective and efficient systems development. As a result, the BSM program has experienced
project cost increases and schedule delays.
Since breaking down releases into smaller, more manageable pieces, the IRS and its contractors are doing better at meeting cost and schedule estimates.
In 2004, the IRS decided to revise existing project cost and schedule estimates and began creating estimates for smaller pieces of work, known as subreleases. Prior to revising the estimates, the Commissioner reported the IRS and its contractors had not met cost or schedule estimates for any of its projects. Since breaking down releases into smaller, more manageable pieces as suggested by the Clinger-Cohen Act,[3] the IRS and its contractors are doing better at meeting cost and schedule estimates. However, some projects still encounter cost increases and schedule delays of greater than 10 percent.[4]
The severely reduced modernization funding levels over the last several years have been a concern for the IRS and the IRS Oversight Board. With the possibility of continued reductions to the BSM program in the coming years, the IRS’ ability to provide the level of service taxpayers expect will be affected. If the BSM program continues to show improvement, we believe additional funding should be considered. However, we caution against any drastic increases in funding, as they may exceed the IRS’ ability to effectively and efficiently manage the BSM program.
Since FY 2002, our BSM annual assessments have cited four specific challenges the IRS needs to overcome to deliver a successful modernization effort: 1) implement planned improvements in key management processes and commit necessary resources to enable success, 2) manage the increasing complexity and risks of the BSM program, 3) maintain the continuity and strategic direction with experienced leadership, and 4) ensure contractor performance and accountability are effectively managed. The IRS is at a juncture where it can build upon the successes and lessons learned from the first 8 years of the BSM program; however, we continue to believe the eventual success of the modernization effort will depend on how well the IRS deals with these four specific challenges.
Systems modernization
should remain a material weakness for the IRS.
The Federal Managers’ Financial Integrity Act of 1982[5] requires each Federal Government agency to prepare for Congress and the President an annual report that identifies material weaknesses and the agency’s corrective action plans and schedules. Since 1995, the IRS has identified and reported systems modernization as a material weakness. We believe systems modernization should remain a material weakness for the IRS based on open modernization corrective actions and the significance of the BSM program to external stakeholders.
Response
The Chief
Information Officer responded he was pleased this report recognized the
significant actions the IRS has taken to meet the BSM program’s many
challenges. He also provided some of the
benefits BSM projects are accruing to both taxpayers and the IRS, including the
issuance of over $3.3 billion in refunds by the Customer Account Data
Engine project. Management’s complete response to the
draft report is included as Appendix XII.
Copies of this report are also being sent to
the IRS managers affected by the report observations. Please contact me at (202) 622-6510 if you
have questions or Margaret E. Begg, Assistant Inspector General for Audit
(Information Systems Programs), at (202) 622-8510.
Extensive
Changes Signal a New Phase of the Business Systems Modernization Program
Assessment of the
First 8 Years of the Modernization Program
Four Major Challenges
Will Continue to Exist in the Next Phase of the Modernization Program
The Modernization
Effort Should Remain a Material Weakness for the Internal Revenue Service
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix III – Report Distribution List
Appendix IV – Glossary of Terms
Appendix V
– Business Systems Modernization Funding Timeline
Appendix VI
– Highest Priority Initiatives
Appendix
VII – Enterprise Life Cycle Overview
Appendix
VIII – Business Systems Modernization Projects and Status
Appendix IX
– Modernization Project Cost and Schedule Estimates and Revisions
Appendix X – Open Recommendations Related to the Systems Modernization
Material Weaknesses
Appendix
XII – Management’s Response to the Draft Report
The Internal Revenue Service (IRS) Restructuring and Reform Act of 1998[6] requires the Treasury Inspector General for Tax Administration (TIGTA) to annually evaluate the adequacy and security of the IRS’ information technology. This report provides our assessment of the IRS Business Systems Modernization (BSM) program for Fiscal Year (FY) 2006.
The BSM program is a complex effort to modernize the IRS’ technology and related business processes. According to the IRS, this effort will involve integrating thousands of hardware and software components. All of this must be done while replacing outdated technology and maintaining the current tax system.
The BSM program is in its eighth year and has received approximately $2.1 billion for contractor services.
The BSM program is in its eighth year and has received approximately $2.1 billion for contractor services. Additionally, the IRS had spent $170 million through FY 2005 and plans to spend an additional $50 million in FY 2006 to manage the BSM program.[7]
This review was performed at the Modernization and Information Technology Services organization facilities in New Carrollton, Maryland. The information presented in this report is derived from current and previous TIGTA, Government Accountability Office (GAO), and IRS Oversight Board reports and discussions with appropriate BSM officials. We also intended to review the MITRE Corporation’s[8] annual report on the BSM program, which is designed to provide up-to-date recommendations for adjustments and corrections to BSM program activities.[9] The results of this study were presented to the IRS in November 2005; however, the formal report had not been finalized by the time our review concluded in April 2006.
Compilation of information for this report was conducted
during the period January through April 2006.
Previous audits and our limited analyses were conducted in accordance
with Government Accounting Standards. Detailed
information on our audit objective, scope, and methodology is presented in
Appendix I. Major contributors to the
report are listed in Appendix II.
The BSM program’s efforts to modernize the IRS technology over the last 8 years with funds exceeding $2 billion have achieved mixed results. During that period, the IRS made changes to the program in response to many difficult challenges. For example:
Extensive changes signal the beginning of a different phase and approach for the entire modernization effort.
The IRS changes over the last several years were performed within the existing structure and conceptual design of modernization. This past year, the IRS began taking dramatic actions by restructuring and redesigning significant areas within the BSM program. Some examples include the IRS taking over the role of systems integrator from the PRIME contractor, developing a new Modernization Vision and Strategy, and changing its approach from completely replacing current business systems to using current business systems to accomplish modernization. We believe these extensive changes signal the beginning of a different phase and approach for the entire modernization effort.
The IRS is taking over the primary role of systems integrator from
the PRIME contractor
The IRS initially partnered with the PRIME contractor to achieve its overall goal of modernizing business processes and systems to improve service quality, timeliness, reliability, and privacy. The IRS relied on the PRIME contractor to act as a systems integrator to find and manage the best expertise and technical resources to achieve the IRS’ organizational goals. This partnership was envisioned to deliver world-class practices which would enable the IRS to modernize and provide top-quality service to taxpayers.
Since its inception, the partnership between the IRS and the PRIME contractor has experienced difficulties, and the envisioned world-class practices have not been fully realized. In 2003, due to the continued challenges confronting the BSM program, the IRS started to lose confidence in the PRIME contractor’s ability to meet its commitments in modernizing the IRS’ business systems. In May 2004, the IRS stated it was evident the PRIME contractor needed to significantly improve its performance. In 2005, the IRS began transitioning numerous program management activities away from the PRIME contractor and taking over the primary role as the systems integrator for all projects. The systems integrator role is a significant operational change for the IRS, requiring new procedures, personnel, and offices.
As directed by the House and Senate Appropriations Committees, the
IRS is developing a new version of the Modernization Vision and Strategy
In May 2005, the GAO recommended the IRS Commissioner fully revisit the BSM Vision and Strategy and develop a new set of long-term goals, strategies, and plans that are consistent with the IRS’ budgetary outlook and management capabilities.[10] The IRS needed a new Vision and Strategy because the last Modernization Vision and Strategy dated back to FY 2002 and no longer reflected reality.
In January 2006, the House and Senate Appropriations Committees approved the release of FY 2006 funds for the BSM program and directed the IRS to provide a draft of its current Modernization Vision and Strategy to the GAO. The final version of the Modernization Vision and Strategy was due as we were completing our report and was not considered as part of our assessment.
The IRS is changing its approach from
completely replacing current business systems to using current business systems
to accomplish modernization
In the past, the IRS’ approach to modernizing was an enormous development effort aimed at replacing its current business systems. The IRS has recognized the need for an integrated approach to enterprise modernization and is now focusing on a flexible, more realistic approach that seeks to use current business systems as well as current and future information technology investments to accomplish modernization. Instead of scrapping and replacing the capabilities of hundreds of current IRS systems, the IRS will have existing systems evolve into reusable services. While use of this approach differs from the modernization program of the past, we are unclear how the IRS will identify and leverage the strengths of existing systems to achieve modernization goals. The new Modernization Vision and Strategy being developed by the IRS may provide additional clarification in this area.
Assessment
of the First 8 Years of the Modernization Program
As mentioned previously, the BSM program’s efforts to modernize IRS technology over the last 8 years have achieved mixed results. While the BSM program has experienced setbacks, the IRS and its contractors have taken program-level actions that resulted in project-level successes. Figure 1 provides examples of significant program-level initiatives undertaken in the first 8 years of the BSM program; it is not intended to show every program-level accomplishment.
Figure
1: BSM Program-Level Actions
|
YEAR |
BSM INITIATIVES |
|
1999 |
The IRS received funding
based on the approval of the first BSM Expenditure Plan. |
|
2000 |
The modernized IRS
organization was officially inaugurated or “stood up.” |
|
2001 |
A comprehensive Enterprise
Architecture providing a strategic view of BSM initiatives was approved, and
a defined Enterprise Life Cycle[11]
methodology was completed. |
|
2002 |
The IRS took steps to
slow the pace of the BSM program to match management capacity and capability. |
|
2003 |
Four studies, including
a benchmarking analysis, were conducted to assess the health of the BSM
program. |
|
2004 |
The IRS implemented
numerous program management improvements such as establishing the
Requirements Management Office, implementing a Resource Capacity Model, and
creating a Performance Engineering Office. |
|
2005 |
The IRS took over the
primary role of systems integrator from the PRIME contractor. |
Source: TIGTA and GAO reports.
The IRS continues to deliver modernized
systems benefiting taxpayers and the Federal Government
Along with accomplishments made at the BSM program-level, the IRS and its contractors have completed modernized projects that provide significant benefits to taxpayers and the Federal Government. Some of these projects include the Customer Communications, Internet Refund Fact of Filing (IRFoF), e-Services, Customer Account Data Engine (CADE), and Modernized e-File (MeF) projects.[12] Figure 2 provides a snapshot of some of the project-level successes launched by the IRS and its contractors.
Figure
2: BSM Launched Projects
Figure 2 was removed due to its size.
To see Figure 2, please go to the Adobe PDF version of the report on the
TIGTA Public Web Page.
The IRS reports the following examples of BSM projects that have provided significant tangible benefits to taxpayers and the Federal Government.
Since our last annual assessment,[16] the IRS and its contractors have continued to complete modernized projects and deliver benefits to taxpayers and the Federal Government. Some of the most recent project-level accomplishments include implementing additional releases of the MeF and CADE projects, as well as a partial first release of the F&PC project. In addition, the IRS and its contractors recently received the Innovative Information Technology Award from the Government Computer News for the e-Services and MeF projects.
The IRS has struggled to manage program-level and project-level
activities
While the successes achieved are commendable, the IRS and its contractors have struggled to develop mature management capabilities and implement defined and repeatable processes necessary for effective and efficient systems acquisition and development. Figure 3 shows some of the challenges experienced within the BSM program and some of the actions the IRS has taken over the last 8 years in response to these challenges.
Figure
3: Roadmap of BSM Management Efforts
Figure 3 was removed due to its size.
To see Figure 3, please go to the Adobe PDF version of the report on the
TIGTA Public Web Page.
Cost increases and schedule delays have been significant but
performance is improving
Since the beginning of the modernization effort, BSM projects have experienced cost increases and schedule delays. We reported in the FY 2003 BSM annual assessment that certain modernized projects were experiencing cost increases of over $35 million and delays of up to 14 months from original estimates.[17] One year later, cost increases for projects had grown to over $86 million and schedule variances had grown to as much as 30 months.[18] Due to cost increases and schedule delays, pressure began mounting from various oversight groups to deliver a successful program. The IRS Oversight Board stated the “…IRS and its PRIME contractor cannot continue to operate in a business-as-usual manner.”[19] Figure 4 shows a history of some of the cost and schedule overruns within the BSM program over the last 8 years.
Figure
4: Cost Increases and Schedule Delays by
Calendar Year
Figure 4 was removed due to its size.
To see Figure 4, please go to the Adobe PDF version of the report on the
TIGTA Public Web Page.
Since breaking down releases into smaller, more manageable pieces, the IRS and its contractors are doing better at meeting cost and schedule estimates.
In 2004, the IRS decided to revise existing project cost and schedule estimates and began creating estimates for smaller pieces of work, known as subreleases. Prior to revising the estimates, the Commissioner reported the IRS and its contractors had not met cost or schedule estimates for any of its projects. Since breaking down releases into smaller, more manageable pieces as suggested by the Clinger-Cohen Act,[20] the IRS and its contractors are doing better at meeting cost and schedule estimates. In 2005, IRS executives also stated they were concerned the existing cost and schedule variance methodology did not accurately depict variances within the BSM program and held meetings with the GAO and the TIGTA to discuss better ways of presenting and characterizing cost and schedule estimates. Based on these meetings, the IRS plans to use a different methodology to calculate and present cost and schedule estimates in future Expenditure Plans.
Using the existing methodology, some projects still encounter cost increases and schedule delays of greater than 10 percent.[21] For example, the Integrated Financial System[22] project recently encountered a $9 million (60 percent) cost increase, and the MeF[23] project encountered a $7.5 million (32 percent) cost increase. The GAO review of the IRS’ FY 2006[24] Expenditure Plan reported results consistent with our analysis showing cost and schedule variances still exist for modernization projects.
Due to the complexity of BSM projects, it is unrealistic to think every project will meet its exact cost and schedule estimates. However, several project segments are meeting cost and schedule estimates or are within a 10 percent threshold. The next challenge for the IRS and its contractors will be to control significant cost and schedule variances and build upon its initial successes with smaller pieces of work to deliver the BSM program within expectations.
A return on investment for the BSM program has not been determined
A reasonable measure of value to determine the cost effectiveness of the BSM program would be to measure the return on investment taxpayers and the Federal Government have received from the BSM effort over the last 8 years. However, a logical assessment of the return on investment for the BSM program has not been determined because of the following factors:
Four Major Challenges Will
Continue to Exist in the Next Phase of the Modernization Program
Since FY 2002, our annual assessments have cited four specific challenges the IRS needs to overcome to deliver a successful modernization effort. The IRS has continuously taken corrective actions in response to our recommendations to address these four challenges. However, the IRS’ recent and planned future changes have not eliminated the four challenges. Our reviews over the last year found each of the four challenges still exists, and one of these challenges has expanded.
Challenge 1: Implement
planned improvements in key management processes and commit necessary resources
to enable success
Since our last annual assessment,[28] we found the IRS continues to need improvement in the areas of requirements management, cost and schedule estimation, and project justification. We have issued recommendations in each of these areas in prior audits. Recent reviews show the IRS is making progress but continues to struggle with defining and institutionalizing these key management processes.
Requirements management
Recent reviews of two modernized projects[29] showed requirements testing processes are not being followed and system requirements are being deferred to future project releases. During the completion of the MeF Release 3.2 project, testing activities used by the project team did not provide assurances the system requirements expected to be deployed were the requirements that were actually deployed. The project team did not trace the system requirements or update the System Requirements Report with changes during the testing process. In addition, a planned performance requirement to display tax and information returns within defined time periods was not delivered as part of the MeF Release 3.2. This performance requirement was originally planned to be delivered as part of the MeF Release 1 in February 2004.
For CADE Release 1.3.2, the project team did not timely finalize requirements and did not have appropriate staffing to fix a high number of failed tests during the systems testing process. This contributed to the deferral of eight requirements to future releases, and an unknown number of taxpayers will not receive the benefits associated with the CADE processing. We have reported on these and similar testing process issues since 2003.[30]
Cost and schedule estimation
Due to significant cost increases and schedule delays, the BSM program has been criticized for its ineffective cost and schedule estimation capabilities. A recent audit[31] shows the IRS and its contractors have not always followed cost and schedule estimation processes within the BSM program. In addition, we determined the IRS is instituting different cost and schedule estimation processes due to its assumption of the systems integrator role.
Project justification
Office of Management and Budget Circular A-11 Exhibit 300, Capital Asset Plan and Business Case,[32] for major information technology investments requires Federal Government agencies to conduct an Alternatives Analysis to provide estimated cost and benefit information on viable alternatives. This, in turn, assists management in determining the most effective approach for a project. During a recent review,[33] the IRS provided a draft Exhibit 300 showing alternatives with the most viable solutions for the F&PC project. We could not verify the IRS’ decision to purchase commercially viable software for the F&PC project because project documentation contained errors and information could not be supported. The new Modernization Vision and Strategy is now considering the alternative of using current business systems instead of using the commercial software purchased for the F&PC project.
We reported similar findings during a prior review of four other business cases in April 2005.[34] All four business cases contained deficiencies, did not comply with Office of Management and Budget requirements and, in some cases, did not comply with the IRS’ own Exhibit 300 Business Case Guide. Due to the importance of this issue, we are currently conducting a followup audit to determine the status of the IRS’ corrective actions.
Challenge 2: Manage the increasing complexity and risks of
the BSM program
As stated earlier, the IRS is taking over the role of systems integrator from the PRIME contractor, which will increase complexities and introduce new challenges for the BSM program. The IRS created and is currently working on several Highest Priority Initiatives to develop, implement, and execute a variety of activities (e.g., expanded cost and schedule estimation, quality assurance, and requirements management roles).[35] Many of the initial Highest Priority Initiatives concerning the transition have already been completed. Proper management and operation of these transitioning areas are critical to the success of the BSM program.
In addition to complexities at the program level, the IRS and its contractors face further complexity at the project level. For example, the CADE project is the IRS’ highest priority technology project and represents the core foundation of modernized systems. It has experienced significant cost and schedule setbacks during the initial releases involving the simplest accounts, and future releases will be more complex. The future of the CADE project is uncertain due to several factors.