TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
Correspondence Imaging System Performance Has Improved, but Additional Measures Are Needed to Ensure That the System Performs As Expected
July 9, 2008
Reference Number: 2008-20-130
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
Phone Number |
202-622-6500
Email Address | inquiries@tigta.treas.gov
Web Site |
http://www.tigta.gov
July 9, 2008
MEMORANDUM FOR CHIEF INFORMATION OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Correspondence Imaging System Performance Has Improved, but Additional Measures Are Needed to Ensure That the System Performs As Expected (Audit # 200720011)
This report presents the results of our review to determine
whether the Internal Revenue Service (IRS) had established and implemented
adequate internal controls to ensure the stability and performance of the
Correspondence Imaging System (hereafter referred to as the CIS or System) and to
ensure proper planning for its future merger with the Accounts Management Services
system.[1] This review was part of the Treasury
Inspector General for Tax Administration Fiscal Year 2007 Annual Audit Plan
coverage.
Impact on the Taxpayer
The IRS has spent approximately $32
million to develop and implement the CIS to scan and convert the millions of
pieces of incoming taxpayer correspondence to digital images. While the IRS has taken steps to improve the System’s
performance, the System continues to experience periods of instability and
performance issues. The IRS needs to
further improve System performance and prevent the same instability and
performance problems from recurring.
Until this work is complete, the CIS could continue to experience
periods of instability and performance issues that might affect the IRS’
ability to provide efficient and effective service to taxpayers.
Synopsis
The IRS is responsible for processing individual and business taxpayer
correspondence and forms at 10 campus[2] locations.
As of January 2008, the IRS had spent approximately $32 million to
develop and implement the CIS to scan and convert incoming taxpayer
correspondence to digital images. With
this System, IRS customer service representatives can access correspondence
electronically and use new automated procedures that should benefit the IRS and
taxpayers.
In January 2007, the CIS became so unstable that the IRS decided to shut it down for approximately 1 month and revert to use of manual procedures for processing taxpayer correspondence. The IRS was able to identify and resolve the specific issues that caused the January 2007 problems. However, the System continues to experience random periods of instability and performance issues. The IRS and its contractor did not follow appropriate requirements management processes for developing, documenting, and testing System performance requirements.[3] If the IRS does not follow critical requirements management processes, the CIS could continue to experience instability and performance issues.
In 2007, the IRS submitted a CIS business case providing key
information for Budget Year 2009 to the Department of the Treasury and the Office
of Management and Budget. However, this business case did not
disclose the IRS’ plans to merge the CIS with the Accounts Management Services
system, and the IRS overstated the estimated costs and benefits of the CIS by
$115.4 million and $86.2 million, respectively.
As a result, information used by the Department of the Treasury and the
Office of Management and Budget to make financial budget decisions related to
the CIS project was incomplete and inaccurate. Subsequent to the completion of audit
fieldwork, the IRS requested approval to merge the systems as instructed by
Department of the Treasury guidance.
Recommendations
The Chief Information Officer should ensure that 1) corrective actions taken subsequent to our audit work to address the open recommendations identified by the Incident Analysis Team[4] have been completed and were effective in improving CIS performance, 2) adequate actions are taken to reduce and prevent the same instability and performance issues from recurring after the CIS is merged with the Accounts Management Services system, and 3) IRS project officials make a timely request for Department of the Treasury approval to merge and close investments and restate expected costs and benefits.
Response
IRS management agreed with all of our recommendations. Corrective actions taken or planned include 1) providing a document showing the successful completion of each Incident Analysis Team recommendation, action, and result to the Customer Service Executive Steering Committee[5] and the Treasury Inspector General for Tax Administration, 2) implementing the Accounts Management Services system with a complete re-architecture of the CIS, 3) giving presentations to the Executive Steering Committees as part of the enterprise governance process to reinforce the need to secure Department of the Treasury approval when a decision to merge investments has been made, and 4) preparing the new CIS business case (Office of Management and Budget Exhibit 300) required to effectively merge the CIS and Accounts Management Services system investments and restating the expected costs and benefits. Management’s complete response to the draft report is included as Appendix VI.
Copies of this report are also being sent to the IRS managers affected by the report recommendations. Please contact me at 202-622-6510 if you have questions or Preston B. Benoit, Acting Assistant Inspector General for Audit (Information Systems Programs), at 202-622-5894.
System Performance Requirements
Were Not Adequately Managed and Tested
Information in the Correspondence
Imaging System Business Case Was Incomplete and Inaccurate
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix
V – Incident Analysis Team Report Recommendations and Corrective Actions
Appendix
VI – Management’s Response to the Draft Report
Abbreviations
|
CIS |
Correspondence Imaging System |
|
IRS |
Internal Revenue Service |
The Internal Revenue Service (IRS) processes
individual and business taxpayer correspondence and forms at 10 campus[6] locations.
IRS customer service representatives and tax examiners handle millions
of cases containing correspondence generated in multiple formats by
taxpayers. Correspondence might include
items such as taxpayer-generated letters, responses to tax notices, amended tax
returns, and claims documents, which are processed by customer service representatives
from the Accounts Management function within the Wage and Investment Division.
The IRS has
spent about $32 million to develop and implement the CIS.
As of January 2008, the IRS had spent approximately
$32 million to develop and implement the Correspondence Imaging System (hereafter
referred to as the CIS or System) to scan and convert incoming taxpayer
correspondence to digital images. Prior
to implementation of the CIS, IRS personnel performed manually intensive
procedures to process correspondence items.
The prior manual system made it difficult for the IRS to meet its goal
of resolving taxpayer inquiries, claims, and adjustment requests in a timely
manner.
The purposes of the CIS are to 1) provide an
automated process for electronically imaging all correspondence items with
improved inventory controls, and 2) reduce the time needed to resolve taxpayer
requests. With this System, customer service
representatives can access correspondence electronically while simultaneously
working with other existing management systems to more effectively process
taxpayer information. The IRS expects to
process about 7 million CIS cases each year, with as many as 8,200 users on the
System at peak processing times. Some
manual procedures are still needed to process correspondence items. However, the conversion of paper documents
into electronic media and the new automated procedures provide significant
benefits to both the IRS and taxpayers.
The CIS was
developed and implemented in two different releases.[7]
Release 1 related to Individual Master File[8] accounts and was deployed between Calendar
Years 2003 and 2005 at the following five IRS campuses:
Release 2 provided
additional system functionality and was deployed at three additional Individual
Master File campus locations. This
Release also included the ability to process Business Master File[9] accounts in two IRS campuses. The IRS has deployed or is scheduled to
deploy Release 2 at the following five campuses:
This review was
performed at the Atlanta Campus and at the offices of the CIS administrators in
The Internal
Revenue Service Has Taken Steps to Improve the Performance of the
Correspondence Imaging System
The CIS has significantly improved the
timeliness and efficiency of processing taxpayer correspondence cases in the
campus locations where it has been deployed.
The IRS relies on an effectively operating CIS to provide high employee
and, ultimately, customer satisfaction by reducing the burden of handling large
volumes of paper documents. IRS customer
service representatives and tax examiners nationwide are now using a number of
automated processes as a result of the CIS to more efficiently deliver work and
better manage inventories.
In January 2007, the CIS experienced
unacceptable levels of instability and performance problems that affected the IRS’
ability to conduct critical business functions.
The IRS took several actions to improve the performance of the System. For example, it identified the main reason
why the January 2007 System problems occurred and took steps to fix the
problems. The IRS also formed an
Incident Analysis Team[10] in February 2007 to assess additional
performance problems affecting the System.
Actions taken to identify and resolve stability and
performance problems
In January 2007, the
CIS was upgraded with a third-party software product that had unexpected
stability and performance effects on the System. Because of the stability problems, the IRS
decided to shut down the System for approximately 1 month and revert to use of manual
procedures for processing taxpayer correspondence. The performance problems also significantly
delayed final deployment of the CIS to the
In late February 2007, the IRS was able to identify the problems caused
by the software upgrade and focus its resolution efforts on correcting them. As part of the resolution process, the IRS
installed a monitoring tool to assess, track, and report the functions and
performance of the System. These actions
seemed to address the specific problems caused by the software upgrade. However, the CIS continued to experience
stability and performance issues.
In response, the IRS formed the Incident Analysis Team to investigate
the causes of the System’s instability and to recommend changes that would address
those problems. The Team analyzed all
CIS problems that occurred from January 10 through June 17, 2007, and
classified them into six major categories.
The results of the Incident Analysis Team analysis were completed in
June 2007, and the Team prepared a report containing seven recommendations to address the performance problems. The IRS developed 26 corrective actions in
response to the 7 recommendations.
As of
January 2008, the IRS had taken steps to complete 18 of the 26 corrective
actions. The remaining eight open
corrective actions correspond to five of the recommendations made in the
Incident Analysis Team report. Specific
details on the status of the recommendations and corrective actions are
included in Appendix V. According to the
IRS, it is continuing work to complete the corrective actions by their assigned
due dates.
The Correspondence Imaging System Continues to Experience
Random Periods of Instability and Performance Issues
The IRS assesses the stability of the CIS based on System
availability. The System is deemed
stable if downtime (any period when users cannot use the CIS to work and close
cases) does not exceed 4 hours during a 14-day period. The IRS measures downtime by how long the System
is unavailable and the number of locations affected. The results are cumulative. For example, if the CIS was not available for
2 hours and the downtime affected 8 IRS locations, the cumulative downtime
would be 16 hours.
The CIS
continues to experience periods of instability and performance issues.
Based on the IRS criteria for System stability, CIS
performance has significantly improved.
In fact, the IRS reported the System was operational more than 98
percent of the time during Fiscal Year 2007.
The IRS declared the CIS stable on August 21,
2007, because it had not had any downtime longer than 4 hours for a 14-day period. However, the System continues to experience
random periods of instability and performance issues. The IRS was unable to maintain a
consistent level of stability, and the System was unavailable for significant
periods of time on several occasions during the last 3 months of Calendar Year 2007.
For example,
the CIS was down:
·
On October 17 approximately
1.5 hours affecting all 8 locations resulting in 12 site hours of impact.
·
On October 31 approximately
0.5 hour affecting all 8 locations resulting in 4 site hours of impact.
·
On November 19 approximately
2.3 hours affecting all 8 locations resulting in 18 site hours of impact.
·
On December 10 approximately
7.5 hours affecting 3 locations resulting in 23 site hours of impact.
·
On December 11 approximately
2.3 hours affecting all 8 locations resulting in 18 site hours of impact.
·
On December 19 approximately
1.4 hours affecting all 8 locations resulting in 12 site hours of impact.
We believe that the continued instability and performance
problems persist, at least in part, because the IRS has not completed all of the corrective actions to address the
recommendations made by the Incident Analysis Team. As of January 2008, 8 of the 26
corrective actions were still open. For example, one of the open corrective actions
is to test the Disaster Recovery/Replication process. This corrective action was scheduled for
completion on December 15, 2007, but is still outstanding. IRS customer service representatives rely
heavily on the CIS to perform critical day-to-day operations. A complete and tested disaster recovery
process provides the ability to effectively respond to disasters that could affect
the System. If the CIS continues to experience random
periods of instability and performance issues, the
IRS could:
Management Action: Subsequent to the
completion of audit fieldwork, the IRS provided a report dated April 9, 2008,
showing that the remaining open corrective actions were closed. However, we did not obtain additional
information and validate the closure of the actions.
Recommendations
The Chief Information Officer should ensure that:
Recommendation
1: The corrective
actions taken subsequent to our audit work to address the open recommendations identified
by the Incident Analysis Team have been completed and were effective in improving
CIS performance.
Management’s
Response:
IRS management agreed with this recommendation. The Applications Development organization,
partnering with the Enterprise Operations organization, provided a document
showing the successful completion of each Incident Analysis Team recommendation,
action, and result to the Customer Service Executive Steering Committee[12]
and the Treasury Inspector General for Tax Administration. The stability, availability, and performance
improvements achieved through implementation of the Incident Analysis Team
recommendations resulted in unanimous approval by the Customer Service
Executive Steering Committee to close Incident Analysis Team status reporting
on May 28, 2008.
Recommendation
2: Adequate actions are taken to reduce and prevent the same
instability and performance issues from occurring after the CIS is merged with
the Accounts Management Services system.
Management’s Response: IRS management agreed with this recommendation. The IRS will implement Accounts Management Services system Release 1.3, which is a complete re-architecture of the CIS. This will ensure that adequate actions are taken to reduce and prevent instability and performance issues once the CIS has merged with the Accounts Management Services system.
System Performance Requirements Were Not Adequately Managed and Tested
The IRS and its contractor did not follow appropriate requirements management processes for developing, documenting, and testing performance requirements. For example, the IRS developed 17 performance requirements for CIS Release 1 but could not provide us with documentation to verify that those performance requirements were adequately tested and accepted by all stakeholders. For CIS Release 2, the IRS relied on the performance requirements developed from Release 1. According to the IRS contractor working on the System, the 17 performance requirements developed by the IRS were not sufficient and did not provide adequate information to be measurable. Therefore, the contractor developed and tested other performance requirements, which were not reviewed and approved by IRS officials. The IRS provided us with the performance tests conducted by the contractor. However, none of the test results were validated or measured against pre-defined and approved performance requirements.
The IRS Modernization and Information Technology Services Handbook states that the stakeholders of a system should define the performance requirements to identify and document the needs and capabilities of the system. Requirements should be unambiguous, traceable, complete, verifiable, and measurable. Stakeholders are also responsible for validating and approving the system requirements with test results that are documented and compared to expected results.
Requirements management processes are designed to ensure that a system will perform in accordance with user expectations. If the IRS does not follow critical requirements management processes, the CIS could continue to experience instability and performance issues. In addition, the IRS might be unable to achieve all of the business performance goals expected from deploying the System. Over the past 3 years, the IRS has been unable to achieve three of the four business performance goals set for the System. Figure 1 provides the CIS business performance goals and results for Fiscal Years 2005 - 2007.
Figure 1: CIS Performance Goals
|
Performance Goals |
Fiscal Year 2005 Result |
Fiscal Year 2006 Result |
Fiscal Year 2007 Result |
|
Reduce the number of days to close cases to |
44 calendar days |
50 calendar days |
46 calendar days |
|
Increase System availability to 99.97%. |
No Measure |
No Measure |
98.21% |
|
Reduce the over-age case percentage to 15%. |
13% |
23% |
28.4% |
|
Increase the number of cases controlled on the System to 85%. |
80.1% |
80.1% |
86.1% |
Source:
IRS-provided performance measures worksheet for Fiscal Years 2005 and
2006 and the CIS business case (Office of Management and Budget Exhibit 300) submission.
While other factors such as staff reductions might contribute
to the IRS’ inability to achieve the stated CIS performance goals, appropriate
requirements management processes are critical to the success of systems
project development. We have previously
reported on weaknesses in the IRS’ requirements management process[13] and have provided recommendations to improve the process. The IRS is currently working on several
corrective actions for those recommendations that, if taken appropriately, will
address the same requirements management issues discussed in this report. Therefore, we are including no additional
recommendation related to this issue.
Information in the Correspondence Imaging System Business Case Was Incomplete and Inaccurate
The IRS has provided information to the Department of the Treasury and the Office of Management and Budget to make financial budget decisions related to the CIS project. The primary tool used to report the information was the CIS business case (Office of Management and Budget Exhibit 300). However, some information in the System business case was incomplete and inaccurate and, as a result, the IRS overstated the estimated costs and benefits of the System by $115.4 million and $86.2 million, respectively.
A project business case
provides key information related to IRS investments on capital assets and
information technology projects. Key
information includes estimated costs, benefits, and financial indicators for
the investments. It is used by IRS,
Department of the Treasury, and Office of Management and Budget officials to
ensure that Federal Government resources are spent wisely and to improve upon
asset management.
All information contained in a project business case should be accurate,
succinct, and up-to-date.
Information included in a business case represents the baseline for an investment and includes cost, schedule, and performance goals. Any major changes to information technology investments (e.g., a significant change to the scope of an investment or the merging of two or more investments) must be approved by the IRS Chief Information Officer, the Department of the Treasury, and the Office of Management and Budget before being implemented. A Baseline Change Request should be prepared to document the proposed changes to the investment baseline and be approved prior to submission of a new business case.
In Calendar Year 2006, the IRS decided to merge the CIS with the new Accounts Management Services system. The merger is scheduled to be completed in November 2008 and will result in the retirement of the CIS. However, the IRS did not prepare and submit a Baseline Change Request for the planned merger before it submitted the Budget Year 2009 CIS business case in 2007. Because the planned merger was not approved by the Department of the Treasury and the Office of Management and Budget, the Budget Year 2009 business case did not disclose the IRS’ plans to merge the CIS with the Accounts Management Services system. The CIS business case showed that the System would remain operational through Fiscal Year 2017. The IRS also projected costs and benefits in the System business case through Fiscal Year 2017, even though the System would no longer exist after Fiscal Year 2009. The Accounts Management Services system Budget Year 2009 business case did report the planned merger, but it was not consistent with the information presented in the CIS business case.
The CIS business case contained inaccurate information because the IRS did not take timely actions to prepare a Baseline Change Request. Several IRS officials did not fully understand the requirements for preparing a business case or a Baseline Change Request in situations where two information technology systems are merged.
We have found similar problems related to the reliability of information contained in IRS business cases in prior audits.[14] For those audits, we issued reports with several recommendations designed to improve the reliability of IRS business cases.
Management Action: Subsequent to the completion of audit fieldwork, the IRS provided a copy of a May 13, 2008, memorandum to the Department of the Treasury Capital Planning and Investment Control Desk Officer requesting approval of the Account Management Services system and CIS merger. The IRS requested approval to merge the systems as instructed by Department of the Treasury guidance in the Information Technology, Capital Planning and Investment Control Policy Guide, Appendix H - Closing and Merging Major Information Technology Investments.
Recommendation
Recommendation 3: The Chief Information Officer should ensure that IRS project officials make a timely request for Department of the Treasury approval to merge and close investments and restate expected costs and benefits.
Management’s Response: IRS management agreed with this recommendation. They will give presentations to the Executive Steering Committees as part of the enterprise governance process to reinforce the need to secure Department of the Treasury approval when a decision to merge investments has been made. IRS project officials will prepare the new Exhibit 300 required to effectively merge the CIS and the Accounts Management Services system investments and will restate the expected costs and benefits.
Appendix I
Detailed Objective, Scope, and Methodology
The overall audit objective was to determine whether the IRS had established and implemented adequate internal controls to ensure the stability and performance of the CIS and to ensure proper planning for its future merger with the Accounts Management Services system.[15] To accomplish this objective, we:
I. Determined whether the IRS had taken adequate actions to ensure the stability and optimal performance of the CIS by appropriately addressing the recommendations from the Incident Analysis Team[16] report.
A. Reviewed status reports on actions taken by the IRS to address each of the nine recommendations in the Incident Analysis Team report.
B. Determined whether the Incident Analysis Team report appropriately identified the causes of the stability problems within the CIS.
II. Determined whether the IRS was achieving the technical and business performance standards originally planned for delivery of the CIS. We determined whether the CIS consistently has met or is meeting technical and business performance standards.
III. Determined whether the IRS had adequately planned for merger of the CIS with the Accounts Management Services system.
A. Determined how the CIS will be merged with the Accounts Management Services system.
B. Evaluated the collaboration efforts between CIS and Accounts Management Services system officials to ensure that the CIS appropriately merges with the Accounts Management Services system.
IV.
Determined whether the IRS had ensured that the business
case for the CIS accurately reflected future plans for the CIS.
A.
Determined whether the planned merger of the CIS with the
Accounts Management Services system was properly reported in the business case for
each project.
B.
Determined how future updates to the business cases will
reflect future plans for the CIS and the Accounts Management Services system.
Appendix II
Major Contributors to This Report
Margaret
E. Begg, Assistant Inspector General for Audit (Information Systems Programs)
Scott
Macfarlane, Director