TREASURY
INSPECTOR GENERAL FOR TAX ADMINISTRATION
Oversight of the Electronic Fraud Detection System Restoration Activities Has Improved, but Risks Remain
March 29, 2007
Reference Number: 2007-20-052
This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.
Redaction Legend:
3(a) = Identifying Information
- Name of an Individual or Individuals
3(d) = Identifying Information - Other Identifying Information of an Individual
or Individuals
Phone Number |
202-927-7037
Email Address | Bonnie.Heald@tigta.treas.gov
Web Site |
http://www.tigta.gov
March 29, 2007
MEMORANDUM FOR CHIEF INFORMATION OFFICER
FROM: Michael R. Phillips /s/ Michael R. Phillips
Deputy Inspector General for Audit
SUBJECT: Final Audit Report – Oversight of the Electronic Fraud Detection System Restoration Activities Has Improved, but Risks Remain (Audit # 200620042)
This report presents the results of our review to determine whether the Internal Revenue Service
(IRS) adequately monitored the contractors’ development efforts in 2006 to
ensure the Electronic Fraud Detection System (hereafter referred to as EFDS or
System)[1] was delivered in time for the 2007 Filing
Season. This audit is a follow-up to a prior Treasury
Inspector General for Tax Administration audit.[2]
Impact on the Taxpayer
The EFDS
is the primary information system used to support the Criminal Investigation
Division’s Questionable Refund Program, which is a nationwide program
established in January 1997 to detect and stop fraudulent and fictitious claims
for refunds on income tax returns. During Processing Year 2006, the System was
not operational because the IRS and its contractors were unable to launch a
web-based version of the EFDS application (Web EFDS), resulting in an estimated
$318.3 million in fraudulent refunds being issued as of May 19, 2006. The IRS has improved controls over the EFDS
restoration activities including executive governance and project
management. As a result, project risks
are being identified and mitigation actions are being taken to ensure the System
is implemented and fraudulent refunds stopped during Processing Year 2007.
On April 19, 2006, all system development activities for the Web EFDS were stopped and all efforts were focused on restoring the client-server EFDS for use in January 2007. The restoration effort requires the contractors to prepare the System and the related databases for Processing Year 2007 by starting with the Processing Year 2005 EFDS and updating it with the 2006 and 2007 tax law changes. Therefore, the System restoration work to be completed by the contractors involves the routine annual update of the System with tax law changes and does not contain the level of complexity involved in redesigning it into a web-based system.
In the prior EFDS audit, we reported the IRS did not ensure the EFDS project had the required executive oversight, manage the System risks effectively, monitor contractor performance effectively, and use performance-based contracts. The EFDS project also was improperly classified as a steady state project in the business case. During this audit, we determined that IRS management completed several corrective actions in response to our prior audit report.
IRS
management implemented executive oversight and improved project management
controls. However, the Federal
Government may not receive the full amount of the equitable adjustment.
The IRS improved executive oversight of the EFDS project by requiring
the status and risks of the project be reported at various meetings. Additionally, project management controls were
improved. For example, regular meetings are held with stakeholders
and contractors to ensure tasks are on target for timely completion and risks
are addressed. If tasks are not
completed when scheduled, the effect on the overall schedule is determined and remedial
actions are taken, if needed.
The EFDS Project Office
also obtained project management support from contractor Booz Allen Hamilton,
Inc., and obtained independent assessments of the System from the MITRE Corporation
at an estimated cost of $1,722,132. These
expenses are considered inefficient use of resources because the expenses would
not have been incurred if the Web EFDS had been implemented in Processing Year
2006 (see Appendix IV).
Although project management controls have
improved, as of the time of our review on December 8, 2006, risks remained
as several critical tasks had not been completed. For example, the EFDS
(applications and 3 years of data) must be loaded into the production
environment, final integration testing must be completed, and the required Enterprise
Life Cycle documents must be prepared.
This audit was conducted while the IRS was
performing restoration activities to implement the System in Processing Year
2007. Any changes that occurred since we
completed our analysis in December 2006 are not reflected in this report. As a result, this report may not reflect the
most current status of the EFDS project.
According to the IRS, the System was placed into production on January
16, 2007.
During this audit, we also determined the Contracting Officer’s Technical Representative oversight of the Computer Sciences Corporation (CSC) had not changed significantly and the EFDS Project Office is in the process of drafting procedures for monitoring acquisitions. Meanwhile, compensating controls, such as the improvements in project management, mitigate the oversight risks.
The IRS recently issued a contract for an estimated amount
of $3,080,004 for restoration work to be performed from November 1, 2006,
through February 24, 2007. We reviewed the contract and found that
payment of the contractor’s fee is not dependent on the timely delivery of specific
System deliverables or milestones. The
contract also established a cost sharing amount not to exceed $3,080,004 as an
equitable adjustment amount to compensate the IRS for the
cost to restore the client-server EFDS. However, the agreement does not
include a provision that would refund
the unused equitable adjustment
to the IRS and the cost sharing commitment is exclusively related to delivering
a client-server EFDS in January 2007.
Based on our review of the EFDS project work breakdown structure (i.e., a list of all tasks required to complete the project) it does not appear the CSC has $3,080,004 worth of work remaining on the restoration project. The EFDS Executive agreed with this conclusion and stated the CSC has verbally agreed to work on two application changes unrelated to the restoration work to ensure the IRS will receive the $3,080,004 equitable adjustment. However, the contract states the CSC’s cost sharing commitment is exclusively related to delivering a client-server-based System and will not apply to any Federal Government directed scope increases. Therefore, the IRS will be obligated to pay the contractor’s fee if a functional EFDS is not implemented timely and the IRS may not receive the entire equitable adjustment.
Recommendation
We recommended the Chief Information Officer work with the Director, Procurement, to ensure the IRS receives all of the $3,080,004 equitable adjustment from the CSC. If the entire adjustment is not received by the end of the original period of performance stated in the contract, the IRS should request the CSC pay the IRS the difference between the $3,080,004 and the credit the IRS received during the period of performance. Alternatively, the IRS should request the application of the remaining equitable adjustment credit owed to the IRS to invoices for future EFDS-related task orders or for other work being performed by the CSC.
Response
IRS management agreed with the recommendation and prepared a modification to the task order to ensure the IRS receives the full equitable adjustment. The modification, signed by the IRS and the CSC on February 23, 2007, extends the base period of performance and includes additional work within the scope of the cost sharing agreement. Management’s complete response to the draft report is included as Appendix IX.
Copies of this report are also being sent to the IRS managers affected by the report recommendation. Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.
Executive Oversight of the Electronic Fraud Detection System Has
Improved
Contracting Activities Have Improved,
but a Cost Reimbursement Issue Remains
Appendices
Appendix
I – Detailed Objective, Scope, and Methodology
Appendix
II – Major Contributors to This Report
Appendix
III – Report Distribution List
Appendix
IV – Outcome Measures
Appendix V –
Electronic Fraud Detection System Management
Appendix
VI – Electronic Fraud Detection System Oversight
Appendix
VII – Electronic Fraud Detection System Project Timeline
Appendix
VIII – Glossary of Terms
Appendix
IX – Management’s Response to the Draft Report
|
CIO |
Chief Information Officer |
|
COTR |
Contracting Officer’s Technical Representative |
|
CSC |
Computer Sciences Corporation |
|
EFDS, System |
Electronic Fraud Detection System |
|
ESC |
Executive Steering Committee |
|
IRS |
Internal Revenue Service |
|
MITRE |
MITRE Corporation |
|
MITS |
Modernization and Information Technology
Services |
|
PY |
Processing Year |
|
Web EFDS |
Web Electronic Fraud Detection System |
The Electronic Fraud Detection System (hereafter referred to as the EFDS or System)[3] is an automated compliance system designed to maximize fraud detection when tax returns are filed and prevent the issuance of fraudulent refunds. The EFDS is the primary information system used to support the Criminal Investigation Division’s Questionable Refund Program, which is a nationwide program established in January 1977 to detect and stop fraudulent and fictitious claims for refunds on income tax returns.
In January 2006, the Internal Revenue Service (IRS) planned to launch a web-based version of the EFDS application (Web EFDS) after failing to implement the Web EFDS in January 2005 because of system development problems. However, the IRS and its contractors were unable to provide a functioning Web EFDS to prevent fraudulent refunds during Processing Year (PY) 2006. During PY 2006, the System was not operational, resulting in an estimated $318.3 million in fraudulent refunds being issued as of May 19, 2006.
On April 19, 2006, all system development activities for the Web EFDS were stopped, and all efforts were focused on restoring the client-server EFDS for use in January 2007. The restoration effort requires the contractors to prepare the System and the related databases for PY 2007 by starting with the PY 2005 EFDS and updating it with the 2006 and 2007 tax law changes. Therefore, the System restoration work to be completed by the contractors involves the routine annual update of the System with tax law changes and does not contain the level of complexity involved in redesigning the System into a web-based system.
Five contractors are involved in various EFDS activities. Three of the contractors are working to restore the System for PY 2007, while the remaining two contractors provide program management support. The responsibilities of the five contractors include the following:
· Computer Sciences Corporation (CSC), the primary contractor, is responsible for delivering a fully operational client-server-based System in January 2007. As of December 11, 2006, the total amount paid to the CSC for System restoration work was $2,613,953. In addition, a task order with an estimated cost of $3,080,004 was approved on October 24, 2006, for restoration work to be performed through February 24, 2007.
· Systems Research and Applications Corporation is responsible for providing and maintaining data-mining techniques used by the EFDS. As of December 11, 2006, the total amount paid to the Systems Research and Applications Corporation for the System restoration was $167,584. In addition, a task order with an estimated cost of $420,648 was approved on July 28, 2006, for work to be performed through July 31, 2007. The remaining funds available for this task order are $336,859.
· Anteon Corporation is responsible for providing maintenance support for the EFDS client-server application and database. A task order was approved on August 15, 2006, with an estimated cost of $1,500,000 for work to be performed between April 11, 2006, and February 24, 2007. Because the work performed by Anteon Corporation is critical to the System restoration, it was allowed to begin work before the task order was approved. As of December 11, 2006, the total amount paid to the Anteon Corporation for the EFDS restoration was $707,006. The remaining funds available for this task order are $792,994.
· Booz Allen Hamilton, Inc. is responsible for providing EFDS Project Office support. A task order with an estimated cost of $1,201,378 for project management support services was awarded July 6, 2006, for work to be performed through July 1, 2007.
· MITRE Corporation (MITRE) is responsible for providing independent assessments of the System restoration activities. A task order with an estimated cost of $103,024 was approved on September 14, 2006, for work to be performed through December 31, 2006.
This review is a follow-up to a prior
Treasury Inspector General for Tax Administration audit.[4] This review was performed at the Modernization and Information Technology Services (MITS)
organization offices in New Carrollton, Maryland, and
Executive Oversight of the Electronic Fraud Detection System Has Improved
The Clinger-Cohen Act of 1996[5] requires agencies to use a disciplined Capital Planning and Investment Control process to acquire, use, maintain, and dispose of information technology assets. The Office of Management and Budget Circular A-11, Preparation, Execution, and Submission of the Budget, dated June 2006, requires each agency to include with its annual budget submission an information technology investment portfolio, commonly referred to as an Exhibit 53, containing the information technology investment title, description, amount, and funding source. For each major information technology investment, the Office of Management and Budget requires agencies to include Circular A-11 Exhibit 300, Capital Asset Plan and Business Case, with their budget submissions.
The IRS’ Capital Planning and Investment Control process for managing information technology projects established an executive governance process for monitoring projects that included the MITS Enterprise Governance Committee, the MITS Enterprise Governance Investment Management Subcommittee, and Executive Steering Committees (ESC) responsible for specific projects. Major projects with costs of more than $5 million per year or total lifecycle costs of more than $50 million were to be governed by the executive governance process. Formal agendas, presentations, and meeting minutes are prepared for each ESC meeting including documenting key decisions and assignments. To assess the controls over the EFDS project, we reviewed the policies and procedures applicable to the project and determined whether they were implemented effectively.
In the prior EFDS audit, we reported the Exhibit 300 improperly classified the EFDS as a steady state project. This was improper because, at the time, the IRS was in the process of developing the Web EFDS. In addition, information in the Exhibit 300 was not consistent and presented the EFDS as both a steady state system and a system under development.
The EFDS project did not have continuous ESC oversight as required by the Capital Planning and Investment Control process. Instead, there was ESC oversight from June 2002 until July 2003. Afterwards, oversight was provided by Business Systems Development organization executives who were also responsible for managing the maintenance and development work for more than 325 IRS systems.
We also reported that key decisions relating to the Web EFDS development were not adequately documented. Consequently, we made the following recommendations:
IRS
management implemented executive oversight and completed several corrective actions
in response to our prior audit report. As
of December 8, 2006, the EFDS Project Office reported the project was on
schedule and implementation was expected to occur on January 16, 2007.
During this audit, we determined that IRS management implemented executive governance oversight and completed several corrective actions in response to our prior audit report. For example, the EFDS project was assigned to the Compliance ESC.[6] Discussion items, actions, and decisions resulting from these meetings are documented in the meeting minutes. This corrective action addresses Recommendation 1 from the prior audit report. The System is also included in the Senior Management Dashboard Review. The System risks, issues, and mitigation strategies identified at the Senior Management Dashboard Review meetings are documented and tracked. The Senior Management Dashboard Review is attended by one or more executives from the Enterprise Services organization and representatives of the projects under review. This corrective action partially addresses Recommendation 2 from the prior audit report. The IRS Commissioner is also briefed monthly on the status of the System activities by the CSC (see Appendix VI for a list of recurring executive briefings).
The IRS stopped the Web EFDS development and is restoring the client-server EFDS for use in PY 2007. The EFDS Project Office revised the Exhibit 300 to correctly support classifying the System restoration as a steady state project. On September 11, 2006, the IRS submitted a revised Exhibit 300 which was approved by the Department of the Treasury. This corrective action addresses prior audit Recommendation 3. The remaining corrective actions, which are to evaluate other projects and assign them the appropriate oversight, are in process and scheduled to be completed by April 1, 2007.
Continuous executive oversight of a project helps to ensure risks are identified and mitigated. As of December 1, 2006, the EFDS Project Office reported the project is on schedule and implementation is expected to occur on January 16, 2007.
Electronic Fraud Detection System Restoration Project Management Controls Have Been Improved, but Risks Remain
The Department of the Treasury
Publication 84-01, Information System Life Cycle Manual, dated March
2002, states that general standardization of life cycle management ensures
systems are developed, acquired, evaluated, and operated efficiently, within
prescribed budget and schedule constraints, and are responsive to mission
requirements. In addition, the IRS system development
guidelines (currently, the Enterprise Life Cycle - Lite) stipulate that, as
part of the information system life cycle management process, project
management should identify project risks early and manage them before they
become problems. The risk management
process encompasses the identification of risk issues, assessment of risk to
define probability and impact, preparation and implementation of risk
mitigation and risk contingency plans, and continuous monitoring of those
actions to ensure effectiveness. Risk
management is used to ensure critical areas of uncertainty are surfaced early
enough to be addressed without adversely affecting cost, schedule, or
performance.
In the prior EFDS audit, we reported that the risks were not managed effectively;
status meetings with stakeholders were held, but the meeting results were not documented
sufficiently, if at all; individuals were not held accountable for timely
completion of tasks; a process to adequately and independently confirm the
completion of tasks had not been established nor documented; and key management
documents were not prepared or properly maintained. As a result, we made
the following recommendations:
·
Recommendation 4: The CIO
should ensure project risks are identified properly and plans are prepared to
reduce the risks affecting the successful development of the project.
·
Recommendation 5: The CIO
should ensure the proper system development life cycle methodology is implemented
for the EFDS development, based on the types of changes being made to the
system.
For the current client-server EFDS restoration project, the EFDS Project Manager monitors the
contractor and IRS progress and performance to ensure the project is on
schedule. During the Web EFDS
development, the EFDS Project Manager was also the EFDS/Questionable Refund
Program Section Chief and performed some of the CSC Contracting
Officer’s Technical Representative (COTR)
duties. For the EFDS restoration project,
the three duties were assigned to separate individuals. See Appendix V for a list of the individuals
responsible for the EFDS project. IRS management
stated that spreading out the assignments made the project less difficult to
manage. In addition, the CSC no longer maintains the project work
breakdown structure (i.e., a list of all tasks and the tasks completion dates required to complete the
project timely). Instead, it is maintained by Booz Allen Hamilton,
Inc. which is providing program management support to the EFDS Project
Office. The project tasks are divided into shorter manageable increments to
facilitate task monitoring, validation, and inclusion on the project schedule.
Improvements in project management controls include holding regular meetings
with stakeholders and contractors to ensure tasks are on target for timely
completion and risks are addressed. If
tasks are not completed when scheduled, the impact on the overall schedule is
determined and remedial actions are taken, if needed. Stakeholder involvement ensures that
activities and decisions adequately address the business concerns and completed
tasks are satisfactory. Examples of
meetings held include the Weekly Stakeholder Status Meetings, the Weekly
Technical Meetings, and the Filing Season Readiness Meetings (see Appendix VI for
a list of recurring meetings).
EFDS
project management and risk identification and mitigation have improved. For example, if tasks are not completed when
scheduled, the impact on the overall schedule is determined and remedial
actions are taken, if needed.
In addition, a process was established, documented, and
implemented to monitor the status and verify the satisfactory completion of
tasks. Risks and mitigation activities discussed
during the weekly stakeholder status meetings are documented in status reports
and/or meeting minutes and a database maintained by Booz Allen Hamilton, Inc. This corrective action and the executive
oversight discussed above address prior audit Recommendation 4.
The Enterprise Life Cycle Project
Office also performed an analysis to determine what Enterprise Life Cycle - Lite documents should be
produced for a steady state project. On October
30, 2006, the Compliance ESC gave the EFDS Project Office approval to limit the
Enterprise Life Cycle documents to five required documents (Business Systems
Requirements Report, Requirements Traceability Matrix, Test Plan, Transition
Management Plan, and 508 Compliance). Figure
1 provides the status of the EFDS Project Office’s preparation of the documents
as of October 31, 2006. This corrective
action addresses prior audit Recommendation 5.
Figure 1: Status of
|
Document Name |
Preparation Status |
|
Business Systems Requirements Report |
Completed |
|
Requirements Traceability Matrix |
Completed |
|
Test Plan |
Completed |
|
Transition Management Plan |
In Planning |
|
508 Compliance |
In Process |
Source: EFDS Project Office.
As previously stated, the EFDS Project Office contracted with Booz Allen
Hamilton, Inc. to provide program/project management support for the restoration of the client-server EFDS
application. The estimated cost
of this program management support is $1,201,378. The contract states Booz Allen Hamilton, Inc.
employees will:
·
Help to accurately monitor and timely report risks,
issues, project status, and action items.
·
Provide technical architecture support to help
assess technical issues caused by the PYs 2006 and 2007 changes.
·
Maintain the work breakdown structure and enter start
and completion dates into the schedule.
When a completion date has not been met or it appears that a completion
date will not be met, contractor support determines the effect of the delay on
other tasks, the overall schedule, and the stakeholders.
MITRE was also hired as
the IRS’ Federally Funded Research and
The second MITRE study, dated October 5, 2006, will cost an
estimated $103,024 and assessed the client-server EFDS’ readiness to
successfully perform refund fraud detection functions in PY 2007. MITRE issued a preliminary assessment stating the
project is on a path for successful implementation
and there were no significant issues or risks that would prevent
delivery of a functioning system by January 2007. MITRE planned to reassess EFDS project
readiness on or after November 17, 2006, after PY 2006 data loads were
completed and after the Criminal Investigation Division completed its data
quality reviews. However, on December 6,
2006, the EFDS Executive advised us the IRS will not be inviting MITRE to
perform another readiness assessment because the project is on schedule and he
did not want to subject the EFDS Project Office to another third-party review as
it would not provide any new information.
The IRS will spend an estimated $1,722,132 for Booz Allen Hamilton, Inc. project management support and MITRE independent assessments. These expenses are considered inefficient use of resources because the expenses would not have been incurred if the Web EFDS had been implemented in PY 2006 (see Appendix IV).
Overall, oversight of the client-server EFDS restoration project has improved because management implemented effective project management controls and completed several corrective actions in response to our prior audit report. However, as of December 8, 2006, risks remained as several critical tasks had not been completed.
· The EFDS (applications and 3 years of data) must be loaded into the production environment. The planned completion date is December 29, 2006.
· Final integration testing must be completed. The planned completion date is December 29, 2006.
· Security Certification and Accreditation must be completed. The planned completion date is January 8, 2007.
· Disaster recovery testing will not be performed prior to the January implementation. It is scheduled to occur after the filing season. The tentative test date is September 2007 and this test is included in a broader IRS disaster recovery test.
As a result of improved project management, risks identified thus far have been mitigated and the System restoration is on schedule for the January 16, 2007, implementation.
This audit was conducted while the IRS was
performing restoration activities to implement the System in PY 2007. Any changes that occurred since we completed
our analysis in December 2006 are not reflected in this report. As a result, this report may not reflect the
most current status of the EFDS project.
According to the IRS, the System was placed into production on January
16, 2007.
Contracting Activities Have Improved, but a Cost Reimbursement Issue Remains
The Federal
Acquisition Regulation[7] holds contractors responsible for timely
contract performance; however, the Federal Government is also responsible for
monitoring contractor performance, as necessary, to protect its interest. This monitoring should include comparing a
contractor’s performance plans, schedules, controls, and processes against the
contractor’s actual performance; determining the contractor’s progress; and
identifying any factors that may delay performance. Agencies are also required to develop quality
assurance surveillance plans when acquiring services specifying the work
requiring surveillance and the method of surveillance. The IRS Office of Procurement Policy best
practices state that a planned surveillance effort is necessary to measure
contractor performance and ensure successful completion of tasks.
Contracting Officers are responsible for ensuring performance of all necessary actions for effective contracting, ensuring compliance with the terms of the contract, and safeguarding the interests of the Federal Government in its contractual relationships. Since many of the Contracting Officers’ responsibilities can be delegated to a COTR, the COTR plays a critical role in the technical administration of Federal Government contracts to assure that the Government receives the supplies or services in accordance with the contracts’ specifications. COTR responsibilities usually include monitoring contractor performance and schedule; acknowledging receipt of supplies or services with an acceptance certificate; reviewing, commenting, and accepting or rejecting deliverables, as well as providing written evaluation of each major deliverable; and reviewing and verifying the contractor’s invoices for hours expended and costs incurred.
While contracting officials should always check the mathematical accuracy of invoices to avoid any overpayment to the contractor, cost-reimbursement contracts require a more indepth review of invoices to ensure costs are not incurred prematurely and relate to progress under the contract. As a result, COTR activities should include checking the invoice date against the contract performance period to ensure costs are being billed for the proper time period; comparing the contractor’s billing rates against the contract rates to ensure indirect costs are being properly billed; reviewing the contractor’s time cards, sign-in sheets, and overtime records to help assess the reasonableness of direct costs; and maintaining monthly reports or spreadsheets on costs incurred against the contract amount.
****3(d)**** we made the following
recommendations:
· Recommendation 6: The CIO should ensure contractors are accountable for performance by developing performance-based requirements for new EFDS contracts. The CIO should also consider employing cost-sharing arrangements for future task orders so both the IRS and contractor share the risk of project development cost overruns.
· Recommendation 7: The CIO should ensure COTRs are trained adequately and their duties are performed properly to monitor contractor performance effectively through planned surveillance efforts and independent inspections of contractor work, as described by IRS Office of Procurement Policy best practices.
· Recommendation 8: The CIO and the Director, Procurement, should initiate discussions with the contractor to recover the funds paid to the contractor to restore the old EFDS for use in PY 2005 and any additional costs resulting from nondelivery of a functional Web EFDS.
· Recommendation 9: The CIO should defer additional work on the Web EFDS until the IRS decides who will perform the EFDS work. If some or all of the work will transfer to other business units, the CIO should ensure their requirements are identified before initiating a contract for further development of the Web EFDS. The contract should be opened to competition.
COTR oversight has not significantly changed, but compensating controls mitigate the risks
****3(d)**** We determined
that COTR oversight has not changed significantly. As in the prior audit, the new COTR attends
meetings with the contractors but still depends on EFDS Project management to
provide confirmation of the status of tasks and receipt and acceptability of
deliverables. The EFDS Project Office is aware of this
dependency and has mitigated this risk by obtaining the confirmations from the
stakeholders through its weekly System status reporting process.
The COTR now reviews invoices and obtains feedback from the IRS technical points of contact and EFDS Project Office personnel to confirm technical accuracy of deliverables. However, our review of the controls over the procurement process identified issues similar to those found in the prior audit. ****3(d)**** EFDS Project Office is in the process of drafting procedures for monitoring acquisitions. Corrective actions for Recommendation 7 are scheduled to be completed by January 1, 2007.
The equitable adjustment agreement does not ensure the IRS will receive the full amount of the cost reimbursement
The IRS recently
issued a Treasury Information Processing Support Services-3 cost-plus-fixed-fee
contract that established a base period of performance of November 1, 2006, through
February 24, 2007, for EFDS restoration work at an estimated cost of
$3,080,004. The IRS reported in the
Joint Audit Management Enterprise System that this contract award completed the
corrective action for Recommendation 6 and originally stated that a percentage
of the contractor’s fee would be dependent upon timely delivery of specified
milestones. The Joint Audit Management
Enterprise System was updated subsequently to state that a percentage of the contractor’s
fee was associated with specific deliverables. However, we reviewed the signed contract and
found that payment of the contractor’s fee was not dependent on the timely
delivery of EFDS milestones or specific deliverables, and the contract did not
include milestones. As a result, the Federal
Government’s interest is not protected because it would be obligated to pay the
contractor’s fee if a functional EFDS is not implemented timely. Regarding Recommendation 6 to use
performance-based contracts, the IRS stated that future contracts for
completion of the Web EFDS will be performance-based.
Based on
the contract and the remaining CSC work identified in the work breakdown
structure, the Federal Government may not receive the full equitable
adjustment. However, the EFDS Executive
stated the CSC has verbally agreed to work on additional application changes
(unrelated to the restoration work) to ensure the IRS receives the $3,080,004
equitable adjustment.
The contract also established a cost sharing amount not to exceed $3,080,004 ($2,859,253 cost reimbursement amount and $220,751 fee) as an equitable adjustment amount. The CSC agreed to credit each invoice submitted to the IRS for work performed during the base period of performance for the cost incurred plus a fee. However, the agreement did not include a provision that would refund the unused equitable adjustment to the IRS. The equitable adjustment was included in the contract as a response to Recommendation 8 from the prior audit report.
Based on our October 25, 2006, meeting with the EFDS Project
Manager and our review of the work breakdown structure, most of the CSC’s work
was completed by October 2006. Thus, it
does not appear the CSC has $3,080,004 of work remaining. This is also supported by the EFDS
Executive’s August 3, 2006, comment to CSC and MITS executives, “Since much of the cost for restoring the EFDS
will likely have been incurred before this agreement is finalized, some of the CSC’s
cost sharing will likely be in force after the restoration is complete and the EFDS
is in operations and maintenance.” On December 6, 2006, the EFDS Executive
agreed with our conclusion and explained that, if the contract had been signed
timely, this would not have been a problem.
The EFDS Executive stated the CSC was aware of this and has verbally
agreed to work on two application changes (unrelated to the restoration work) to
ensure the IRS will receive the $3,080,004 equitable adjustment. However, the
contract states the CSC’s cost sharing commitment is related exclusively to
delivering a client-server-based System and will not apply to any Federal Government
directed scope increases. Again, the Federal
Government’s interest has not been protected because the CSC could bill the IRS
for the work that is unrelated to the System restoration without crediting the IRS
for the unused equitable adjustment.
Recommendation
Recommendation 1: The CIO should work with the Director, Procurement, to ensure the IRS receives all of the $3,080,004 equitable adjustment from the CSC. If there is not enough work to be completed by the CSC during the November 1, 2006, through February 24, 2007, period of performance to enable the IRS to receive the full adjustment, the IRS should request that the CSC pay the IRS the difference between the $3,080,004 and the credit the IRS received during the period of performance. Alternatively, the IRS should request the application of the remaining equitable adjustment credit owed to the IRS to invoices for future EFDS-related task orders or for other work being performed by the CSC.
Management’s Response: IRS management agreed with the recommendation and prepared a modification to the task order to ensure the IRS receives the full equitable adjustment. The modification, signed by the IRS and the CSC on February 23, 2007, extends the base period of performance and includes additional work within the scope of the cost sharing agreement.
Appendix I
Detailed Objective, Scope, and Methodology
The overall
objective of this review was to determine whether the IRS
adequately monitored the contractors’ development efforts in 2006 to ensure the
EFDS[8] was
delivered in time for the 2007 Filing Season. To accomplish our objective, we:
I.
Determined whether
the executive monitoring and project management processes were effective to
ensure 2007 Filing Season implementation.
A. Obtained and reviewed the minutes
and briefing materials for the Compliance ESC and Senior Management Dashboard Reviews; the monthly briefings presented by the
CSC (i.e., the PRIME contractor) to the IRS Commissioner; the MITS and Criminal Investigation Division Business Performance
Reviews; the Enterprise Life Cycle Gap
Analysis; and the results of the CIO’s program review of the EFDS project.
B.
Determined the effectiveness of project management controls.
1.
Interviewed EFDS Project Office management to determine how they
monitored contractor progress and performance.
We also obtained and reviewed status reports and project schedules used
to monitor contractor progress and performance and determined whether the
status reports documented when critical problems occurred initially, when they
were elevated for resolution, and how management validated the accuracy of the
schedule.
2.
Obtained and reviewed minutes of the monthly meetings between the
Compliance Domain Director, EFDS Project Manager, Criminal Investigation
Division representative, and CSC personnel to determine the issues and related
resolutions that were discussed.
3.
Determined what risks were identified and whether risk mitigation
plans were prepared.
4.
Interviewed Criminal Investigation Division management to
determine whether they needed the System in advance to prepare and conduct
their training.
5.
Interviewed EFDS Project Office and Criminal Investigation
Division management to determine what contingency plans were developed to
minimize the effect to the Questionable Refund Program in the event the client-server
EFDS was not implemented timely or with full functionalities.
6.
Interviewed the IRS employees responsible for conducting the
System Acceptability Testing for the restored EFDS to determine the status and
results of testing as well as whether the contractor submitted quality
products.
C.
Identified the contractor support
that was obtained to assist the EFDS Project Office in the System restoration.
1.
Interviewed EFDS Project Office management
to identify the contractors and the scope of their work on the restoration.
2.
Obtained the contracts/task
orders/statements of work for the restoration efforts to determine the scope of
work and restoration costs for each contractor and reviewed the CSC
contract/task order/statement of work to determine the amount the IRS would
receive as an equitable adjustment for the Web EFDS not being implemented in
2006.
3.
Validated the invoice amounts
supplied by the COTR by comparing the invoice to information in the IRS’ Web
Request Tracking System.
4.
Reviewed the MITRE report
assessing the System restoration efforts to determine the effect, if any, on
our audit work.
II.
Determined
whether the COTRs for the contracts and task orders were effectively monitoring
and documenting the contractors’ progress and performance on the System
restoration work.
A. Obtained and reviewed policies
and procedures for monitoring contractor progress and performance.
B.
Interviewed the COTRs and
identified their process for monitoring the contractors and performing
independent inspections to ensure the work was on schedule and met the contract
terms and user requirements. We also
obtained and reviewed documentation of independent inspections, if performed.
C.
Obtained and reviewed status
reports and minutes of meetings between the COTRs and contractors working on
the EFDS project, if taken.
III.
Determined
whether effective corrective actions were implemented to address the
recommendations in the prior EFDS audit report, The Electronic Fraud Detection System Redesign Failure Resulted in
Fraudulent Returns and Refunds Not Being Identified (Reference Number
2006-20-108, dated August 9, 2006) and the MITRE report, Electronic Fraud Detection System (EFDS) Project Final Assessment
Report Version 1.0, dated June 9,
2006.
A. Reviewed the Joint Audit
Management Enterprise System to determine the status of the corrective actions.
B.
Obtained documentation to verify
closed corrective actions were implemented.
C.
Interviewed the EFDS Project Manager to determine the IRS’
decision on implementing the MITRE recommendations (e.g., the number of
recommendations agreed to, implemented, rejected, etc.).
Appendix II
Major Contributors to This Report
Margaret E. Begg,
Assistant Inspector General for Audit (Information Systems Programs)
Gary
Hinkle, Director
Danny
Verneuille, Audit Manager
Tina Wong, Lead
Auditor
Phung-Son Nguyen, Senior Auditor
Van
Warmke, Senior Auditor
Olivia
DeBerry, Auditor
Linda
Screws, Auditor
Appendix III
Commissioner C
Office
of the Commissioner – Attn: Chief of
Staff C
Deputy
Commissioner for Operations Support OS
Deputy Commissioner for
Services and Enforcement SE
Chief, Agency-Wide Shared Services OS:A
Chief, Criminal Investigation SE:CI
Deputy Chief Information Officer OS:CIO
Deputy Chief, Criminal Investigation SE:CI
Associate Chief Information Officer, Applications
Development OS:CIO:AD
Director, Procurement
OS:A:P
Director, Refund Crimes
SE:CI:RC
Director, Stakeholder Management OS:CIO:SM
Chief
Counsel CC
National
Taxpayer Advocate TA
Director,
Office of Legislative Affairs CL:LA
Director,
Office of Program Evaluation and Risk Analysis
RAS:O
Office of
Internal Control OS:CFO:CPIC:IC
Audit
Liaisons:
Deputy
Commissioner for Operations Support OS
Deputy
Commissioner for Services and Enforcement
SE
Chief, Agency-Wide
Shared Services OS:A
Director, Procurement
OS:A:P
Director, Program Oversight
Office OS:CIO:SM:
Appendix IV
This appendix presents detailed information on the measurable impact that our recommended corrective actions will have on tax administration. These benefits will be incorporated into our Semiannual Report to Congress.
Type and Value of Outcome Measure:
· Inefficient Use of Resources – Potential; $1,201,378 (see page 5).
Methodology Used to Measure the Reported Benefit:
The EFDS Project Office
has obtained program management support from Booz Allen Hamilton, Inc. The support
that contract employees will provide includes helping to accurately monitor and
timely report risks, issues, project status, and action items; providing
technical architecture support to help assess technical issues caused by the PYs
2006 and 2007 changes; and maintaining the work breakdown structure. If the Web EFDS had been implemented in PY
2006, program management support would not have been required. The estimated cost of the project management
support services is $1,201,378.
Type and Value of Outcome Measure:
· Inefficient Use of Resources – Potential; $417,730 (see page 5).
Methodology Used to Measure the Reported Benefit:
The IRS hired the MITRE to perform a study to determine the root causes of the Web EFDS project performance issues and recommend actions to address those issues, assess the EFDS Web Portal system and render an opinion on its future viability, and recommend actions to apply the lessons from the System situation across the information technology portfolio to improve the delivery of other projects of similar size, scope and complexity. If the Web EFDS had been implemented timely and successfully in PY 2006, the IRS would not have requested the study which is estimated to cost $417,730.
Type and Value of Outcome Measure:
· Inefficient Use of Resources – Potential; $103,024 (see page 5).
Methodology Used to Measure the Reported Benefit:
The IRS hired the MITRE to perform a study to assess the client-server EFDS’ readiness to successfully perform refund fraud detection functions in PY 2007. If the Web EFDS had been implemented timely and successfully in PY 2006, the IRS would not have requested the study which is estimated to cost $103,024.
Appendix V
Electronic Fraud Detection System
Management
|
Title |
Employee’s Name |
Date |
|
CIO/Acting CIO |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
|
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
|
Deputy CIO |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
Associate CIO, Applications
Development |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
|
Deputy Associate CIO,
Applications Development |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
|
Compliance Director/Acting
Compliance Director |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
Chief Enforcement
Division/Acting Chief Enforcement Division |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
EFDS/Questionable Refund
Program Section Chief |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
EFDS Project
Manager/Acting EFDS Project Manager |
****3(a), 3(d)**** |
****3(a), 3(d)**** |
|
****3(a), 3(d)**** |
****3(a), 3(d)**** |
Source: Meetings
with EFDS management and our analysis of MITS organization documents.
Appendix VI
Electronic Fraud Detection System Oversight
Figure 1: Meetings Attended by the EFDS Project Staff, Stakeholders,
Contractors and/or Executives Assigned to Oversee the EFDS Project
|
Meetings |
Frequency |
|
Stakeholder
status meetings are held to discuss and analyze the project status and
schedule, risks, and risk mitigation strategies. |
Weekly |
|
Technical
meetings are held to review and propose solutions to technical issues
regarding the EFDS restoration effort |
Weekly |
|
The COTRs and contractors meet to
discuss the status of the project (e.g., whether work is on schedule and meets
the users’ needs). |
Bi-weekly |
|
Senior Management Dashboard Review meetings are
held to facilitate common understanding of the status of each project among Government
and contractor representatives. Only
problem areas or notable status changes are discussed. |
Monthly |
|
ESC meetings are held to oversee investments and
ensure business risks are known and quantified. |
Monthly |
|
Filing Season Readiness meetings are held to
discuss the status and issues regarding requests for application changes
needed for the filing season. |
Weekly - prior to the filing season. |
Source:
EFDS Project Office and various IRS documents.
Figure 2: Meetings
the EFDS Project Office Reported
It Provides Project Status Briefings
|
Meetings |
Frequency |
|
Commissioner’s Monthly
Meeting |
Monthly |
|
Filing Season
Executive Meeting |
Monthly |
|
Business Performance Reviews |
Quarterly |
|
Operational Reviews of
the Applications Development Domain |
Quarterly |
|
Project Status Review |
Quarterly |
Source: EFDS Project Office.
Appendix VII
Electronic Fraud Detection System Project Timeline
|
April 17, 2006 |
MITS executives
and the IRS Commissioner made the decision to restore the client-server EFDS. |
|
April 19, 2006 |
All system
development activities on the Web EFDS were stopped. |
|
May 31, 2006 |
The System was assigned to the Compliance ESC. |
|
****3(d)**** |
****3(d)**** |
|
June 27, 2006 |
The Senior Management Dashboard Review began including
the EFDS project in its reviews. |
|
July 6, 2006 |
The
IRS approved a task order for Booz Allen Hamilton, Inc. for the period July 2,
2006, through July 1, 2007, with an estimated cost of $1,201,378 for
providing project office support. |
|
July 28, 2006 |
The
IRS approved a task order for Systems
Research and Applications Corporation for the period August 1, 2006, through
July 31, 2007, with an estimated cost of $420,648 for providing and
maintaining data-mining techniques used by the System. |
|
August 15, 2006 |
The
IRS approved a task order for Anteon
Corporation for the period |
|
September 14, 2006 |
The IRS approved a
modification to an existing task order for the MITRE for the period July 3,
2006, through December 31, 2006. The
modification had an estimated cost of $103,024
for independent assessments of the restoration
activities. |
|
October 24, 2006 |
The
IRS approved a task order under
the Treasury Information Processing Support Services – 3
contract for the CSC for the period November 1, 2006, through February 24, 2007, with an estimated cost of $3,080,004 for delivering a
fully operational client-server-based EFDS for PY 2007. |
|
November 6, 2006 |
The IRS completed loading the 2006 daily tax
return information into the EFDS databases. |
|
December 6, 2006 |
The IRS completed
its test of the System application that will be used in |
|
December 29, 2006 |
The IRS is scheduled to complete the loading of the System applications
and 3 years of data into the production environment. |
|
January 8, 2007 |
The EFDS Security Certification and Accreditation is scheduled to be
completed. |
|
January 16, 2007 |
The System is scheduled for implementation in the production
environment. |
Appendix VIII
|
Business Case |
Required by Office
of Management and Budget Circular A-11 (Preparation,
Execution, and Submission of the Budget; dated June 2005) and commonly
called Exhibit 300, Capital Asset Plan and Business Case. Each agency must submit a business case
twice each year for each major information technology investment. |
|
Client-server |
A network
architecture in which clients are personal computers or workstations on which
users run applications. Clients rely
on servers for resources such as files, devices, and even processing power. |
|
Contracting
Officer’s Technical Representative |
Furnishes
technical direction, monitors contract performance, and maintains an arm’s-length
relationship with the contractor. |
|
Cost-Plus-Fixed-Fee
Contract |
A
cost-reimbursement contract that provides for payment to the contractor of a
negotiated fee that is fixed at the inception of the contract. This contract type permits contracting for
efforts that might otherwise present too great a risk to contractors, but it
provides the contractor only a minimum incentive to control costs. |
|
Cost-Reimbursement
Contract |
A contract that provides for payment
of allowable incurred costs, to the extent prescribed in the contract. |
|
Data Loads |
A process of
placing data into a system or database. |
|
Data-Mining
Technique |
A process of
automatically searching large volumes of data for patterns. |
|
|
A required system
development methodology for all nonmodernization projects. |
|
Executive Steering
Committee |
A committee that oversees
investments, including validating major investment business requirements and
ensuring that enabling technologies are defined, developed, and implemented. |
|
Federally Funded Research
and |
An organization that uses private resources to
accomplish tasks that cannot be effectively completed by existing Federal Government
employees or contractors. |
|
Filing Season |
The period
from January through mid-April when most individual income tax returns are
filed. |
|
Information
Technology Investment Portfolio |
A portfolio required
by Office of Management and Budget Circular A-11 and commonly referred to as
an Exhibit 53. This portfolio must be
submitted with each agency’s annual budget submission and contains the
information technology investment title, description, amount, and funding
source. |
|
Joint Audit
Management |
A system used to
document and track the status of recommendations from audit reports and their
corresponding corrective actions. |
|
MITS |
The highest level
recommending and decision-making body to oversee and enhance enterprise
management of information systems and technology. It ensures strategic modernization and
information technology program investments, goals, and activities are aligned
with and support 1) the business needs
across the enterprise and |
|
MITS |
A body that supports
the MITS Enterprise Governance Committee in the realization of the IRS
Capital Planning and Investment Control process and with the management of
the IRS information technology investment portfolio. This Subcommittee provides general
information technology investment portfolio oversight, including operational
analysis reviews and reports, investment prioritization recommendations, and
recommendations for adjustments to the IRS portfolio. |
|
Performance-based Contract |
A contract that provides
for acquiring services on the basis of required results rather than the
methods of performing the work and uses measurable performance standards |
|
Processing Year |
The year in which
tax returns and other tax data are processed. |
|
Quality Assurance Surveillance Plan |
A plan that ensures
services provided by the contractor meet contract requirements. It should specify the work requiring
surveillance and the method of surveillance. |
|
Questionable
Refund Program Computer Identification Program |
An application running on the mainframe computer. The Program was originally developed by the
IRS Inspection Service and run by the Internal Audit function (now the Treasury Inspector General for Tax Administration
Office of Audit). |
|
Security Certification and
Accreditation |
A security
certification is an independent technical evaluation, for the purpose of
accreditation, that uses security requirements as the criteria for the
evaluation. An accreditation is an authorization granted by
a management official to operate the system based on the evaluation of the
security controls. |
|
Senior
Management Dashboard Review |
A
review attended by senior executives, contractors, program directors, and
project managers to ensure program directors and project managers are held
accountable for the project status (e.g., risk, cost, schedule). Emphasis is placed only on problem areas or notable status changes. |
|
Steady State |
Any information
technology investment that is fully operational. |
|
System
Acceptability Testing |
The process of
testing a system or program to ensure it meets the original objectives
outlined by the user in the requirement analysis document. |
|
Task Order |
An order for
services placed against an established contract or with Federal Government
sources. |
|
Treasury
Information Processing Support Services-3 |
Contracts, awarded
in 2006, that provide a broad range of information technology-related
services. |
|
Web
EFDS |
The EFDS
development effort allowing users to access the EFDS via the IRS Intranet. |
|
Web Portal |
An Internet site
or service that functions as a major starting site for users to connect to a
broad array of resources and services, such as email, forums, research tools,
online shopping malls, etc. |
|
Web Request
Tracking System |
A web-based application that allows IRS personnel to
prepare, approve, fund, and track requests for the delivery of goods and
services. It also allows for
electronic acceptance of items delivered and provides an electronic interface
with the automated financial system for payment processing. |
|
Work Breakdown
Structure |
A project schedule
used to manage the tasks, task relationships, and resources needed to meet
project goals. |
Appendix IX
Management’s Response to the Draft Report
The response was
removed due to its size. To see the
response, please go to the Adobe PDF version of the report on the TIGTA Public
Web Page.
[1] See Appendix VIII for a Glossary of Terms.
[2] The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Reference Number 2006-20-108, dated August 9, 2006).
[3] See Appendix VIII for a Glossary of Terms.
[4] The Electronic Fraud Detection System Redesign Failure Resulted in Fraudulent Returns and Refunds Not Being Identified (Reference Number 2006-20-108, dated August 9, 2006).
[5] Pub. L. No. 104-106, 110 Stat. 642 (codified in
scattered sections of 5 U.S.C., 5 U.S.C. app., 10 U.S.C., 15 U.S.C., 16 U.S.C.,
18 U.S.C., 22 U.S.C., 28 U.S.C., 29 U.S.C., 31 U.S.C., 38 U.S.C., 40 U.S.C., 41
U.S.C., 42 U.S.C., 44 U.S.C., 49 U.S.C., 50 U.S.C.).
[6] On October 16, 2006, the MITS Enterprise Governance Committee approved the reconfiguration of the Compliance ESC into the Reporting Compliance and the Filing and Payment Compliance ESCs. On November 15, 2006, the MITS Enterprise Governance Committee approved keeping the EFDS in the Reporting Compliance ESC until the filing season is complete, then the EFDS will be moved to the Criminal Investigation ESC.
[7] 48 C.F.R. ch. 1 (2005).
[8] See Appendix VIII for a Glossary of Terms.
[9] The EFDS was placed in the Compliance domain in the new Applications Development organization.