TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

The Tax Exempt Determination System Release 1 Delivered Only a Small Portion of the Expected Benefits and Significantly Exceeded Cost Estimates

September 26, 2006

Reference Number: 2006-10-174

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Phone Number | 202-927-7037

Email Address | Bonnie.Heald@tigta.treas.gov

Web Site | http://www.tigta.gov

September 26, 2006

MEMORANDUM FOR COMMISSIONER, TAX EXEMPT AND GOVERNMENT ENTITIES

DIVISION

FROM: Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

SUBJECT: Final Audit Report – The Tax Exempt Determination System Release 1 Delivered Only a Small Portion of the Expected Benefits and Significantly Exceeded Cost Estimates (Audit # 200610010)

This report presents the results of our review to determine whether implementation of the Tax Exempt Determination System (TEDS) Release 1 improved the Tax Exempt and Government Entities (TE/GE) Division’s processing of Short Form Application for Determination for Employee Benefit Plan (Form 5307) and provided electronic Employee Plans (EP) determination information to other TE/GE Division programs.

Implementation of TEDS Release 1 was expected to provide significant benefits in the receipt, handling, and processing of Forms 5307 to the EP function and other TE/GE Division programs that need access to Form 5307 information. The enhancements were supposed to assist TE/GE Division management with a large influx of determination applications received as a result of a remedial amendment period.[1]

Impact on the Taxpayer

TEDS Release 1 was designed to reduce the processing time for Form 5307 determination applications and improve service to customers. However, the automation of closing processes for these applications and the automation of processes for issuing determination letters were not operating as envisioned. As a result, the new System did not significantly improve the processing of Forms 5307. This is important because the time to process all types of determination applications increased from 208 calendar days to 324 calendar days between Fiscal Years 2004 and 2005, the period of time in which the new System was operational.

Synopsis

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Division’s processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned.

Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the System even if it had worked effectively. As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications. Also, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

Delays in the delivery of TEDS Release 1, as well as the System’s failure to deliver many of the projected benefits, severely limited its impact on the processing of Form 5307 determination applications.

In addition to the failure of TEDS Release 1 to deliver many of the projected benefits, investments associated with the development of TEDS Release 1 were not appropriately tracked, which prevented TE/GE Division senior management from receiving the information needed to effectively evaluate their investment in the TEDS. TEDS Release 1 was developed from December 2001[2] to March 2004. The TE/GE Division used a business case model for estimating the costs and benefits to justify the need for the project and to provide a basis for estimating the cost associated with the development of the TEDS. The actual cost to develop, implement, and maintain TEDS Release 1 (through February 17, 2006) was over $16.9 million, which was approximately $2.3 million higher than that estimated in August 2003. However, the August 7, 2003, TEDS Business Case was not appropriately updated to reflect the severely curtailed capabilities of the System, the 16 percent increase in cost, and the 5-month delay in delivery of the System. Thus, TE/GE Division management did not use the Business Case to fully evaluate investment decisions. This is especially critical because TEDS Release 2 is under development by the TE/GE Division, where the Director, Exempt Organizations function, is responsible for overseeing its implementation.

Recommendations

We recommended the Director, Business Systems Planning, ensure tasks needed to achieve the system functionality for future releases of the TEDS are clearly established to provide greater assurance that expected benefits will be realized. We recommended the Director, Exempt Organizations, implement processes to ensure the TEDS Business Case is revised when changes to the project’s scope occur, increases in the project’s target completion date are extended by more than 10 percent, or project costs are increased by more than 10 percent, so the most current information is available to make informed investment decisions; monitor the progress of the System’s development; and evaluate whether business benefits are realized. We also recommended the Director, Exempt Organizations, fully evaluate investment decisions by adopting a business case model that includes processes for tracking actual costs of processing the determination applications to compare with the estimated costs.

Response

TE/GE Division management agreed with the findings, recommendations, and measurable benefits on tax administration included in the report. Specifically, TE/GE Division management has taken corrective actions to ensure the criteria for automating case closure processes are as complete as possible for the available data, to increase the flexibility of the criteria governing automated case closure and automated letter generation systems in TEDS Release 2, and to ensure processes are in place to identify all requirements. TE/GE Division management has implemented corrective actions to ensure information needed to make investment decisions, monitor the progress of the System’s development, and evaluate whether business benefits are realized is provided to TE/GE Division managers responsible for making decisions regarding TEDS Release 2 investments. Further, TE/GE Division management has taken action to ensure actual costs are tracked and compared against estimated costs. Management’s complete response to the draft report is included as Appendix V.

Copies of this report are also being sent to the Internal Revenue Service managers affected by the report recommendations. Please contact me at (202) 622-6510 if you have questions or Nancy A. Nakamura, Acting Assistant Inspector General for Audit (Headquarters Operations and Exempt Organizations Programs), at (202) 622-8500.

Table of Contents

Background

Results of Review

Critical Portions of the Tax Exempt Determination System Release 1 Did Not Operate As Expected, Which Prevented Expected Benefits From Being Realized

Costs and Benefits for Developing the Tax Exempt Determination System Release 1 Were Not Appropriately Tracked and Monitored

Recommendation 1:

Recommendations 2 and 3:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Outcome Measure

Appendix V – Management’s Response to the Draft Report

Abbreviations

CAS	Customer Account Services
EDS	Employee Plans and Exempt Organizations Determination System
EP	Employee Plans
FY	Fiscal Year
IRS	Internal Revenue Service
TEDS	Tax Exempt Determination System
TE/GE	Tax Exempt and Government Entities

Background

The Tax Exempt and Government Entities (TE/GE) Division uses the Employee Plans and Exempt Organizations Determination System (EDS) to process and control Employee Plans (EP) and Exempt Organizations determination letter[3] applications. For EP customers, the determination letter process is a key interaction with the Internal Revenue Service (IRS) because a favorable determination letter gives the employer or plan sponsor the assurance a pension plan is qualified for favorable tax treatment, as long as the plan is operated under the terms on which the determination letter was issued.

However, TE/GE Division management identified shortcomings with the EDS and the determination letter process, which affected their ability to achieve their mission and strategic performance goals. For example, TE/GE Division management determined their performance was constrained by outdated technology that lacked the capacity to handle the required workload, fulfill statutory IRS responsibilities under the Internal Revenue Code, and meet expectations of the EP customers. These shortcomings included:

· Average time to work a case is too long.

· Cycle time^[4] to respond to simple inquiries and status changes is too long.

· Level of accuracy in determinations is unacceptably low.

· The EDS can not meet expected peak volume demand.

· Significant manual processes, paper handling, and tracking are inefficient.

· Nonautomation of case grading, classification,^[5] or assignment is inefficient.

· Postal costs are excessive because of an inefficient practice of mailing case files.

To address the shortcomings, the TE/GE Division redesigned its business processes in a Business Concept of Operations[6] document dated November 14, 2001. The concept included defining new business processes to be used for processing EP and Exempt Organizations determination letter applications. TE/GE Division management decided to invest in building a new system to replace the EDS and enable the IRS to meet current and future determination letter processing needs of both EP and Exempt Organizations customers.

In December 2001, the TE/GE Division Investment Executive Steering Committee[7] approved a multiple release strategy for the new system, the Tax Exempt Determination System (TEDS). Overall, the TEDS was expected to reduce cycle time, processing time, and processing costs, while implementing process improvements that were to enhance the quality of determinations and the quality of service delivered to customers. TEDS Release 1 was to provide the technical infrastructure for all future releases of the TEDS and would redesign the process for one of the simpler EP determination request forms, Short Form Application for Determination for Employee Benefit Plan (Form 5307). TEDS Release 1 would process the Forms 5307 from receipt by the IRS to issuance of a determination letter or acknowledgment notice. Specifically, implementation of TEDS Release 1 was expected to provide the following system enhancements:

· Control the receipt and handling of Forms 5307.

· Convert hard-copy Forms 5307 and other submitted documents to electronic versions that can be accessed in the TEDS to perform determination processing.

· Automate case grading and classification of Forms 5307 for case assignment.

· Automate case closure for Forms 5307 through the use of business rules.^[8]

· Automate the issuance of determination letters for Forms 5307.

· Generate management information reports for monitoring the processing of determination applications.

Due to several tax law changes, the TE/GE Division anticipated receiving a significant increase in the number of EP determination applications associated with a remedial amendment period.[9] TE/GE Division management had set the goal of having TEDS Release 1 operational in Fiscal Year (FY) 2004 to handle the additional EP determination applications.

A prior Treasury Inspector General for Tax Administration audit[10] found that Business Systems Planning program management’s planned use of a modified Enterprise Life Cycle and added project management techniques[11] for the development of the TEDS complied with IRS guidelines for the initial stages of systems development. However, TEDS project management did not implement these techniques as planned. This resulted in the following inefficiencies:

· Requirements gathering led to delays in project development.

· Performance monitoring did not ensure project objectives were completed on schedule or within budget.

· Risk management^[12] did not identify potential problems that could affect completion of TEDS development.

In the prior audit, we recommended TEDS project management identify and explain cost, benefit, schedule, and contractor performance variances in the next Business Case[13] update. The Business Case update should include a report of the actual business benefits realized as compared to projected benefits. We also recommended TEDS project management fully implement the planned risk management process. TE/GE Division management agreed with the recommendations and implemented actions to address the problems identified by the audit.

This review was performed at the IRS National Headquarters Business Systems Planning and EP Program offices in Washington, D.C.; the Cincinnati Submission Processing Site[14] in Covington, Kentucky; the EP Rulings and Agreements Determinations program, the EP Determinations Quality Assurance program, and the TE/GE Division Customer Account Services (CAS) call site in Cincinnati, Ohio; and the EP Examinations function in Baltimore, Maryland. The audit work was conducted during the period January through April 2006 in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

Critical Portions of the Tax Exempt Determination System Release 1 Did Not Operate As Expected, Which Prevented Expected Benefits From Being Realized

The delivery of TEDS Release 1 on March 16, 2004, did not significantly improve the TE/GE Division’s processing of Form 5307 determination applications or assist in providing electronic EP determination information to other TE/GE Division programs, as originally envisioned. Because TEDS Release 1 was implemented a couple of months after the remedial amendment period and the EP function had already started to receive a large number of applications, the EP function would not have realized the full benefits of the system even if it had worked effectively. As a result, the delay in the delivery of the System severely limited the impact TEDS Release 1 had on the processing of Form 5307 determination applications. In addition, while electronic Form 5307 information was available to TEDS users, its impact was limited because the use of electronic images had not been fully incorporated into TE/GE Division operating procedures.

Enhancements expected from TEDS Release 1 did not significantly improve the processing of EP Form 5307 determination applications

For each of the planned system capabilities for TEDS Release 1, we determined whether the System was operating as intended as well as the reasons why any system capabilities were not operating. The TE/GE Division successfully ensured TEDS Release 1 system capabilities for controlling and converting hard-copy Form 5307 determination applications into electronic case files were implemented. It also successfully automated some of the case assignment processes by systemically grading and classifying Form 5307 determination applications. While no data existed at the time of our audit to assess the impact of these enhancements, the implementation of these additional capabilities improved Form 5307 processing by reducing the number of human resources needed to perform these functions.

However, key portions of the TEDS Release 1 functionality did not deliver the expected benefits projected in the August 7, 2003, TEDS Business Case, which greatly limited the impact the System had on the processing of determination applications. As described in more detail below, the automation of closing processes for Form 5307 determination applications that met selected criteria as well as the automation of processes for issuing determination letters were not operating as envisioned. In addition, management information reports that were to be used to monitor the determination applications after IRS receipt and initial processing were not operating as expected and were not being used.

The TE/GE Division had planned on systemically closing a portion of Form 5307 determination applications by programming selected processing procedures or business rules into the case evaluation processes. Determination applications that met specific criteria were supposed to be automatically closed with a determination letter automatically issued to the customer with no human intervention. However, during pilot testing from July 28, 2003, through March 16, 2004, the EP function determined many of the cases proposed for automatic closure were at a significant risk of being inappropriately closed because sufficient evidence was not available to ensure the plans met all of the criteria for tax-exempt status.

The automated case closure feature was designed to expedite the processing for determination applications and to eliminate the need for EP personnel to prepare and send determination letters to customers. Combined, these enhancements were expected to yield monetary benefits resulting from “productivity increases that translate into reinvestment opportunities for the IRS.” For example, the TEDS Business Case from August 2003 showed the TE/GE Division had estimated achieving over $788,000 in monetary benefits for FYs 2004 and 2005 resulting from reduced labor costs[15] for processing determination applications.

Several issues prevented the EP function from implementing the automated closure feature:

• The criteria (or business rules) for automated closure did not account for instances where Forms 5307 included additional issues or amendments, which is a common practice even for pre-approved plans.
• The criteria that govern the automated case closure system capability are custom coded and do not provide the ability for the user to update or change the criteria without the assistance of a programmer for common differences such as legislative changes.
• The pressure of a tight time constraint for TE/GE Division project management to implement the TEDS Release 1 prior to the influx of Forms 5307 due to the remedial amendment period.
• The research process during the development of the TEDS was not effective to identify all of the requirements that were needed to program the TEDS.

At the time of our review, Form 5307 determination applications that originally met the criteria for automated closure were being classified as proposed auto-closures. The proposed auto-closures required a review by revenue agents to determine whether additional actions were needed to approve the application and issue a favorable determination letter. While there is a benefit to using the proposed auto-closure feature, the estimated labor savings of over $788,000 did not materialize because the automated case closure and automated letter generation features were not operating as envisioned.

EP function management and TEDS project management advised us the automated case closure feature described in the March 15, 2002, TEDS Business Concept of Operations will not be available in future releases of TEDS applications unless significant changes are made to either the criteria for applying business rules or the Form 5307. The only EP Form that is expected to be closed automatically is the Notice of Merger, Consolidation or Transfer of Plan Liabilities (Form 5310-A), which is a merger notice that is not required to be reviewed because no determination or acknowledgement letter is required.

TEDS Release 1 was also intended to yield nonmonetary benefits, including the following:

· Reduced cycle time for processing EP determination applications.

· Improved customer satisfaction as measured by the customer satisfaction survey score.

· Improved quality of determination letters.

However, as shown in Figure 1, cycle time for processing applications increased despite the piloting and implementing of the TEDS in FY 2004.

Figure 1: Increased Cycle Time for EP Determination Applications

Figure 1 was removed due to its size. To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The TE/GE Business Performance Reviews showed the TE/GE Division attributed the increased cycle time to a higher than anticipated volume of determination applications received from adopters of pre-approved plans and the extension of time in which plan sponsors had to submit their determination applications. In FY 2004, the EP function estimated approximately 42,000 determination applications would be received; the actual volume was 70,610 receipts.

However, the increases in cycle time could have been significantly mitigated with successful implementation of the automated case closure feature described by the August 7, 2003, TEDS Business Case. This Business Case showed “a significant variable impacting the benefits associated with TEDS Release 1 is the percentage of cases that will be closed automatically with TEDS.” The TE/GE Division expected approximately 32 percent of the Forms 5307 would be closed automatically without human intervention. Because the automated case closure feature was not operating as envisioned by the Business Case, the Forms 5307 that would have been auto-closed were identified for proposed auto-closures, which required a review by revenue agents when the System was placed into production in FY 2004. This contributed to increasing cycle time from 176 calendar days to 324 calendar days between FYs 2003 and 2005.

During this same time period, TEDS Release 1 did not contribute towards the performance measures for improving customer satisfaction and improving the quality of determination letters. Figure 2 presents the decline in these two performance measures.

Figure 2: Decreased Customer Satisfaction and Letter Quality

Figure 2 was removed due to its size. To see Figure 2, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The TE/GE Business Performance Reviews attributed a decrease in the level of satisfied customers from 69 percent in FY 2003 to 61 percent in FY 2005 primarily to the large increase in workload resulting from the determination applications received from adopters of pre-approved plans and the corresponding delay in issuing determination letters. During this same period, the quality of determination letters decreased from 80 percent to 76 percent. The TE/GE Business Performance Reviews for the EP function attribute the decrease in quality in FYs 2004 and 2005 to use of a more rigid definition of the timeliness standards, which no longer allowed for reviewer judgment or mitigating circumstances. This was coupled with a large build-up of inventory for determination applications.

The automated letter generation system capability did not deliver the expected benefits described in the August 7, 2003, TEDS Business Case because TEDS Release 1 lacked the functionality to:

• Create correspondence templates.
• Provide authorized users the ability to change the letter template as well as the effective dates associated with each change.

TEDS Release 1 also did not include the capability to monitor the processing of the Form 5307 determination applications from receipt by the IRS to issuance of a determination letter or acknowledgement notice. TE/GE Division management advised us the TEDS report feature for the initial receipt and processing of the Form 5307 determination applications was being used. However, the report feature was not being used to control the inventory of determination applications when they are ready for assignment to a revenue agent. TE/GE Division management advised us the report feature used to control the inventory of determination applications was not critical to TEDS Release 1 because the inventory was controlled on the EDS. As a result, the TE/GE Division has been relying exclusively on the EDS for inventory control reports and uses two “Balancing Reports”[16] to reconcile the two Systems. TE/GE Division management determined the combination of the Balancing Reports and the EDS inventory reports was sufficient to control inventory and that incurring additional costs to create the TEDS Release 1 reports would not be practical because the EDS was the inventory system for processing and controlling the Form 5307 determination applications. However, the EDS was the System TE/GE Division management stated was inefficient in the March 15, 2002, TEDS Business Concept of Operations.

Project management is a carefully planned and organized effort to accomplish a specific one-time effort (e.g., implementation of a new computer system). Project management includes defining project goals and objectives, specifying tasks on how the goals will be achieved and what resources are needed, and associating budgets and timeliness for completion.

Because the effort to implement TEDS Release 1 was not sufficiently planned and monitored to ensure tasks would achieve project goals, there was less assurance that system capabilities would operate as planned, expected benefits would be delivered, and the System would be implemented within estimated project cost.

TEDS Release 1 did not benefit other TE/GE Division operations not directly associated with the processing of Form 5307 determination applications

System documentation showed several TE/GE Division operations not associated with the processing of EP determination applications would also benefit from the implementation of TEDS Release 1. The August 7, 2003, TEDS Business Case identified the TE/GE CAS function as one operation that would be using TEDS functionality, and the March 15, 2002, TEDS Business Concept of Operations identified the EP