The Internal Revenue Service Is Successfully Taking Steps to Transition Modernization Activities From the PRIME Contractor; However, Difficult Challenges Remain

TREASURY INSPECTOR GENERAL FOR TAX ADMINISTRATION

The Internal Revenue Service Is Successfully Taking Steps to Transition Modernization Activities From the PRIME Contractor; However, Difficult Challenges Remain

October 24, 2006

Reference Number: 2007-20-003

This report has cleared the Treasury Inspector General for Tax Administration disclosure review process and information determined to be restricted from public release has been redacted from this document.

Phone Number | 202-927-7037

Email Address | Bonnie.Heald@tigta.treas.gov

Web Site | http://www.tigta.gov

October 24, 2006

MEMORANDUM FOR CHIEF INFORMATION OFFICER

FROM: Michael R. Phillips /s/ Michael R. Phillips

Deputy Inspector General for Audit

SUBJECT: Final Audit Report – The Internal Revenue Service Is Successfully Taking Steps to Transition Modernization Activities From the PRIME Contractor; However, Difficult Challenges Remain (Audit # 200620037)

This report presents the results of our review to determine whether the Internal Revenue Service (IRS) was sufficiently planning, obtaining resources, and executing modernization activities to transition activities from the PRIME contractor [1] for the Business Systems Modernization program.

In January 2005, due to budget reductions and concerns about the adequacy of the PRIME contractor’s performance, the IRS began transitioning numerous activities from the PRIME contractor and taking over the primary role as the systems integrator for all projects. The systems integrator role is a significant operational change for the IRS, requiring new procedures, personnel, and offices.

Impact on the Taxpayer

The IRS has successfully taken the first steps to transition modernization activities from the PRIME contractor and is taking additional actions to build on these initial steps. However, the IRS has not developed performance measures for all of the transitioned activities that can be measured. As a result, it does not have an objective method of determining if transitioned activities are being accomplished effectively and taxpayer funds are being used efficiently.

Synopsis

The IRS has effectively assumed 2 (14 percent) of 14 modernization activities. For example, the IRS filled 8 of the 10 vacant positions in the Modernization Program Engineering Office and eliminated all contractor support. In addition, the IRS simplified the Enterprise Life Cycle, which had been previously maintained by the PRIME contractor.

For the remaining 12 activities, the IRS has taken good first steps and established an initial capability. For example, the IRS developed the Enterprise Transition Strategy, which documents the overall strategy for realizing the vision and goals of the IRS. In addition, the IRS completed its first update to the Modernization Vision and Strategy since 2002.

Until additional actions are taken and the IRS has more time to consistently use new processes and strengthen its new capabilities, it is difficult to judge the eventual success of the IRS’ actions. Due to the significance of the transitioned activities, it will be important to measure how effectively activities are being accomplished as improvements are made.

The IRS has not developed performance measures for all of the transitioned modernization activities that can be measured. Most of management’s focus has been on hiring staff to perform transitioned activities and getting processes and procedures developed and implemented. However, we believe it is now time to begin measuring progress.

Recommendation

To ensure the IRS is successful in performing modernization activities that have been transitioned from the PRIME contractor, we recommended the Chief Information Officer ensure performance measures are developed for each IRS modernization activity that can be measured.

Response

The Chief Information Officer agreed with our recommendation and provided information regarding transition areas where the IRS has made progress since the conclusion of our audit. For example, the IRS implemented new change management procedures and developed a suite of templates to support requirements development and management. To ensure the IRS is successful in performing modernization activities that have been transitioned from the PRIME contractor, the IRS will develop a plan to identify potential performance metrics for each measurable transitioned modernization activity. Management’s complete response to the draft report is included as Appendix VI.

Copies of this report are also being sent to the IRS managers affected by the report recommendation. Please contact me at (202) 622-6510 if you have questions or Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs), at (202) 622-8510.

Table of Contents

Background

Results of Review

The Internal Revenue Service Has Successfully Taken the First Steps to Perform Transitioned Modernization Activities

The Internal Revenue Service Is Taking Additional Actions to Improve Its Ability to Perform Transitioned Modernization Activities

The Internal Revenue Service Should Measure How Effectively Transitioned Modernization Activities Are Being Performed

Recommendation 1:

Appendices

Appendix I – Detailed Objective, Scope, and Methodology

Appendix II – Major Contributors to This Report

Appendix III – Report Distribution List

Appendix IV – Glossary of Terms

Appendix V – Independent Assessment of the Status of Transitioned Modernization Activities

Appendix VI – Management’s Response to the Draft Report

Abbreviations

ELC	Enterprise Life Cycle
GAO	Government Accountability Office
HPI	Highest Priority Initiatives
IRS	Internal Revenue Service
TIGTA	Treasury Inspector General for Tax Administration

Background

In 1998, the Internal Revenue Service (IRS) initiated the Business Systems Modernization program to modernize its outdated computer systems and associated processes. To facilitate the success of its modernization efforts, the IRS hired the Computer Sciences Corporation as the PRIME contractor.[2]

The IRS initially partnered with the PRIME contractor to achieve its overall goal of modernizing business processes and systems to improve service quality, timeliness, reliability, and privacy. The IRS relied on the PRIME contractor to act as a systems integrator to find and manage the best expertise and technical resources to achieve the IRS’ organizational goals. In January 2005, due to budget reductions and concerns about the adequacy of the PRIME contractor’s performance, the IRS began transitioning numerous activities from the PRIME contractor and taking over the primary role as the systems integrator for all projects. The systems integrator role is a significant operational change for the IRS, requiring new procedures, personnel, and offices.

IRS officials used the Highest Priority Initiatives (HPI), a program of continual process improvement with executive-level oversight, to transition high-priority modernization activities from the PRIME contractor to the IRS. Under the HPI process, IRS executives identify the highest priority programs every 6 months and appoint an IRS executive and senior manager to oversee the execution of each HPI. Figure 1 provides a listing of the HPI activities we reviewed as part of this audit.

Figure 1: HPI Activities

Activity	Brief Description
Human Capital	Design a strategy to recruit and hire staff with needed skills for modernization.
Program Management	Design a training program to enhance the program management skills of the IRS staff.
Systems Engineering	Hire 80 percent of the engineering staff needed in the Modernization Program Engineering Office.
Standard Contracting Language	Develop language for all contractors to follow when competing for IRS modernization contracts.
Quality Assurance	Develop a process for implementing quality assurance in the IRS modernization program.
Integrated Schedule and Baseline Management	Develop a plan to implement integrated schedule and baseline management.
Cost and Schedule Estimation	Develop cost and schedule estimation processes and identify gaps between the IRS’ existing capabilities and the Software Engineering Institute’s estimation requirements.
Change Management	Revise the change request processes.
Extensible Markup Language	Establish a working group and issue draft Naming and Design Rules.
Business Rules	Establish a limited initial operational capability for business rules including having the appropriate skills and processes in place.
Requirements Management	Develop a Concept of Operations, establish an office, and begin pilot operations.
Enterprise Transition Strategy	Develop volume 1 of the Enterprise Transition Strategy.
Modernization Vision and Strategy	Develop a high-level modernization vision and strategy document.
Enterprise Life Cycle (ELC)	Revise the ELC Guide to include paths common to IRS modernization projects.

Source: Treasury Inspector General for Tax Administration (TIGTA) summary of HPI status reports.

The IRS also changed its requirements for acquiring systems and the contractor’s requirements for developing systems. Prior to the transition of modernization activities, the IRS required the PRIME contractor to maintain a Capability Maturity Model^® Software Acquisition Level 3 rating and a Capability Maturity Model^® Software Development Level 3 rating. However, under the new structure, IRS officials stated they only require contractors to maintain a Capability Maturity Model^® Software Development Level 2 rating and are using the HPI process instead of the Capability Maturity Model^® to ensure systems are acquired properly.

IRS officials informed us the HPIs included activities the PRIME contractor was not performing well, but did not include all of the activities the IRS would take over. Since the activities included in the HPIs were viewed by the IRS as the high-level activities transitioning to the IRS, our review focused on the 14 HPIs identified jointly by the IRS and the audit team.[3] Our review did not include any of the activities which were not controlled through the HPI process.

This audit was performed at the Modernization and Information Technology Services organization’s office in New Carrollton, Maryland, during the period October 2005 through June 2006. The audit was conducted in accordance with Government Auditing Standards. Detailed information on our audit objective, scope, and methodology is presented in Appendix I. Major contributors to the report are listed in Appendix II.

Results of Review

The IRS has successfully taken the first steps to transition activities away from the PRIME contractor.

The IRS has successfully taken the first steps to transition modernization activities away from the PRIME contractor and is taking additional actions to build on these initial steps. Until these additional actions are taken and the IRS has more time to consistently use new processes and strengthen its new capabilities, it is difficult to judge the eventual success of the IRS’ actions. Due to the significance of the transitioned activities, it will be important to measure how effectively activities are being accomplished as improvements are made.

The Internal Revenue Service Has Successfully Taken the First Steps to Perform Transitioned Modernization Activities

As part of the HPI process, IRS officials created 6-month individual plans with goals, known as closure criteria, for each transitioned activity. We reviewed the individual plans and determined the IRS established appropriate closure criteria. We also reviewed the closed HPIs and determined the IRS satisfied the closure criteria.

In January 2005, the IRS learned it would be receiving a larger than expected budget cut. As a result, it had to assume modernization activities quickly because it could no longer afford to pay for all of the PRIME contractor’s services. Therefore, the plans and actions we reviewed were viewed as the steps needed to establish an initial capability. Additional steps to build on this capability are now underway. We determined the IRS is effectively performing 2 (14 percent) of the 14 modernization activities. For the remaining 12 activities, the IRS has taken good first steps and established an initial capability; however, ongoing actions need to be completed before we can determine whether the IRS will be successful in performing transitioned activities. Figure 2 graphically presents our opinion of the IRS’ success in performing modernization activities.

Figure 2: TIGTA’s Opinion on the IRS’ Success in Performing Modernization Activities

Figure 1 was removed due to its size. To see Figure 1, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

The IRS effectively performed the following two transitioned modernization activities.

Systems Engineering – The IRS accomplished its goal of filling 8 of the 10 vacant positions in the Modernization Program Engineering Office and eliminated all contractor support. Project teams that have recently used the services of this Office provided positive feedback.

ELC – The IRS simplified the ELC, which had been previously maintained by the PRIME contractor, to include only the information commonly needed by IRS modernization projects. In addition, the IRS ELC staff has begun assisting project teams. Project teams that have recently used the services of the ELC staff provided positive feedback.

For the remaining 12 modernization activities, the IRS has successfully taken first steps and is taking additional actions to build on these initial steps.

The Internal Revenue Service Is Taking Additional Actions to Improve Its Ability to Perform Transitioned Modernization Activities

Whether the IRS will be successful in performing transitioned modernization activities is difficult to judge at this time due to the need for the IRS to consistently use new processes and take additional actions to improve on the initial capabilities it has established.

The IRS has initiated additional actions to improve its capabilities

During our review, the IRS initiated additional actions to improve its capability to perform transitioned modernization activities. In some cases, the enhancements go beyond the activities once performed by the PRIME contractor. The following are several examples of transitioned modernization activities and ongoing actions the IRS is taking to be successful in performing the activities.[4]

Integrated schedule and baseline management

An Integrated Master Schedule provides a schedule for project development and integration of all modernization projects. To develop an Integrated Master Schedule, the IRS developed an implementation plan. As part of a follow-up HPI, the IRS subsequently developed templates for an integrated work breakdown structure and schedule. To fully implement this capability, project teams will need to consistently use the templates and the IRS will need to begin managing dependencies between projects using the Schedule.

Cost and schedule estimation

The IRS analyzed its cost and schedule estimation capability and identified needs it has to address to achieve estimation best practices. Due to resource constraints, the IRS’ initial work has been limited to drafting a Cost and Schedule Estimation Concept of Operations and providing independent cost estimates to modernization projects.

We interviewed three project teams that had used the independent cost estimates. Two project teams stated the independent estimates assisted them in obtaining a lower price for contracted services. The other project team stated it did not find the independent cost estimate to be as useful.

Management Actions: We determined the draft Cost and Schedule Estimation Concept of Operations did not provide for obtaining feedback from project teams that had used the services of the cost and schedule estimation team. Upon discussing this with IRS officials, the IRS revised its process to obtain feedback from project teams.

Standard contracting language

The IRS developed standard contracting language to ensure it receives the same information from all modernization contractors. The contracting language has been used in a task order for one project release. As we were concluding our audit, the IRS was ensuring other contracting officials had access to the language. Until the IRS is consistently using the new language, it is difficult to judge whether the IRS will be successful in ensuring all modernization contractors comply with a standard set of requirements.

The Enterprise Transition Strategy and the Modernization Vision and Strategy

The IRS developed the Enterprise Transition Strategy, which documents the overall strategy for realizing the vision and goals of the IRS. In addition, the IRS completed its first update to the Modernization Vision and Strategy since 2002. As we were completing our audit, the IRS was creating the Enterprise Transition Plan from the Enterprise Transition Strategy and the Modernization Vision and Strategy and developing procedures to ensure the Modernization Vision and Strategy is updated yearly. This is being tracked as part of a follow-up HPI. Until the IRS consistently performs this activity, it is difficult to judge whether the IRS will be successful in updating its Enterprise Transition Plan yearly.

Business rules and requirements management

The Business Rules and Requirements Management Office has been fully staffed. However, the Government Accountability Office (GAO) recently reported modernization projects were not consistently following requirements development and management practices, and the IRS would not have a full suite of requirements processes until March 2007.[5] We interviewed two project teams that have received assistance from the Business Rules and Requirements Management Office. One project team stated the assistance helped explain to the contractor how the business rules should be traced to the system design. The other project team believed the assistance was good although some responsibilities were not clearly defined.

The IRS is taking corrective actions based on the GAO recommendation and it will take time for the IRS to mature its Business Rules and Requirements Management capabilities. Until then, it is difficult to judge whether the IRS will be successful in managing business rules and requirements.

The IRS has made progress in hiring staff with the required skills to perform transitioned modernization activities

In Fiscal Year 2005, the IRS received approval to hire 47 employees[6] to perform transitioned modernization activities. As of March 2006, the IRS had filled 28 (60 percent) of the 47 positions.[7] According to the IRS, it has been focusing on the most critical activities and accomplishing some of the work through contractors until it can complete its hiring.

We reviewed the job announcements and applications submitted for 8 (29 percent) of the 28 employees hired as of March 2006 and determined the new employees have the required key skills and abilities to perform transitioned modernization activities.

The remaining vacancies are in the Systems Integration function. The IRS learned through the applications review and interview process some positions are difficult to fill. As a result, it had to close some vacancy announcements without selecting any of the applicants. The Systems Integration function has hiring plans to fill these remaining positions. In addition, the Deputy Associate Chief Information Officer, Systems Integration, informed us the IRS was expanding its search by advertising in the newspaper to attract nonGovernment applicants.

While we determined the IRS has successfully taken the first steps to transition modernization activities, the IRS now needs to focus on completing additional actions and measuring its success in performing transitioned activities.

The Internal Revenue Service Should Measure How Effectively Transitioned Modernization Activities Are Being Performed

The Software Engineering Institute’s Capability Maturity Model^® Integration is a process improvement approach that provides organizations with the essential elements of effective processes. The Model provides best practices that can be used to guide process improvement across a project, a division, or an entire organization. One of the Capability Maturity Model^® Integration goals is to provide measurement results that can be used to objectively evaluate how effectively standards and procedures are being accomplished.

The IRS has not developed performance measures for all of its transitioned modernization activities that can be measured. For example, the Quality Assurance Office has done a good job of developing measures and is developing additional measures, while the IRS has planned for, but not developed, measures for other activities such as business rules and requirements management.

Most of management’s focus has been on hiring staff to perform transitioned activities and getting processes and procedures developed and implemented. However, we believe it is now time to begin measuring progress. If performance measures are not developed and analyzed, IRS management will not have an objective method of determining if transitioned activities are being performed effectively and taxpayer funds are being used efficiently.

Recommendation

Recommendation 1: To ensure the IRS is successful in performing modernization activities that have been transitioned from the PRIME contractor, the Chief Information Officer should ensure performance measures are developed for each transitioned modernization activity that can be measured. When traditional performance measures are difficult to develop, the IRS can obtain feedback from modernization stakeholders and users to measure customer satisfaction.

If performance measures are not developed and analyzed, taxpayer funds could be used inefficiently.

Management’s Response: The Chief Information Officer agreed with our recommendation and will develop a plan to identify potential performance metrics for each measurable transitioned modernization activity.

Appendix I

Detailed Objective, Scope, and Methodology

The overall audit objective of this review was to determine whether the IRS was sufficiently planning, obtaining resources, and executing modernization activities to transition activities from the PRIME contractor[8] for the Business Systems Modernization program. To accomplish this objective, we:

I. Determined the specific activities being transitioned from the PRIME contractor to the IRS by discussing systems integration program areas with IRS executives to ensure all transition program areas were identified.

II. Determined whether the IRS established and met appropriate closure criteria for the 14 HPIs identified jointly by the IRS and the audit team.[9]

A. Evaluated whether the closure criteria were appropriate for the HPIs.

B. Determined whether the closure criteria for the closed HPIs were satisfied prior to closure of the HPIs.

III. Determined whether the IRS hired appropriate staff to perform modernization activities.

A. Analyzed staffing charts and identified employees hired to fill new positions.

B. Selected and reviewed a judgmental sample of 8 (29 percent) of 28 employees hired as of March 2006 to fill new positions. We used judgmental sampling because we did not plan to project our sample results.

1. Obtained the job announcements and applications submitted by the eight employees.

2. Determined whether the employees were qualified for the position by reviewing the key skills and abilities contained in the eight employee applications.

C. Discussed remaining staffing vacancies with IRS officials to determine hiring plans.

IV. Determined whether there was evidence indicating the transition of activities from the PRIME contractor to the IRS was occurring, or had occurred and was effective.

A. Interviewed IRS executives to obtain their views on how the transition of systems integrator roles was progressing and methods used to monitor the transition.

B. Obtained evidence showing the systems integrator program areas had been implemented.

Appendix II

Major Contributors to This Report

Margaret E. Begg, Assistant Inspector General for Audit (Information Systems Programs)

Gary Hinkle, Director

Troy Paterson, Audit Manager

Wallace Sims, Lead Auditor

Tina Wong, Senior Auditor

Charlene Elliston, Auditor

Appendix III

Report Distribution List

Commissioner C

Office of the Commissioner – Attn: Chief of Staff C

Deputy Commissioner for Operations Support OS

Associate Chief Information Officer, Applications Development OS:CIO:B

Associate Chief Information Officer, Enterprise Services OS:CIO:ES

Deputy Associate Chief Information Officer, Applications Development OS:CIO:AD

Director, Procurement OS:A:P

Director, Stakeholder Management OS:CIO:SM

Deputy Associate Chief Information Officer, Business Integration OS:CIO:ES:BI

Deputy Associate Chief Information Officer, Systems Integration OS:CIO:ES:SI

Chief Counsel CC

National Taxpayer Advocate TA

Director, Office of Legislative Affairs CL:LA

Director, Office of Program Evaluation and Risk Analysis RAS:O

Office of Internal Control OS:CFO:CPIC:IC

Audit Liaisons:

Associate Chief Information Officer, Applications Development OS:CIO:B

Director, Procurement OS:A:P

Director, Program Oversight OS:CIO:SM:PO

Appendix IV

Glossary of Terms

Term	Definition
Baseline	A baseline consists of a specified set of documents, software, and other items defined as final (or point-in-time) products for a project. A baseline establishes a predefined point from which to evaluate project progress.
Business Rule	A business rule is a statement that defines or constrains some aspect of the business.
Capability Maturity Model^®	The Capability Maturity Model^® is a structured process that helps organizations improve their abilities to consistently and predictably acquire and develop high-quality information systems. Organizations that have implemented Capability Maturity Model^® processes have seen dramatic improvements in their abilities to meet planned time periods, reduce errors, and increase value on dollars invested.
Change Request	A change request is the medium for requesting approval to change a baselined product or other controlled item.
Concept of Operations	The Concept of Operations is a framework that includes a defined vision, strategic goals, operational themes, and program capabilities. It identifies key organizational concepts required to achieve the organization’s vision.
Earned Value Management	Earned value management involves measuring actual cost and work accomplished against the budgeted cost and planned work scheduled. Variances are analyzed for decision making.
Enterprise Life Cycle	The ELC is a structured business systems development method that requires the preparation of specific work products during different phases of the development process.
Extensible Markup Language	Extensible Markup Language is the universal format for structured documents and data on the Internet.
Human Capital	Human capital is defined by the National Academy of Public Administration as the “identification of competencies and skills … needed to realize an organization’s mission and operating goals.” According to the GAO, acquiring and developing staffs whose size and skills meet agency needs is one of the most pervasive challenges now facing the Federal Government.
Information Technology Modernization Vision and Strategy	Information Technology Modernization Vision and Strategy establishes a 5-year plan that drives investment decisions, addresses the priorities around modernizing front-line tax administration and supporting technical capabilities, and leverages existing systems (where possible) and new development (where necessary) to optimize capacity, manage program costs, and deliver business value on a more incremental and frequent basis.
Integrated Master Schedule	The Integrated Master Schedule provides a schedule for project development and integration of all modernization projects.
Milestone	Milestones provide for “go/no-go” decision points in a project and are sometimes associated with funding approval to proceed.
Monte Carlo Analysis	Monte Carlo simulation is a complex mathematical technique that has been used for many years in the space program and other industries. It helps estimate the probability of meeting specified future goals.
PRIME Contractor	The PRIME contractor is the Computer Sciences Corporation, which heads an alliance of leading technology companies brought together to assist with the IRS’ efforts to modernize its computer systems and related information technology.
Release	A release is a specific edition of software.
Software Engineering Institute	The Software Engineering Institute is a Federally Funded Research and Development Center whose purpose is to help others make measured improvements in their software engineering capabilities.
Systems Integrator	The systems integrator entity is responsible for coordinating development projects into the modernized IRS systems.
Task Order	A task order is an order for services planned against an established contract.
Work Breakdown Structure	A work breakdown structure is a deliverable-oriented grouping of project elements that organizes and defines the total scope of the project.

Appendix V

Independent Assessment of the Status of Transitioned Modernization Activities

Although the IRS has always had responsibilities for these activities, the systems integrator role is a significant operational change for the IRS. In the past, the IRS provided oversight of the PRIME contractor’s activities and personnel. In its new role as systems integrator, the IRS assumed more responsibility for developing new procedures, hiring and providing expert personnel, and implementing new offices to lead modernization activities.

Chart 1 provides a listing of the modernization activities included in our review, a brief description of each of the activities, major accomplishments by the IRS, and our opinion as to whether the IRS has been effective in performing the modernization activities or additional actions are underway for the IRS to be successful in performing transitioned modernization activities.

Chart 1: Listing of Transitioned Modernization Activities

Modernization Activity	Brief Description	Major Accomplishments	TIGTA Opinion on Effectiveness
Human Capital	Design a strategy to recruit and hire staff with needed skills for modernization.	The IRS used a model to determine how many additional employees were needed to perform modernization activities, and a recruiting strategy to attract and hire critically skilled individuals to address modernization needs was developed.	Additional Actions Underway – The IRS has begun hiring employees with the needed skills. However, it has not been able to fill all positions and it has not been able to fully implement its recruiting strategy. The IRS has a hiring plan to fill remaining positions and has created a function under the direction of the Associate Chief Information Officer, Applications Development, to implement the recruiting strategy.
Program Management	Design a training program to enhance the program management skills of the IRS staff.	This program area was completed near the end of our audit work; therefore, we reviewed only the plan for this area. According to IRS status reports, the IRS provided training to its Acquisition Project managers, including courses in Earned Value Management and Monte Carlo Analysis.	Additional Actions Underway – The next step for the IRS will be to consistently manage its projects using the skills recently obtained during training sessions.
Systems Engineering	Hire 80 percent of the engineering staff needed in the Modernization Program Engineering Office.	The IRS hired 8 of 10 employees needed to fully staff the Modernization Program Engineering Office.	Effective – The IRS staffed the Modernization Program Engineering Office and eliminated all contractor support. Project teams provided positive feedback on the performance of the new employees.
Standard Contracting Language	Develop language for all contractors to follow when competing for IRS modernization contracts.	The IRS developed templates containing language for all contractors to follow when competing for IRS modernization contracts. The contracting language has been used in a task order for one project release.	Additional Actions Underway – Although one project used the contracting language during our review, the IRS informed us it was preparing to ensure other contracting officials had access to the language and planning to issue instructions requiring Contracting Officers and Contracting Officers Technical Representatives to begin using the language in the templates.
Quality Assurance	Develop a process for implementing quality assurance in the IRS modernization program.	The IRS developed a framework for implementing quality assurance activities in the modernization program. The Quality Assurance staff has developed Quality Assurance plans for three modernization projects. In addition, the Quality Assurance Office has developed performance measures for the projects, and reports measurement results monthly to the Associate Chief Information Officer, Applications Development.	Additional Actions Underway – The Quality Assurance Framework has been developed and performance measures are captured and reported monthly to IRS executive management. While the Quality Assurance Office has substantively assumed its new role, it has plans to develop additional measures and has not used all of its new procedures.
Integrated Schedule and Baseline Management	Develop a plan to implement integrated schedule and baseline management.	The IRS created an Implementation Plan to develop an Integrated Master Schedule. As part of the work to accomplish the Plan, the IRS subsequently developed templates for an integrated work breakdown structure and schedule.	Additional Actions Underway – The IRS developed the integrated work breakdown structure and schedule templates during our review. To fully implement this capability, project teams will need to consistently use the templates and the IRS will need to begin managing dependencies between projects controlled on the Integrated Master Schedule.
Cost and Schedule Estimation	Develop cost and schedule estimation processes and identify gaps between the IRS’ existing capabilities and the Software Engineering Institute’s estimation requirements.	IRS officials stated, due to resource constraints, their initial work has been limited to drafting a Cost and Schedule Estimation Concept of Operations and providing independent cost estimates to modernization projects. In addition, the IRS analyzed its cost and schedule estimation capability and identified needs it has to address in order to achieve estimation best practices.	Additional Actions Underway – The IRS is in the process of finalizing its Cost and Schedule Estimation Concept of Operations and incorporating a process to obtain feedback from projects that use its services. Also, the IRS has identified gaps between its current capabilities and the Software Engineering Institute’s recommended capabilities it has yet to address due to resource constraints.
Change Management	Revise the change request processes.	This program area was completed near the end of our audit work. Therefore, we reviewed only the plan for this area. According to IRS status reports, the IRS completed revisions to the change request system and revised its change processes.	Additional Actions Underway – On June 9, 2006, the IRS revised its Change Request Procedure to reflect the IRS as having the primary responsibility for change request processing. In addition, it made changes to the change request tracking system to match its revised procedure. The IRS’ next step will be to monitor and ensure implementation of its recently revised procedure.
Extensible Markup Language	Establish a working group and issue draft Naming and Design Rules.	The IRS established a working group and issued draft Naming and Design Rules. Several project teams are using Extensible Markup Language products.	Additional Actions Underway – The IRS stated it was implementing a framework, with the Naming and Design Rules being the first part of the framework. The IRS is currently working on common components, the third part of its framework.
Business Rules	Establish a limited initial operational capability for business rules, including having the appropriate skills and processes in place.	The IRS established and staffed the Business Rules office and developed a Concept of Operations.	Additional Actions Underway – We interviewed project personnel from one project that received assistance from the Business Rules staff. Project personnel provided positive feedback. In our opinion, it will take time for the IRS to mature its Business Rules capabilities. The Business Rules staff will need more experience with additional projects to learn lessons and improve processes. Until then, it is difficult to judge whether the IRS has fully implemented its new role of managing business rules.
Requirements Management	Develop a Concept of Operations, establish an office, and begin pilot operations.	The IRS established and staffed the Requirements Management Office and developed a Concept of Operations.	Additional Actions Underway – We interviewed project personnel from one project that received assistance from the Requirements Management staff. Project personnel provided positive feedback; however, some responsibilities were not clearly defined. Also, the GAO recently reported modernization projects were not consistently following requirements development and management practices and the IRS would not have a full suite of requirements processes until March 2007.[10] It will take time for the IRS to mature its Requirements Management capabilities. Until that time, it is difficult to judge whether the IRS has fully implemented its new role of managing requirements.
Enterprise Transition Strategy	Develop volume 1 of the Enterprise Transition Strategy.	The Enterprise Transition Strategy has been developed and will become part of the Enterprise Transition Plan.	Additional Actions Underway – As we were completing our audit, the IRS was creating the Enterprise Transition Plan using the Enterprise Transition Strategy.
Modernization Vision and Strategy	Develop a high-level modernization vision and strategy document.	The IRS completed the Modernization Vision and Strategy and the IRS Oversight Board approved it. In addition, the Associate Chief Information Officer, Applications Development, informed us the Modernization Vision and Strategy has been approved by the Office of Management and Budget.	Additional Actions Underway – The IRS completed its first update to the Modernization Vision and Strategy since 2002. According to IRS officials, the IRS learned valuable lessons as part of this exercise and is developing procedures to ensure the Modernization Vision and Strategy will be updated yearly.
Enterprise Life Cycle	Revise the ELC Guide to include paths common to IRS modernization projects.	The IRS has revised the ELC Guide to include only the information commonly needed by modernization projects. Also, the ELC staff has begun assisting projects by developing ELC tailoring plans for two projects.	Effective – The ELC Guide was revised and project teams that have recently used the services of the ELC staff provided positive feedback.

Source: Transitioned activities chosen by the TIGTA audit team, with input from IRS officials, IRS status reports, and documentation provided by the IRS.

Appendix VI

Management’s Response to the Draft Report

The response was removed due to its size. To see the response, please go to the Adobe PDF version of the report on the TIGTA Public Web Page.

[3] The IRS identified a fifteenth transition area; however, we determined it did not relate to the transition from the PRIME contractor to the IRS.

[4] See Appendix V for our independent assessment of the status of each program transition area.

[5] Business Systems Modernization: IRS Needs to Complete Recent Efforts to Develop Policies and Procedures to Guide Requirements Development and Management (GAO-06-310, dated March 2006).

[6] The 47 positions approved in Fiscal Year 2005 were allocated to 3 modernization functions as follows: 19 to Business Integration, 6 to Program Management, and 25 to Systems Integration. The positions allocated equal 50 because some approved positions were not funded and the IRS used existing hiring authority to fill 3 vacancies.

[7] As of June 30, 2006, the IRS informed us it had filled all but six positions.

[9] The IRS identified a fifteenth HPI; however, we determined it did not relate to the transition of activities from the PRIME contractor to the IRS. See Appendix V for a listing of the 14 HPIs.

[10] Business Systems Modernization: IRS Needs to Complete Recent Efforts to Develop Policies and Procedures to Guide Requirements Development and Management (GAO-06-310, dated March 2006).