Audit of the Enterprise Work Management Solution (EWMS): Lessons Learned for Future Large Information Technology Projects

formed a work management group to evaluate the feasibility of an enterprise-wide work management system that could be shared and integrated across divisions: - Toronto Water - Parks, Forestry and Recreation - Transportation Services - Solid Waste Management Services The Enterprise Work Management Solution program is expected to improve work management and service response times by integrating the divisions and the City's shared services, including 311. Enterprise Work Management Solution is being implemented in three phases across these multiple City divisions. This audit focused on phase one of the Enterprise Work Management Solution implementation, which started in 2013, and management anticipates completing it by the end of 2023. Planning is underway for the implementation of phases two and three. This audit examined how the Enterprise Work Management Solution program was planned and governed, including whether the program is meeting its budget and timelines, divisions' operational needs, and overall program objectives. Our findings and recommendations for the Technology Services Division on the Enterprise Work Management Solution program related to the following areas: A. Strengthening Project Planning Processes B. Strengthening Project Governance C. Enhancing Controls to Improve Project Execution D. Strengthening Project Risk Management and Controls We have made 14 recommendations in our report. Implementation of these recommendations will improve project planning, governance, and execution. The key lessons learned from this audit need to be applied not only to the Enterprise Work Management Solution program, but also to other large information technology (IT) projects of the City to improve overall project management and implementation. In addition, the findings and recommendations reported in this report may have relevance to agencies and corporations. The Auditor General has recommended the City Manager to forward this report to major agencies and corporations for their review.

A performance measurement framework is established to measure the achievement of technology project goals on a regular basis until the project is completed. 2. City Council request the Chief Technology Officer to leverage the Technology Services Division's Stakeholder Management Plan to improve stakeholder engagement, coordination, and communication for the remaining phases of implementing the Enterprise Work Management Solution. 3. City Council request the Chief Technology Officer to review and improve the existing project status monitoring criteria and dashboards to include: a. A methodology to accurately measure the implementation progress against budget spent and timelines; b. An update on the assessment of the benefits realization and potential losses that could incur as a result of increased expenditures and project delays; and c. Potential short-term and long-term risks the project may face that would impact budget, timelines, and realizing intended benefits in a timely manner. 4. City Council request the Chief Technology Officer to report to City Council on the status of the Enterprise Work Management Solution program by June 30, 2024: a. Total expected cost of implementation; b. Status of benefits realized for divisions that have completed the implementation; and c. Planned benefit realization for the entire Enterprise Work Management Solution program. 5. City Council request the Chief Technology Officer to develop criteria and requirements for Executive Steering Committee meetings based on project size and complexity, to ensure a consistent approach is followed. 6. City Council request the Chief Technology Officer to develop a consistent mechanism for documenting Executive Steering Committee meetings, key decisions and required actions in the meeting minutes. 7. City Council request the Chief Technology Officer to review and modify the existing Enterprise Work Management Solution's program governance structure to: a. Develop criteria and a process for escalation related for project delays to the Executive Steering Committee; and b. Establish a process to monitor and respond to emerging risks that may cause project delays, budget overruns, and impact on benefits realization. 8. City Council request Chief Technology Officer to expedite the implementation of the Enterprise Work Management Solution to: a. Utilize unused license subscriptions; b. Evaluate current license subscriptions and discontinue those that are not required; and c. Develop a plan in coordination with Division Heads to discontinue legacy systems, in order to minimize risks of legacy applications becoming non-operational and save on renewal and maintenance costs and present the plan to the Executive Steering Committee. 9. City Council request the Chief Technology Officer to work with Divisions to establish a governance framework to ensure that: a. Business processes are well defined in the planning phase before implementation; b. Role-based training is tailored for each business role before implementation; and c. User acceptance testing includes end user devices for usability testing prior to implementation. 10. City Council request the Chief Technology Officer in coordination with the Chief Financial Officer and Treasurer to review the existing reporting process to City Council for major technology project budgets and costs and implement: a. A process to ensure that consolidated budget and expenditures are reported to City Council for major cross divisional technology projects; and b. A process to ensure that all expenditures from inception to date related to a major technology project are included in the Technology Service Division's Budget Notes and Quarterly Capital Variance Reports. 11. City Council request the Chief Technology Officer to: a. Establish criteria as to what constitutes critical documents for a project; and b. Develop a process to ensure critical documents are maintained and retained throughout the project management lifecycle, with final versions stored in the program site according to the City's Record Retention policy. 12. City Council request the Chief Technology Officer to ensure that: a. Plans for the Enterprise Work Management Solution upgrade and Cloud migration are supported by a detailed risk assessment; and b. An action plan to address potential risks and delays is approved by all stakeholders. 13. City Council request the Chief Technology Officer to develop a process to incorporate previous Auditor General audit report recommendations related to project management and governance of technology systems, and software license management, into the Technology Services Division's project management framework. 14. City Council request: a. The City Manager forward this report to the major agencies and corporations for review. b. The heads of agencies and corporations review the issues and recommendations included in this report and consider the relevance to their respective organizations for implementation.