Northern Ireland Assembly Flax Flower Logo

Public Accounts Committee

First Thematic Report

Together with the Minutes of Proceedings of the Committee
Relating to the Report and the Minutes of Evidence

Ordered by The Public Accounts Committee to be printed 10 June 2010
Report: NIA 65/09/10R Public Accounts Committee

Session 2009/2010
Eighteenth Report

Membership and Powers

The Public Accounts Committee is a Standing Committee established in accordance with Standing Orders under Section 60(3) of the Northern Ireland Act 1998. It is the statutory function of the Public Accounts Committee to consider the accounts, and reports on accounts laid before the Assembly.

The Public Accounts Committee is appointed under Assembly Standing Order No. 56 of the Standing Orders for the Northern Ireland Assembly. It has the power to send for persons, papers and records and to report from time to time. Neither the Chairperson nor Deputy Chairperson of the Committee shall be a member of the same political party as the Minister of Finance and Personnel or of any junior minister appointed to the Department of Finance and Personnel.

The Committee has 11 members including a Chairperson and Deputy Chairperson and a quorum of 5.

The membership of the Committee since 9 May 2007 has been as follows:

Mr Paul Maskey 3 (Chairperson)
Mr Roy Beggs (Deputy Chairperson)

The Lord Browne 9
Mr John Dallat
Mr David Hilditch 7
Mr Trevor Lunn
Mr Patsy McGlone 2,6
Mr Mitchel McLaughlin
Mr Stephen Moutray 10
Ms Dawn Purvis
Mr Jim Shannon 5

1 Mr Mickey Brady replaced Mr Willie Clarke on 1 October 2007
1 Mr Ian McCrea replaced Mr Mickey Brady on 21 January 2008
1 Mr Jim Wells replaced Mr Ian McCrea on 26 May 2008
2 Mr Thomas Burns replaced Mr Patsy McGlone on 4 March 2008
3 Mr Paul Maskey replaced Mr John O'Dowd on 20 May 2008
4 Mr George Robinson replaced Mr Simon Hamilton on 15 September 2008
5 Mr Jim Shannon replaced Mr David Hilditch on 15 September 2008
6 Mr Patsy McGlone replaced Mr Thomas Burns on 29 June 2009
7 Mr David Hilditch replaced Mr George Robinson on 18 September 2009
8 Rt Hon Jeffrey Donaldson replaced Mr Jim Wells on 18 September 2009
9 The Lord Browne replaced Rt Hon Jeffrey Donaldson on 19 April 2010
10 Mr Stephen Moutray replaced Mr Jonathan Craig on 19 April 2010

Table of Contents

Report

Introduction

Recurring Themes

Summary of Project Management Shortcomings Identified Since 2007

Instances of Recurring Themes Identified by the Committee

Table of Issues by Project

Appendix 1

Minutes of Proceedings

Appendix 2

Grid of Occurrence of Shortcomings by Committee Inquiry

List of Abbreviations used in the Report

The Committee - Public Accounts Committee

e-PIC - Electronic Planning Information for Citizens

IDB - Industrial Development Board

LPS - Land and Property Service

PFI - Public Finance Initiative

PPE - Post Project Evaluation

Introduction

1. At its forward work planning session on 19 November 2009, the Public Accounts Committee (PAC) agreed to produce a First Thematic Report to address themes which it has encountered in numerous inquiries into administrative and financial systems and controls within Northern Ireland Government departments.

2. The Committee wished, in light of current economic conditions, to recap on lessons learned in the back catalogue of reports it has covered, with the aim of reaffirming crucial messages to improve Government stewardship of public money.

3. The Committee believes that as these themes continue to recur in its work, learning can be reinforced by presenting them in a thematic format for consideration throughout Government.

4. The Committee further believes that a thematic report offers an opportunity to reinforce the value of the public audit and scrutiny process that is central to democratic accountability.

5. Accordingly, this report reviews the following inquiries undertaken by the Committee:

  • Report on the Upgrade of the Belfast to Bangor Railway Line 01/07/R
  • Report on Springvale Educational Village Project 4/07/08R
  • Transfer of Surplus Land in the PFI Education Pathfinder Projects 11/07/08R
  • Report on Use of Consultants 16/07/08R
  • Report on Statement of Rate Levy and Collection 13/08/09R
  • Report on Shared Services for Efficiency — A Progress Report 21/08/09R
  • Report on Delivering Pathology Services: The PFI Laboratory and Pharmacy Centre at Altnagelvin 16/08/09R
  • Report on the PFI Contract for Northern Ireland's New Vehicle Testing Facilities 35/08/09R
  • Report on the Review of Assistance to Valence Technology: A Case Study in Inward Investment 02/09/10R
  • Report on the Performance of the Planning Service 36/09/10R

Recurring Themes

6. Since 2007, major Government projects examined by the Committee have regularly incurred increased costs, been completed later than planned and failed to deliver the anticipated benefits to the public.

7. These disappointing outcomes have been due to shortcomings at different points in the project lifecycle.

8. The Committee's reports have highlighted flaws which must be guarded against at each stage of a project, and these are demonstrated in this report in terms of:

  • Project planning
  • Project management
  • Action to protect interests of the taxpayer
  • Post-project evaluations

Project Planning

People

9. Successful delivery of major projects requires the availability and application of the right skills at the right levels. A number of cases heard by the Committee have exposed worrying skills deficits in areas such as IT, accounting and project management. Lack of access to talented, commercially orientated specialists undermines the public sector's ability to negotiate effective outcomes with private sector contractors.

10. The Committee has reinforced the need for Departments to build project teams of personnel who demonstrate the key skill sets required, such as economists, accountants, and experts in procurement, contracts and the systems or services to be delivered. Project teams must be equipped with skills and experience proportionate to the size, complexity and importance of the project.

11. Successful projects require a proper assessment and allocation of risk, opportunity, cost and deliverable outcomes. Having the right people enables sufficiently competent and rigorous testing of options and sound economic, viability and risk analysis. Controls must be adhered to and each member of the team must exercise responsibility and a challenge function so that safeguards cannot be circumvented.

Project Planning

Appraisal

12. Robust and accurate appraisal of options is a pre-requisite for the successful delivery of major projects. Too often, however, this process is not undertaken well. The Committee has experience of many appraisals which significantly under-estimated costs with the result that the projects inevitably exceeded their budgets.

13. Unduly optimistic and misleading appraisals of this nature ultimately do nothing to ensure the success of projects, but rather undermine their viability. This optimism bias is not limited to the experience of this PAC, but was also signalled recently to be a key concern of the analagous Westminster committee.[1]

14. Failures in appraisal are often mirrored in poor specification and scoping of projects. The Committee has seen a wide range of projects which have been inadequately scoped in the initial stages, with the result that significant changes are required during the life span of the projects. This often gives the contractors a significant negotiating advantage with the public sector client and undermines value for money principles of open competition.

15. Pilot testing of new systems is essential, and failure to do so adequately has caused real problems in a number of projects leading to adverse cost and performance implications. Shortcuts with testing must never be taken, no matter how important the deadline, as the cost of any subsequent flaws can be very substantial in terms of fixing an underperforming live system.

16. The Committee highlights investment in IT and new technology as areas of concern. Here, pre-contractual objective testing, live piloting and properly scoped specifications are crucial.

Project Management

17. It is disappointing to note in a number of projects the failure by parent Departments to exercise sufficient control and oversight of projects. Departments must ensure that project promoters have a firm grip on the viability of projects from the start, and exert a strong challenge function to test viability and affordability at the right time and in the right way.

18. The Gateway process offers a key control during the life of the project and should help to improve project delivery and effectiveness. The Committee has therefore been disappointed to note that not all major projects have been subject to the discipline of Gateway. Departments must ensure that the Gateway process is used at the appropriate stages on all major programmes and projects.

19. Public bodies also need to be aware of the risks of placing reliance on a small number of individuals in a project team. Major projects should be fully supported by a properly costed and appropriately skilled internal project management structure. In some cases, there has been a disturbing lack of staff continuity and a number of vacancies in the project management functions which undermines the bodies' ability to hold the contractor to account.

20. Contract management is another area which the Committee considers needs to be strengthened. Contracts must be properly structured with penalty clauses and conditions to secure the prompt payment of monies due. Similarly, to be more competitive the public sector needs to be able to assess the profitability of contracts to their private partner, so open-book accounting must become a standard requirement across public sector contracts and intangible benefits such as intellectual property rights must be valued.

21. A recurrent theme in projects which experience delivery problems is that they have often been undertaken in a climate where the body involved has faced a multiplicity of competing policy, operational and related challenges. Public bodies need to be realistic and careful about the number of complex change management projects which they can resource and deliver at the same time given the short supply of skilled and experienced project management specialists.

Actions to Protect Interests of the Taxpayer

22. The Committee has heard evidence about major projects where it is clear that controls are not robust or have been circumvented, costs have escalated unchecked or there has been wasted expenditure. There are also several projects where the benefits of the project have not materialised or have been much lower than planned.

23. Appraisal, monitoring of changes in scope and adherence to controls and expert judgments must be maintained throughout the life of the project to avoid incremental variations building up to a significant and costly departure from the initial objectives of the project.

24. A key control in terms of protecting the taxpayers' interests is to ensure that appropriate clawback arrangements are in place. Where such arrangements are defective, the public sector's long-term interests are compromised and it risks losing its share of future development gains or profits arising from the future sale or transfer of public sector assets such as land and buildings.

Post-Project Evaluations

25. Post-project evaluation is a requirement in all major projects. Despite this, the Committee has taken evidence on a significant number where this basic, best practice has not been followed. It is particularly disconcerting when this has happened in cases of projects with multiple failings where there are clearly lessons to be learnt for the benefit of the wide public sector.

Summary of Project Management Shortcomings Identified Since 2007[2]

Project Planning – Project Specification

Shortcomings

Actions Required

Project specifications have been incomplete or inadequate. In one instance, a project as initially tendered was seriously over-specified.2

When considering procurement programmes or projects, departments must fully consider all options and potential risks to the procurement process before going to the market. Failure to do so will make it more difficult to demonstrate that value for money has been secured.

Project planning – economic appraisals and business cases

Shortcomings

Actions Required

Project costs under-estimated The viability of projects must be fully assessed at an early stage, with detailed updates carried out as circumstances develop. The aim of a rigorous assessment of viability is not to "trip up" a project; rather, it is a mechanism to ensure that potential weaknesses are identified at an early stage and can be addressed.
Appraisals have been overly-optimistic on projects' prospects for success and in some cases actually manipulated or contrived to produce favourable results Departments should comprehensively review and challenge appraisals and business cases to ensure that viability of projects is fully assessed at an early stage with detailed updates as carried out as circumstances develop.
Warnings from departmental economists over the reliability of data in appraisals have been disregarded Where legitimate, evidence-based concerns regarding the accuracy of cost estimates or viability are highlighted to senior management, Departments must exert robust challenge, insist on independent validation of key cost estimates, and must not accept unsupported assertions.
Appraisals have been unduly rushed due to time constraints and a desire to rush projects through Time has often been a key factor in limiting the public sector's ability to negotiate a good deal for the public purse;the public sector must be careful to avoid being time constrained when dealing with the private sector.
For many consultancy contracts, business cases were simply not produced Departments must complete business cases of an appropriate quality to justify using consultants and to define the scope and outputs. The Committee recommends that, for all major consultancy projects (over £75,000), Centres of Procurement Expertise are always used to assist departments and related bodies in preparing robust business cases.

Project planning – system testing

Shortcomings

Actions required

System testing for complex and risky projects has been inadequate and not carried out in conditions which reflected the live environment Shortcuts with testing must never be taken, no matter how important the deadline, as the cost of any subsequent flaws can be very substantial in terms of fixing an underperforming live system.

Project management – general issues

Shortcomings

Actions required

No re-appraisal of projects when it became clear that costs would far exceed budget, or when there was overwhelming evidence that project was failing Lack of oversight by Departments of projects being delivered by arm's-length bodies Departments should comprehensively review and challenge appraisals and business cases to ensure that viability of projects is fully assessed at an early stage with detailed updates carried out as circumstances develop.The Gateway process offers a key control during the life of the project and should help to improve project delivery and effectiveness. Departments must ensure that the Gateway process is used at the appropriate stages on all major programmes and projects. Monitoring arrangements for high-risk projects must ensure that roles and responsibilities are always clearly stated, that they are allocated to an appropriate level, and that monitoring meetings take place regularly, are formally minuted and outcomes reported promptly to senior management. Effective financial planning and management of projects are pre-requisites for success. Funding Departments and project promoters must get a firm grip on these fundamental elements of control, from the outset and throughout the life of a project.
Insufficient in-house expertise to manage projects: Teams too small and have insufficient skills to match private sector negotiators Insufficient expertise in the areas of economic appraisal, IT and accountancy For major projects that require negotiations with the private sector, the public sector team must be adequately resourced and have the required experience, expertise and skills to at least match those of the private sector. A general pool of experts from the wider public sector who have extensive skills and experience of successfully delivering IT projects should be formed and if necessary enhanced by recruitment. To prepare for future large scale procurements, departments should identify the pool of NICS with the skills, experience and availability to be an SRO and, if necessary, develop and train additional staff for this demanding role. IT systems in the public sector have regularly caused difficulties, particularly if implemented at the same time as policy changes. New IT systems should not go live until major policy changes are finalised. Where this cannot be avoided, such IT projects should be assessed and managed on the basis that there is a high risk of failure, risks must be fully evaluated and adequate resources and contingency plans put in place.
Projects have been implemented with undue haste, which in turn has resulted in poor economic and operational outcomes It is a particular responsibility of senior management to ensure that sufficient time is always devoted to project appraisals and that all aspects are thoroughly assessed, with any weaknesses properly addressed.

Project management – protecting the taxpayer's interests

Shortcomings

Actions required

Right to claim damages waived Public sector bodies must ensure that contracts are properly structured with penalty clauses and conditions which secure the prompt payment of monies due to them
Defective clawback arrangements Clawback arrangements agreed with the private sector should address the public sector's long term interests, preserving its rights to share in future development gains or profits arising following the sale or transfer of assets to connected parties.
Failure to secure profit share for the public sector Public sector teams must be commercially aware and a match for private sector negotiators. Transparent arrangements such as open-book accounting must be put in place by public sector teams to ensure they are aware of the profitability of a deal to the private sector.
Losses incurred as a result of poor consultancy advice not recovered Public bodies should have in place systems to monitor and assess the quality of advice provided by their advisors. Where there is clear evidence that advice or support provided by advisors has not met the standards required and has resulted in loss to the public purse, appropriate action should be taken to recover fees paid and losses incurred.

Project management – post project evaluations

Shortcomings

Actions required

Post project evaluations (PPEs) have simply not been completed, or have not been of the required quality. This means that lessons learnt and good practice identified has not been disseminated across the system for future application Post-project evaluations should always be carried out on major projects and a specific timescale for doing so should be set out at appraisal stage and adhered to by the lead department. As part of the PPE, information must be made available on the full costs of the projects. In addition, lessons learned reports should be circulated to departments and made available to other public sector organisations.

Instances of Recurring Themes Identified
by the Committee

Report on the Upgrade of the Belfast to Bangor Railway Line

26. In January 2001 the Department for Regional Development approved a project, to upgrade the Belfast to Bangor railway line. Originally budgeted to cost £14.7 million, the project experienced a number of difficulties with the result that the final cost was £34 million, more than twice the original forecast.

27. The overspend on the project was the result of failures in several key areas. The economic appraisal seriously underestimated the costs and, the department failed to reappraise the project even when it became clear it would far exceed budget. Translink did not have in place basic management systems and its lack of control of contract variations was akin to giving the contractor a blank cheque. Translink had to enter into a protracted settlement process in which it waived its legitimate rights to claim damages from a poorly-performing contractor. This cost the public purse an extra £8 to £13 million.

28. The Committee's report highlighted a number of lessons. These included the need to ensure that departments more robustly challenge and validate key cost estimates; to ensure a timescale for post-project evaluations is clearly stipulated at economic appraisal stage; and to ensure the departments retain full rights of redress where public bodies are terminating the contracts of consultants. The Committee also highlighted the need for improved governance arrangements, proper records management and full documentation of all management decisions.

Springvale Educational Village Project

29. In 1998 the Department for Employment and Learning entered a project as main funder with the University of Ulster and Belfast Institute of Further and Higher Education, now Belfast Metropolitan College. The project was to deliver three components, namely a main campus costing £59 million; an applied research and start-up centre costed at £8 million; and an outreach centre estimated at £4 million.

30. Failures in project appraisal, financial planning and management of the project by the Department, and the unexpected withdrawal of the University from the project; meant that of the planned outcomes, only the outreach centre was delivered.

31. The Committee found that a number of the project appraisals failed to identify what proved to be key issues in relation to viability and affordability at a sufficiently early stage, when they could have been easily addressed. The partners did not successfully address the issues when they did emerge with the result that the anticipated benefits were not delivered.

Report on the Transfer of Surplus Land in the PFI Education
Pathfinder Projects

32. Between June 1999 and October 2000 five contracts were let for six Education PFI Pathfinder projects comprising four schools and two further and higher education colleges. Four of the five contracts contained clauses dealing with the transfer of surplus assets from the public sector to PFI operators. The value agreed, through negotiation, for these surplus assets was over £23 million.

33. The Committee found that the Department of Education made the projects unnecessarily complex, the team did not seem to have the skills to match the private sector negotiators, and that the Department's rush to deliver these schemes added further to their complexity and weakened the public sector's ability to negotiate a good deal. This was a key factor which was exploited by the private sector.

34. The inclusion of land in the PFI contracts meant the assets could not be sold on the open market, and failures to value, measure and maximise the potential of the land before transfer meant that the Department took six years to achieve its aim of recovering the value of the land while the private sector partners could achieve market value in a flourishing property market.

35. The Committee recommended that when considering the disposal of a site, public bodies must adhere to the basic principles of defining the site precisely and valuing it accordingly. It stipulated that public bodies must ensure that they engage the Land and Property Service, and where appropriate recognised professional valuers, from the outset and ensure that valuations are updated on a regular basis.

36. It said that public bodies must learn the lessons from these cases when agreeing contract provisions such as "Contractor Monetary Thresholds" and "extras" and ensure that they are comprehensive and transparent and proposed that public bodies ensure that a standard approach is applied in the drawing up of such contract provisions. The Committee also made recommendations to improve clawback arrangements, requiring that contracts are properly structured with penalty clauses and conditions which secure the prompt payment of monies due to them.

Report on the Use of Consultants

37. Over the 5 year period to 2007, spending by Northern Ireland Departments on consultants more than doubled and, in 2006-07 alone, amounted to £42 million. The Committee examined the use of consultants and found that this spending was not always well managed and best practice guidance was not being adhered to across the public sector.

38. For instance, the use of consultants should always be justified by completion of a business case. If approved, procurement should be by competitive tendering. The Committee was concerned that Departments regularly failed to complete a business case, and asked DPF to assess compliance with the requirements to complete a business case, tender competitively and complete a post-project evaluation.

39. The Committee heard evidence in previous sessions where the performance of consultants actually cost the public purse additional expense, yet there has been no recompense. DFP and DRD were unable to provide a single example where the results of a post-project evaluation had been used to recover losses arising from poor consultancy advice.

40. To avoid non-competitive tendering and resulting conflicts of interest the Committee recommended that consultancy be procured centrally via Framework Agreements and managed by Centres of Procurement Expertise. The Committee also warned against post-hoc approval of irregular expenditure when delegated limits have been breached.

Report on Statement of Rate Levy and Collection

41. This report examined the management of rate collection activity by Land and Property Service (LPS), an agency of the Department of Finance and Personnel during a time of considerable change in policy, systems and management structures.

42. The Committee prioritised this inquiry given the importance of rate revenue collection to both local and central government finances. Rates assessments amount to some £1 billion annually.

43. The Committee found that the agency was trying to do too much at once by changing policy, management and systems simultaneously. A fundamentally flawed risk assessment failed to give enough weight to systems testing of the new IT system, transfer of front-line staff to implementation of rate reform and the financial impact of postponing rate collection.

44. The Committee made recommendations focusing on lessons learned about procurement and testing of major IT systems; on how specifications could be improved; on the expert teams required to undertake a project of such complexity; and on the need to underpin financial governance with anti-fraud measures as safeguards when systems are weakened.

Report on Shared Services for Efficiency: A Progress Report

45. The Northern Ireland Civil Service undertook a major programme of reform, known as the Reform Agenda. This ambitious programme of projects sought to improve NICS capacity and capability by rolling out shared service systems across Government. Central to the programme were seven key projects with an estimated value of £3 billion.

46. A number of issues were identified that to date have hindered the delivery and effectiveness of the projects including inadequate in-house resources, the ability to measure the benefits delivered by each project and inadequate project specifications.

47. The Committee's report highlighted a number of lessons. These included the need to ensure that staff have the necessary abilities and skills to undertake such projects, and a robust project specification to ensure that systems are fit for purpose and deliver to the required standard that can be measured.

Report on Delivering Pathology Services:
The PFI Laboratory and Pharmacy Centre at Altnagelvin

48. In 2001 a PFI deal was entered into by the Departments of Finance and Personnel and Health, Social Services and Public Safety to create new facilities at Altnagelvin Hospital. The PFI Laboratory and Pharmacy Centre, costing £15.2 million, was a key component of the £250 million redevelopment programme for the Altnagelvin Hospital Complex.

49. This major capital programme overall was very successful including the use of exemplar design. However there were important lessons to be learned. These included the delay in settling on the PFI procurement option, the need for projects to be subject to Gateway Review, the need to record internal costs.

50. The Committee's report highlighted that Departments when undertaking major capital programmes must decide early in the procurement process

  • the funding arrangements with value for money being the prime consideration;
  • to maximise the facilities of the centre providing outside services to other Trusts;
  • that the NI Procurement Office consider the benefits of exemplar design to ensure it is given sufficient weight when projects are considered;
  • and to ensure that output specifications must be clearly defined before approaching the market.

Report on the PFI Contract for Northern Ireland's New Vehicle
Testing Facilities

51. In the mid-1990s the Department of the Environment's Driver and Vehicle Agency's vehicle testing equipment was outdated and increasingly prone to breakdown. In March 2001, it signed a Public Finance Initiative (PFI) contract costing £57 million for the design, finance and build of 61 computerised test lanes at its 15 centres.

52. Significant improvements in performance were achieved eventually in this project. However, this came at a financial cost due to incomplete risk transfer by the Agency in the contract. While the contractor is fully responsible for maintaining and operating the equipment, the Agency is responsible for the staff operating it. This created difficulties in trying to determine the respective liabilities for performance failures.

53. For instance, it was difficult to determine liability for the failure to deliver the contractually agreed test times of 18 minutes in the early years of the contract, and the Agency did not recover costs for this failure from the contractor. Rather, these were borne by customers.

54. The Committee was also concerned that the Agency had a complete lack of knowledge about the profitability of the contract to the private contractor.

55. In addition to allocation of risk, pilot testing and transparency, the Committee was concerned at the timing and adequacy of early warning and review systems. Accordingly, it made recommendations promoting system testing and piloting in a live environment; transparency of profitability information concerning the private partner; and expertise in complex projects. The Committee also reiterated the lesson from PFI Pathfinder Projects in Education, that public sector bodies must ensure contracts are properly structured with enforceable penalty clauses and conditions to secure prompt payment of moneys due to them.

Report on the Review of Assistance to Valence Technology:
A Case Study in Inward Investment

56. This report dealt with the 14-year history of a major inward investment project, handled by the former Industrial Development Board (IDB). Announced as the biggest single investment ever made in Northern Ireland, the project was promoted by Valence Technology, a US-based company incorporated in 1989.

57. In 1993, IDB offered Valence a package of capital and employment grants of over £27 million to establish a new large-scale manufacturing facility at Mallusk. In return, the company was to invest £147 million in Northern Ireland and create 660 new jobs at the plant, by March 1998. The aim was to develop and manufacture batteries using a new, lithium-based technology.

58. However, Valence experienced persistent difficulties, and the anticipated technological breakthrough did not materialise. In 2003, some ten years after its first contact with IDB, Valence relocated to China. Invest NI entered into a protracted clawback process which ultimately concluded in July 2007, when it recovered some £5.1 million of the £15 million which it had spent.

59. IDB failed to apply many of the most fundamental elements of its appraisal guidelines. At the time the project was assessed, Valence had not even developed a commercial product and so could not produce a proper Business Plan, an absolute necessity for any meaningful appraisal of viability.

60. The Committee found that controls were circumvented and ignored at all stages of the process, and the Department failed to renegotiate at crucial times.

61. The Committee's recommendations focused on adhering to controls, proper appraisal and objective testing, and evidence based review of the project when things begin to go wrong.

Report on the Performance of the Planning Service

62. This report examined all aspects of Planning Service delivery, performance and customer service.

63. In terms of management of capital projects, the Committee focused on the much anticipated electronic Planning Information for Citizens system (e-PIC), a flagship IT project designed to allow electronic delivery of planning processes.

64. In 2004, Planning Service signed a contract for the e-PIC system, which has now run significantly over time and over budget.

65. Originally due to be delivered in 2006, it is now four years behind schedule, and the anticipated capital costs at the time of the report were projected to be £12·8 million, some 130 per cent above the original budget.

66. The Committee outlined a series of lessons to be rolled out across the public sector as a result of the e-PIC experience.

67. In particular, the Committee underlined the need for comprehensive project specification and for full awareness of the risk of procuring "off-the-shelf" systems which subsequently require considerable adaptation.

68. It was clear to the Committee that Planning Service and their consultants had massively underestimated the extent and cost of the amendments necessary to make the e-PIC system fit for use in Northern Ireland. This fundamental flaw was compounded by limitations in the contract, which meant that additional costs could not be recouped from the contractor.

69. The Committee also highlighted the inadequacy of action to react to reviews of the project, as successive reviews highlighted the poor quality of project management.

70. Failure to manage staff turnover, a poor working relationship with the contractor and disconcertingly, the payment of bonuses to senior staff managing the project were also highlighted in the Committee's report.

71. Accordingly, the Committee made the following recommendations about lessons to be learned from the e-PIC experience:

  • The e-PICs project represents a catalogue of project management failures. The Committee recommends that Planning Service take robust action to ensure that there are no further delays on the project. It is essential that the project is completed no later than 2010, so that a fully functional system is handed over to local government in 2011.
  • The Committee recommends that, where a health check or Gateway review on a business-critical project such as e-PIC highlights significant management shortcomings, senior staff in Departments must be prepared to step in and act decisively to remedy them.

72. The Committee considered that there were major lessons to be learnt from the mismanagement of the e-PIC project and recommended that these lessons are disseminated throughout the wider public sector:

(i) the need for proper and comprehensive project specification from the outset to reduce the potential for problems later on;

(ii) the need for an appropriate form of contract which enables the public sector to impose penalties and recoup the costs in the event of poor contractor performance;

(iii) the need to recognise that the option of procuring an existing, off-the-shelf, product for adaptation is inherently high risk and not a substitute for a proper project specification;

(iv) the neeed for continuity of key personnel throughout the project development;

(v) the need to build relationships and communicate openly with the contractor in order to resolve problems quickly and efficiently.

Appendix 1 — Table of issues by project

PAC report Key Project Management issues
Upgrade of the Belfast to Bangor Railway Line Project over budget and not delivered on time Lack of departmental oversight of management of project by NDPB Poor quality economic appraisal Project not re-appraised in light of cost increases PPE not completed five years after project completion No Translink projects subject to Gateway review Lack of procurement accreditation No control over contract variations Right to claim damages waived Poor project specification Poor audit trail No investigation into potential fraud / impropriety Poor standards of corporate governance within Translink
Springvale Educational Village Project Poor oversight of the project by DEL as project sponsor and main funder Failure to address affordability and viability of the project Poor joined-up government between the Department (project sponsor) and University (project promoter) Inadequate project appraisal process – appraisals overly-optimistic on the viability of the project Project outcomes very disappointing
Transfer of Surplus Land in the PFI Education Pathfinder project Project delivered too hastily, limiting Department's ability to negotiate good deal for the public purse Inadequate clawback arrangements Poor contract design – surplus land should not be bundled into PFI deals Failure to achieve market value for surplus land Failure to seek VLA advice Insufficient Departmental resources to negotiate a good deal for the public purse
Use of Consultants Frequent failure by Departments to produce Business Cases Insufficient in-house expertise within Departments to manage consultancy appointments Lack of competitive tendering for consultancy appointments £2 million of DRD contracts not awarded through Centres of Procurement Expertise Poor quality of project appraisals PPE's either not carried out, or if carried out, results not shared across Departments No clawback or recovery of losses for poor performance
Report on Statement of Rate Levy and Collection 2006-07 Project delivered too hastily (complex and large IT system) Flawed risk assessment Inadequate system testing Incomplete project specification Cost escalations in project Findings of Gateway review not acted on – risks had been flagged up Lack of in-house IT and accountancy expertise to negotiate with private sector contractors Poor quality data transfer
Delivering Pathology Services: The PFI Laboratory Project not subject to full Gateway review Internal costs incurred on project not recorded Good practice – contract management officer appointed and commercial interests of public sector protected Good practice – use of exemplar design
Shared Services for Efficiency – a Progress Report Inadequate project specifications for Account NI, HR Connect and Network NI Workplace 2010 – failure to re-value properties due to be transferred to the private sector Inadequate in-house resources to manage projects Difficulties in measuring benefits delivered by projects No Gateway review for Centre of Applied Learning
Performance of the Planning Service e-PIC IT project - Cost and time overruns Inadequate project specification Risks of procuring an "off-the-shelf" system
PFI Contract for Northern Ireland's new vehicle testing facilities Failure to transfer key risk to PFI contractor Inadequate pilot testing of system
Review of Assistance to Valance Technology: A Case Study on inward investment Issues over clawback and failure to properly secure assistance Poor audit trail Project not re-appraised in light of serious difficulties Project appraisal rushed / too hasty Economic test of project based on unrealistic, overly optimistic assumptions Failure to consult with VLA over value of property Project Management Group was established, but did not meet Funding cap increased twice on project despite the fact that the project was experiencing major difficulties Serious project under-performance

[1] "Project management must be improved. In particular, public bodies must reduce optimism bias in their planning of projects and be more honest about what can reasonably be achieved and the risks to delivery". Edward Leigh, 'An open letter to my successor as Chair of the Committee of Public Accounts', April 2010.

[2] Report on the upgrade of the Belfast to Bangor Railway Line

Appendix 1

Minutes of Proceedings
of the Committee
Relating to the Report

Thursday, 19 November 2009
Room 144, Parliament Buildings

Present: Mr Paul Maskey (Chairperson)
Mr Roy Beggs (Deputy Chairperson)
Mr Jonathan Craig
Mr John Dallat
Rt Hon Jeffrey Donaldson MP MLA
Mr David Hilditch
Mr Trevor Lunn
Mr Patsy McGlone
Mr Mitchel McLaughlin
Ms Dawn Purvis
Mr Jim Shannon

In Attendance: Ms Aoibhinn Treanor (Assembly Clerk)
Mr Phil Pateman (Assistant Assembly Clerk)
Miss Danielle Best (Clerical Supervisor)
Mr Darren Weir (Clerical Officer)

Apologies:

2:02 pm The meeting opened in public session.

4. Draft Forward Work Programme.

Members received a briefing from the Northern Ireland Audit Office on the reports available for selection for the forward work programme.

Agreed: Members agreed the scheduling of reports for evidence sessions for the period January – April 2010.

Members considered a summary of proposals resulting from the Committee's recent business planning meetings.

Agreed: Members agreed the proposals as a basis on which to proceed.

[EXTRACT]

Thursday, 13 May 2010
Room 144, Parliament Buildings

Present: Mr Paul Maskey (Chairperson)
The Lord Browne
Mr John Dallat
Mr David Hilditch
Mr Trevor Lunn
Mr Mitchel McLaughlin
Mr Patsy McGlone
Mr Stephen Moutray
Ms Dawn Purvis
Mr Jim Shannon

In Attendance: Ms Aoibhinn Treanor (Assembly Clerk)
Mr Phil Pateman (Assistant Assembly Clerk)
Miss Danielle Best (Clerical Supervisor)
Mr Darren Weir (Clerical Officer)

Apologies: Mr Roy Beggs (Deputy Chairperson)

2:00 pm The meeting opened in public session.

8. Structure of the Draft Thematic Report

Members were briefed by the C&AG on the scope of the thematic report and noted the proposed structure of it.

Agreed: Members agreed to change the scope of the report to a thematic report on projects.

[EXTRACT]

Thursday, 3 June 2010
Room 144, Parliament Buildings

Present: Mr Paul Maskey (Chairperson)
Mr Roy Beggs (Deputy Chairperson)
Mr John Dallat
Mr David Hilditch
Mr Jim Shannon

In Attendance: Ms Aoibhinn Treanor (Assembly Clerk)
Mr Phil Pateman (Assistant Assembly Clerk)
Miss Danielle Best (Clerical Supervisor)
Mr Darren Weir (Clerical Officer)

Apologies: The Lord Browne
Mr Trevor Lunn
Mr Patsy McGlone
Mr Mitchel McLaughlin
Mr Stephen Moutray
Ms Dawn Purvis

2:39 pm The meeting opened in public session.

8. Consideration of the Draft Thematic Report

The Committee agreed to consider this item at its next meeting on 10 June 2010.

[EXTRACT]

Thursday, 10 June 2010
Room 144, Parliament Buildings

Present: Mr Paul Maskey (Chairperson)
Mr Roy Beggs (Deputy Chairperson)
The Lord Browne
Mr John Dallat
Mr David Hilditch
Mr Patsy McGlone
Mr Mitchel McLaughlin
Mr Stephen Moutray
Ms Dawn Purvis

In Attendance: Ms Aoibhinn Treanor (Assembly Clerk)
Mr Phil Pateman (Assistant Assembly Clerk)
Miss Danielle Best (Clerical Supervisor)
Mr Darren Weir (Clerical Officer)

Apologies: Mr Trevor Lunn
Mr Jim Shannon MP

2:04 pm The meeting opened in public session.

5. Consideration of the Draft Committee Thematic Report

Paragraphs 1 – 13 read and agreed.

Paragraph 14 read, amended and agreed.

Paragraph 15 read and agreed.

Paragraphs 16 - 17 read, amended and agreed.

Paragraph 18 – 25 read and agreed.

2:29 pm Mr McGlone entered the meeting.

Pages 5 – 9 read and agreed.

Paragraphs 26 – 28 read and agreed

Paragraphs 29 – 30 read, amended and agreed.

Paragraphs 31 – 34 read and agreed.

Paragraph 35 read, amended and agreed.

Paragraphs 36 – 37 read and agreed.

Paragraph 38 read, amended and agreed.

Paragraphs 39 – 41 read and agreed.

Paragraph 42 read, amended and agreed.

Paragraphs 43 – 47 read and agreed.

Paragraphs 48 – 49 read, amended and agreed.

Paragraph 51 read, amended and agreed.

Paragraph 52 read and agreed to be incorporated to Paragraph 51.

Paragraph 53 read, amended and agreed.

Paragraphs 54 – 63 read and agreed.

Paragraph 64 read, amended and agreed.

Paragraphs 65 – 69 read and agreed.

Paragraph 70 read, amended and agreed.

Paragraph 71 read and agreed.

Pages 17 – 20 read and agreed.

The Committee agreed to include an additional paragraph to reflect a previous recommendation.

Agreed: Members ordered the report to be printed.

Agreed: Members agreed that the report would be embargoed until 00.01 am on Thursday, 24 June 2010.

[EXTRACT]

Appendix 2

Grid of Occurrence
of Shortcomings by Committee Inquiry

Issue Belfast / Bangor line Springvale PFI Pathfinder Use of consultants Rent levy and coll 06-07 Altnagelvin PFI Shared Services Planning Service Valance MOT2
Poor project outcomes
¦
¦

¦

 

¦

 
 

¦

¦

¦

Issues over the appraisal process

¦

¦

 

¦

 
 
 
 

¦

 
Gateway related issues

¦

 
 
 
 

¦

 
 
 
 
Project specification issues

¦

 

¦

 

¦

 

¦

¦

 
 
Insufficient in-house resources to manage project
 
 

¦

¦

¦

 

¦

 
 
 
Time constraint issues
 
 

¦

 

¦

 
 
 

¦

 
Clawback issues
 
 

¦

¦

 
 
 
 

¦

 
Property related issues (VLA)
 
 

¦

 
 
 

¦

 

¦

 
Failure to re-appraise projects

¦

 
 
 
 
 
 
 

¦

 
Post project evaluation issues

¦

 
 

¦

 
 
 
 
 
 
Procurement accreditation issues

¦

 
 

¦

 
 
 
 
 
 
Inadequate system testing
 
 
 
 

¦

 
 
 
 

¦

Poor audit trail

¦

 
 
 
 
 
 
 

¦

 
Lack of departmental oversight over bodies at arms length

¦

¦