PUBLIC ACCOUNTS COMMITTEE
OFFICIAL REPORT
(Hansard)
‘A Review of the Gateway Review Process’
4 February 2010
Members present for all or part of the proceedings:
Mr Paul Maskey (Chairperson)
Mr Roy Beggs (Deputy Chairperson)
Mr Jonathan Craig
Mr John Dallat
Mr Jeffrey Donaldson
Mr Mitchel McLaughlin
Mr Jim Shannon
Witnesses:
| Mr Geoff Allister | ) | |
| Mr Des Armstrong | ) | Department for Regional Development |
| Mr Paul Priestly | ) | |
| Mr David Carson | ) | Department of Finance and Personnel |
Also in attendance:
| Mr Kieran Donnelly | ) | Comptroller and Auditor General |
| Ms Fional Hamill | ) | Treasury Officer of Accounts |
The Chairperson (Mr P Maskey):
We now move to the evidence session on the Audit Office report ‘A Review of the Gateway Process’. We will address the matters that were raised in that report. Does any member wish to express an interest in this matter? I see that no member wishes to register an interest, so I will continue.
Mr Priestly, who is the accounting office for the Department for Regional Development (DRD), is here to respond to the Committee. You are very welcome, Mr Priestly. Please introduce your colleagues.
Mr Paul Priestly (Department for Regional Development):
I am joined by Geoff Allister, who is chief executive of the Department for Regional Development’s Roads Service, and by Des Armstrong and David Carson, who are from the Central Procurement Directorate (CPD) in the Department of Finance and Personnel (DFP).
The Chairperson:
Thank you. I will start off the questioning. Paragraph 9 of the report states that your Department has undertaken capital projects worth £2 billion since the gateway review process was introduced in 2004. As accounting officer, are you content that not one of those projects has been sufficiently important or of a high enough risk to justify using the gateway review process?
Mr Priestly:
Thank you, Chairman. There are two issues to consider. First, the figure of £2·03 billion that is quoted in figure 2 refers to the capital value of projects, not the actual amount that the Department spent. Secondly, the guidelines within which DRD was operating at the time did not link the size of capital spend to the gateway review process. The process that was in operation at the time required Departments to carry out a risk-potential assessment (RPA) for significant capital programmes, and on the basis of that assessment, they were then required to determine whether those programmes should be subject to the gateway review process. The point that I am trying to make is that there is no direct link between the capital value of a project and whether it should be subject to the gateway review process. Rather, the risk that existed at the time that the policy was in place determined whether a project should be subject to the gateway review process.
The Committee will be aware from the report that the gateway review process should be used for projects that have a risk-potential assessment score of 31 or higher. Since the publication of the Northern Ireland Audit Office (NIAO) report, new guidance has been introduced. That guidance now states that all programmes and projects with a capital value of £20 million or more must be subject to the gateway review process. The new guidance links capital spend to the gateway review process. Of course, as accounting officer for the Department for Regional Development, I will be ensuring that we now comply with that guidance.
The Chairperson:
I will take you back to the point that I am trying to make. As accounting officer, can you tell us whether you or your Department did not think, prior to the publication of the report and the issue of new guidelines, that any of the projects in question were of a high enough importance to justify a gateway review process?
Mr Priestly:
What projects do you have in mind?
The Chairperson:
I am asking you about any of your projects; you have not done any gateway reviews.
Mr Priestly:
In fact, we have carried out a number of gateway reviews. I will repeat the point that the new policy was introduced at the turn of the year. When those projects were in planning, the policy was that there was no link between the size of capital spend and the requirement to carry out a gateway review. The determining factor for whether a review should be undertaken was based on a risk-potential assessment. If a project’s RPA score was lower than 30, there was no requirement to undertake a gateway review. That is the policy that we applied. Despite the fact that the projects in question are large capital programmes, they are not inherently risky and therefore do not have a high RPA score.
The Chairperson:
That is another question, but you are saying that those projects were not risky. Does that mean that they were risk free?
Mr Priestly:
No project is risk free. Our project-management arrangements contain risk-management procedures, which, of course, apply. Although we did not apply the gateway review process, each part of the departmental family, for example, Roads Service, Northern Ireland Water (NIW) or Translink, has its own project-management arrangements that have inbuilt risk-management arrangements. They also have arrangements for trying to drive best value in the projects that they undertake.
The Chairperson:
I will move on, as members have questions to ask on other sections of the report.
My next question is for Mr Armstrong. Paragraph 12 of the report states that the Office of Government Commerce (OGC) regards all projects as being eligible for the gateway review process. However, you consider that it should not apply to low-risk projects in the North of Ireland. Over the past years, the Committee has seen many cases where so-called low-risk projects have gone wrong. Why did you decide to limit the gateway review process to higher-risk projects?
Mr Des Armstrong (Department of Finance and Personnel):
Perhaps before I begin, I should give the Committee some background to public procurement policy in Northern Ireland; I think that that will be relevant to our discussion. When the Executive approved the Northern Ireland public procurement policy, they established a procurement board that the Minister of Finance and Personnel chairs. The board includes the permanent secretaries of each Department and representatives of the Central Procurement Directorate. Centres of procurement expertise (COPES) were also established. CPD works with those centres of procurement expertise on procurement policy and best practice before submitting proposals to the procurement board for endorsement.
Procurement is a devolved matter, and because of that, CPD and the COPES do not necessarily always take the OGC guidance. In fact, on some occasions, we could be criticised for doing that. The guidance that was produced for the gateway review process was particular to Northern Ireland. We believe that the focus was to use the review process effectively on medium- and high-risk projects and to rely on the established programme, project-management and governance procedures to ensure that low-risk projects were being properly managed. The gateway review process was part of a range of initiatives that we were using to improve the likely success of construction procurement on the projects in question.
The Chairperson:
OK. That has set the scene.
Mr Donaldson:
Thank you. Mr Priestly, paragraph 11 states that in identifying projects for review, the Department for Regional Development applied the guidance that DFP issued. That guidance requires all projects to be risk assessed. However, paragraph 16 of the report states that NIW policy was to not assess any conventionally procured projects, and paragraph 17 states that Roads Service assessed only those projects that were worth more than £10 million. That suggests that the guidance was not fully applied.
I hear what you said about risk assessing. If a project is ruled out by the introduction of some other criteria, for example, a £10 million threshold, do you get to ascertain whether it is risk averse? Do you think that that ruled out projects that should have been reviewed?
Mr Priestly:
The policy that existed at the time allowed Departments to set a de minimis threshold. The capital projects for which my Department is responsible are very large. As I said to the Chairperson, they are often not inherently risky. For example, an investment of £10 million in a road scheme will buy 1 km of dual carriageway with a grade-separated junction. Such schemes use standard technology and engineering; it is not inherently risky. It is not as though we are breaking new ground or doing anything that is particularly innovative.
Given the size and scale of the capital programme that we are taking forward, it would not be a good use of resources to subject all our programmes that are below that de minimis level to the gateway review process. We carry out proper risk management as part of our project-management arrangements.
Mr Geoff Allister (Department for Regional Development):
Paragraph 38 of the report states that:
“It would not be practical or economic to subject all projects to Gateway review irrespective of the level of risk”.
The OGC guidance behind that says that Departments may set de minimis levels. I emphasise that the full Roads Service risk-management process is carried out on all road schemes, regardless of whether they are above or below the de minimis level.
The vast majority of road projects are predictable in nature, and the associated risks are well understood. We have quite a track record of having delivered road schemes successfully, and we have skilled and professional staff. However, there is another important point to consider. Road scheme development is heavily regulated across the United Kingdom through one document in particular, the ‘Design Manual for Roads and Bridges’, which we follow. When taking forward our approach to the guidance, we looked at some of the schemes that we had recently worked on and considered how the scheme would look if we were to implement the guidance. For example, we looked at the scheme that we had carried out at Tillysburn, which was a road and rail bridge at Tillysburn, near Belfast City Airport. We were looking at a scheme that had a live railway running alongside it, it had one of the heaviest traffic roads in Belfast running over the top of it, it was in a very confined site and there were strict regulations over the acquisition of railway time, for example. That scheme scored as low risk following the risk-potential assessment on that scheme.
As Mr Priestly said, we also looked at the value for money and at what £10 million would buy. It buys about 1 km of dual carriageway, which is not a particularly lengthy. That is some of the understanding behind how we arrived at the de minimis level.
Mr Donaldson:
I am interested that you mentioned railways, because the Public Accounts Committee compiled a report on the upgrade of the Belfast to Bangor railway line and discovered that Translink was not even aware that there was such a thing as the gateway review process. Consequently, Translink had not subjected that project to the gateway review process. Its defence was that the Department had not passed the necessary information to it.
I accept your point about the setting of the de minimis level, but the gateway review process is not only about risk; it is about a lot of other things, such as the reviews that are undertaken. One such review is about establishing whether the scheme or project is good value for money compared with other competing schemes. As you will know, Mr Priestly, Roads Service has many competing schemes in its priority list on which you have to make decisions. Therefore, I think that the gateway review process could be valuable to the Roads Service in giving an independent view of your prioritisation of major capital projects. It would be valuable not only in the implementation of those projects, but in establishing, in the first instance, whether they are value for money in comparison with a competing project.
Paragraph 22 states that the Belfast sewers project, which is worth £136 million and described as one of the biggest civil engineering projects in the UK, was classified as low risk. I am amazed that a project of that size and complexity falls outside the gateway review process. Was that assessment reasonable?
Mr Priestly:
I have to go back to the point that I made to the Chairman at the start. The NIAO report seeks to set the capital value of a project in the context of its riskiness and the need to conduct a gateway review.
When Northern Ireland Water assessed the Belfast sewers project, there was no linkage between the capital value of a project and the need to carry out a gateway review. The Belfast sewers project was assessed in accordance with the methodology that was in place at the time. It did not receive a score sufficient to warrant a gateway review — it did not score higher than 30. Therefore, only internal reviews were necessary. Northern Ireland Water would point out that the project has been subject to considerable internal and external review.
You said that the scheme was a very complex piece of engineering. However, although it was very large, it was not particularly complex, nor was it considered mission critical. It was a piece of standard engineering on which tried-and-tested techniques from elsewhere in the world were used. We must get across the point that building roads or sewers is not new, innovative or risky in an engineering sense. They are very expensive, but they are not particularly high risk.
I would argue that proper project evaluations and project-management arrangements are in place to ensure value for money. Proper competitive tendering and procurement processes that are designed to drive value for money are implemented. Therefore, I am relatively assured that we are getting good value for money in those projects.
The Committee will have seen from the report that we have some difficulty with the gateway review process being described as driving value for money. The OGC states that it is basically a technique for the better management and delivery of programmes and projects. I am a gateway reviewer, and I have been the senior responsible owner (SRO) in receipt of gateway reviews. I have been a member of gateway review teams for medium- and high-risk projects. Not once has any of those reviews produced recommendations for better value for money.
Mr Donaldson:
I respect the opinion of someone who has been a practitioner. I get the impression that you regard the gateway review process as surplus to requirements and as a bit of a waste of time, given that sufficient procedures are already in place. That is the message that I have heard so far.
You have just made the very strong point that, to date, the gateway review process has not achieved anything to prove that something is wrong with the current system. That suggests to me that you have confidence in the current system and that deep down, you do not see the need for the gateway review process. That is not a criticism; it is helpful if the Committee hears that view. If that is the view in Departments, let us hear it.
Mr Priestly:
With respect, you are grossly misrepresenting what I said. I was dealing with the connection between gateway reviews and the claim that they drive better value for money. The gateway review process is about helping those who are involved in managing and delivering a project to ensure that it is likely to be delivered. It is a very helpful process.
Mr Donaldson:
Mr Priestly, with respect, you have said all along in your responses that the gateway review process is not needed for a large number of your projects. I heard you say that even for projects that are above your de minimis threshold, you have sufficient expertise, management and internal processes in place and that the gateway review process does not add significantly to those.
This is not a trap. If you are saying to me that, as a practitioner and a permanent secretary of a Department, you are not sure that the gateway review process adds value to the whole system, I am keen to hear it. From what you are saying, I am kind of hearing that view in some ways, but you seem reluctant to express it openly.
Mr Priestly:
That is not what I am saying, and, with the greatest of respect, I will not have words put in my mouth. The gateway review process has real value in helping Departments and project teams to deliver better the project in which they are engaged.
Mr Donaldson:
Why was it not used for the Belfast sewers project?
Mr Priestly:
At the time, we applied the policy that was in place, and we undertook a risk-potential assessment from which that project emerged as low risk.
Mr Donaldson:
Did you not feel that, even with that, the gateway review process should have been used? There is no requirement on the Department not to use the gateway review process in those circumstances. If you are such a fan of the gateway review process, why not apply it to a big project such as the Belfast sewers project, even if it was low risk? If you are saying that the gateway review process is about assisting the management of a project and to ensure that it delivers, the issue is not about risk; it is about project management. Why do you not use the gateway review process in more of your projects, particularly on a project that is valued at £136 million? If the application of the process is as beneficial as you suggest, why was it not used?
Mr Priestly:
I do not think that I should go back over old ground by saying that the capital value of a project should not be equated with the need to carry out a gateway review. Perhaps we can agree that the new policy establishes a link between the capital value of a project and the need to carry out a gateway review. In future, all projects with a capital value of £20 million or more will be subject to the gateway review process.
Mr Donaldson:
I remain to be convinced, Mr Priestly, but anyway.
Mr Armstrong, paragraphs 25 and 26 present a very confused picture of the Department of Finance and Personnel’s advice on public-private partnership (PPP) projects. Are you content that DFP’s gateway review process co-ordinator role is sufficiently resourced and that it has sufficient capability to provide central management of the gateway review process?
Mr D Armstrong:
A number of important developments have been made since the report was published. Central Procurement Directorate, with the approval of the Office of Government Commerce, attained the status of being an approved hub to carry out the gateway review process for Northern Ireland. Therefore, the responsibility and the management of the gateway review process now falls wholly within the remit of CPD.
We have established a centre of excellence on the wider aspects of programme management and project management. That centre of excellence also has responsibility for managing the gateway review process. That unit is now fully staffed, and the staff are excellent; they are experienced and enthusiastic about the gateway review process and programme and about project management in general. Staff with the necessary experience are in place to deal some of the issues that might arise from the issues that are highlighted in the paragraphs that you mentioned, and as part of the new guidance, we are supporting the process by facilitating conversations between the accounting officer and the director of CPD in instances where disputes may arise about risk assessments and other such issues.
Mr Donaldson:
Paragraph 31 reminds us that the Public Accounts Committee was given an assurance that all major projects would be subject to the gateway review process. Clearly, that is not happening in DRD. What proportion of capital expenditure is covered by the process across the Northern Ireland block?
Mr David Carson (Department of Finance and Personnel):
As has been said, the scope of the gateway review process has been increased through the adoption of recent guidance. For example, current guidance is applied to the investment strategy for Northern Ireland (ISNI). Applying the current guidelines, the gateway review process is used on 67% of projects for which contracts have been let. Benchmarking that we have done shows that more gateway reviews are carried out in Northern Ireland than in any of the other devolved Administrations.
Mr Donaldson:
Paragraphs 9 and 10 show that since 2004, DRD has spent £2 billion on capital investment but that only two projects have been subject to the gateway review process. Is it DFP’s view that the policy was intended to work in that way, or is DRD out of sync with other Departments?
Mr D Armstrong:
The responsibility for ensuring that the gateway review process is carried out falls to the senior responsible owners for the individual projects across the system.
Central Procurement Directorate’s role is to provide advice and support on the gateway review process to individual senior responsible owners. We, collectively, do not have a view on that, because as I mentioned, there are centres of procurement expertise across the system. The role of Central Procurement Directorate is to work with those centres of procurement expertise to deliver best practice and to monitor the arrangements that are in place. We have been doing that as part of the drive to improve construction procurement performance. The senior responsible owner must ensure that he has applied the guidance appropriately.
Mr Donaldson:
Are you content that DRD has done that?
Mr D Armstrong:
DRD has a number of centres of procurement expertise, which have just been reaccredited. That is probably a question for Paul.
Mr Donaldson:
It is a question for DFP.
Mr Carson:
The risk-potential assessment that kicks off the gateway review process is signed by the senior responsible owner, and accountability for that sits with the Department. It would be invidious for us to come in and try to second-guess that decision, because it is based on assessments of a number of factors, all of which influence risk. Those factors are: the capacity to carry out the project; the level of innovation that is involved; the levels of dependencies; and whether there are any commercial or funding arrangements that are unusual or unique in any way. The assessments are, therefore, complex and subjective, and it really would be invidious for us to come in, not having that level of knowledge about the project, to second-guess the SRO’s decisions.
Mr Donaldson:
Are you saying that given that Departments have the expertise and DFP does not, there is complete autonomy in determining how many projects they submit to the gateway review process?
Mr Carson:
No, the Departments do not have complete autonomy. In CPD, we have set up a network of practitioners to provide assurance on the completion of the RPA. Those practitioners have been involved in project management in Departments. It is our intention to test drill. The current guidance refers to the fact that risk-potential assessments must be validated under arrangements with the centre of excellence in CPD. That is a check to ensure that RPAs are completed consistently across the system and that there is a proper assessment of the risks.
Mr Donaldson:
Do you not have a view on whether DRD has submitted sufficient projects to the gateway review process?
Mr D Armstrong:
We do not have the detailed information on the project type. The OGC tested the policy that was in place in Northern Ireland as part of the accreditation process for CPD to become the gateway review hub. As part of that process, the OGC was fairly complimentary about the policy aspects that were in place in Northern Ireland and on the take-up of the gateway review process. That reassured me that the policy that was in place was effective.
We do not have sufficient knowledge of the programmes and projects in the system to be able to determine risk-potential assessments to the extent that we would be able to challenge them and say that something is wrong and that it must be done another way. CPD does not have a role in investigating the risk-potential assessments of other centres of procurement expertise or Departments, nor does it have an audit role. However, it has a role to look at how best practice in Northern Ireland compares with that in other jurisdictions. The OGC’s assessment process and the fact that it gave us accredited status for the gateway review process are important reassurances to us.
Mr Donaldson:
Apart from the work of the Audit Office, is there not really any other system in place to police what Departments are doing with the gateway review process?
Mr D Armstrong:
Since the report has been published, we have been working on policy development, because that is what we do. We examine a review policy and how it has been implemented. The new guidance gives us a greater opportunity to produce management information on projects that are being brought forward and on their risk-potential assessments.
As to the monetary value, the gateway review process starts with a risk-potential assessment being put in place. If one were to take the riskiest project imaginable, the risk-potential assessment score would be 97; four of those marks would be allocated to costs, but 12 marks would be allocated to issues such as managing business process change.
The gateway review process needs to be seen in the context of an overall governance arrangement. That is the way that it was being taken forward in construction procurement. We had a policy framework for construction procurement, and the gateway review process was part of that assurance mechanism.
Mr Donaldson:
Paragraph 4 of the report states that DFP reported savings through the gateway review process of £25 million against the Department’s value-for-money target, yet paragraph 36 states that you saw no recommendations coming out of gateway reviews that referred to value for money. I am curious as to which it is. Do you feel that you were making value-for-money savings from gateway reviews?
Mr D Armstrong:
The general approach to calculating value for money is fairly technical. There are several ways in which one can generate value-for-money savings. We mentioned applying the procurement process properly and ensuring that it is managed through a centre of procurement expertise. That offers greater reassurance, but as to value for money, OGC worked with the National Audit Office to come up with a calculation that could be applied to gateway reviews. We took the benefit of that calculation.
We also produced local guidance on the impact of collaboration and aggregation of contracts, reducing the transactional costs of the procurement process and applying proper programme and contract management. Savings could be identified on a project on that basis. The guidance at that time suggested that if we did not apply the gateway review process and gather in savings from OGC, we could use some of the other mechanisms to generate a calculation of value-for-money savings. It is a reasonably complex approach.
Mr Donaldson:
Generally, do you feel that the gateway review process can deliver value-for-money savings?
Mr D Armstrong:
Points were made about the recommendations that come back, in that they do not specifically state that they suggest value-for-money savings. The recommendations that come back from the gateway review process are about successful overall delivery of the project. The gateway review process is part of our assurance mechanism. When applied appropriately with our guidance, we believe that it will improve the likely success of projects. Therefore, falling out of that, we reckon that those processes will help to secure value for money. There is a bit of science in that calculation.
Mr McLaughlin:
Good afternoon. Mr Armstrong, paragraph 17 of the report states that Roads Service does not carry out risk assessments for projects that are worth less than £10 million. You touched on that point in your evidence. That tells us that Roads Service has, in effect, always operated a de minimis threshold. Do you agree that the new guidance on de minimis limits in guidance note 01/09 changes the policy to fit the previous practice?
Mr D Armstrong:
Policy development in Northern Ireland has always been about looking at how policy has been developed and how it is being implemented. It is also about learning from experience on the ground. The concept of linking risk or the gateway review process to financial value was clearly an issue that we needed to address.
I explained that in the risk-potential assessment, the value scores quite low as a proportion. To try to move the situation forward, we will test the impact of bringing in a financial threshold, but we have approval from the procurement board to review that decision in 12 months. At that point, the procurement board will take a view as to whether the threshold should be moved up or down for the projects that have been subject to that new guidance.
Mr McLaughlin:
I want to develop that theme. Paragraph 40 states that in considering CPD for hub status — some reference has been made to this point already — the Office of Government Commerce recommended that CPD should develop capacity to challenge senior responsible owners on the validity of the assessment scores. What arrangements have been put in place to respond to that? Do you believe that DRD has developed the capacity to robustly challenge the validity of projects that are being assessed or categorised as low risk?
Mr D Armstrong:
I will pass that question to David; he can give you further detail. However, my answer is that DRD now has the capacity to address the issues that the report raised.
Mr Carson:
We have set up portfolio leaders; that was one of the recommendations made after the assessment for hub status. Those people have particular knowledge and expertise of the categories and areas of expenditure in the system. The centre of excellence in CPD will refer RPAs to portfolio leaders so that they can review the way in which they have been scored and assessed. If it appears that an assessment is not in accordance with what the portfolio leader expects to see, he will discuss that with the SRO to try to get agreement on the risk weighting for the project. If they fail to agree, there is an escalation process that stops at the permanent secretary and Des Armstrong. Those are the arrangements that have been put in place, and they will be reviewed after a period to see whether they are working effectively.
Mr McLaughlin:
That is helpful. I do not expect you to have the precise details available now. Is it possible for you to come back and provide the Committee with a number of examples of where that mechanism has responded to issues and where the system was proven to be effective in identifying risk that might otherwise have been overlooked?
Mr Carson:
Yes, certainly. We will give you some concrete examples of how that process operates.
Mr McLaughlin:
Guidance note 01/09 states that low-risk projects must be subject to an internal peer review on the principles of the gateway review project by a team that is independent of the project. Mr Armstrong, what arrangements have been put in place to ensure that those reviews are being carried out regularly and properly?
Mr Carson:
Again, I know the details, so I will answer that question. The centre of excellence in CPD is establishing sub-hubs in the Departments that are based on the system. The idea is that those sub-hubs will provide assurance on peer reviews for low-risk projects. At the moment, we are putting in place memorandums of understanding with the Departments that will set out the various roles —
Mr McLaughlin:
Before you continue, is that a continuation of pre-existing arrangements, or is it, in fact, a response?
Mr Carson:
It is a formalisation of practice. The practice may vary from Department to Department, and it will be more mature in some Departments than in others. However, the sub-hubs will formalise that process and provide greater visibility to the centre of excellence of the fact that that process has taken place.
Mr D Armstrong:
The procurement board has instructed us to review that guidance note after 12 months. We are, therefore, collecting management information that will allow the procurement board to carry out that assessment.
Mr McLaughlin:
Was that review prompted by the fact that the guidance note has not been applied uniformly and that there are problems with its application?
Mr D Armstrong:
Part of the assurance mechanisms that we have been pushing with the Departments is to ensure that they have documented project-management procedures in place.
Mr McLaughlin:
I think that Mr Carson used the expression “subjective assessment”, but I believe that the process should be much more objective. That may be one of the problems about which we are concerned. Therefore, whether the necessary reviews take place depends upon who makes that judgement.
Mr D Armstrong:
That is where the role of the centre of excellence comes in. It will provide advice and support and build up that body of expertise. That is what happens under the new arrangements. We were concerned that we did not have sufficient expertise around the controller to allow us a challenge function of the type that would add value. We cannot go back to the SRO and know all the detail of the project that they have. However, arrangements are now in place to provide the right support and advice, and a referral mechanism is in place in case some issues need to be resolved.
Mr McLaughlin:
I am sorry for interrupting you, Mr Carson. What were you about to say?
Mr Carson:
In addressing your concerns about the subjectivity of the assessment process, I wanted to say that the risk-potential assessment form is being redesigned, and it recognises that the process is subjective. However, that form is designed to assist the senior responsible owner to come to a robust assessment. It does so by first, looking at the consequential impact if the project fails to deliver. That is assessed, and examples are given of what would or would not constitute an impact. Secondly, it looks at the complexity in a number of areas that would affect the project. Again, it would give examples of high-risk and low-risk situations. The SRO is asked to score the project in each of those areas and to arrive at an overall risk assessment for the project. The assessment is no longer a spreadsheet-based system with scoring; it exposes the SRO’s judgements more explicitly. There is greater accountability, but the decisions and judgements rest with the SRO.
Mr McLaughlin:
As well as being captured, would that information be reflected across the range of projects that a senior responsible owner oversees in the span of his or her career? Obviously, the decision is taken at a very senior and responsible level, hence the job description. However, the Department is vulnerable if that individual demonstrates inconsistency or a different standard of judgement.
Mr Carson:
Absolutely.
Mr D Armstrong:
Yes, I think —
Mr McLaughlin:
Does that comparator also exist as an evaluation tool?
Mr D Armstrong:
The role of the senior responsible owner is well and truly outlined in the new guidance. The role is critical in that it has an impact on successful delivery, and it provides the leadership that is needed for the rest of the project. We recognise that SROs may need some additional support, guidance and training.
Mr McLaughlin:
I suppose they would also need protection.
Mr D Armstrong:
Yes. We have been working with the centre of excellence to increase the understanding of how the whole gateway review process and project management in general provides lessons to be learned from projects to the wider programme and project-management community. Part of the process will drive the upscaling and sharing of experience.
Mc McLaughlin:
Procurement guidance note 03/09 from the Central Procurement Directorate states that a gateway review is now mandatory for projects with a value of more than £20 million. That will bring in many of the bigger projects, such as the Belfast sewers project, which is listed in figure 3 of the Audit Office report. That project, as you know, was previously assessed as low risk. Have you estimated how much of the total capital spend under the Programme for Government will be subject to review under the updated guidance?
Mr Carson:
Yes, we looked at the ISNI programme, but as I said, some of the details have yet to be tied down.
In value terms, two thirds of projects for which contracts have been let will be subject to the gateway review process under the £20 million rule. That amounts to around £5 billion worth. Other projects may fall under the process because they are high risk per se, but there is substantial coverage for that.
Mr McLaughlin:
That is useful.
You referred to what I would like to examine next. You gave some examples of what £10 million can achieve and deliver. You talked about the sewers project and its use of well tried-and-tested standard technology, and I understand that example. However, are there other examples of projects that have a budget of in and around £10 million that demonstrate risk but do not score highly on the risk-potential assessment?
Mr Priestly:
Do you mean from a departmental perspective?
Mr McLaughlin:
I want to establish whether the risk-assessment process covers all eventualities.
Mr Priestly:
Mr Armstrong explained that because of the way that the gateway review process has operated until now, the capital value of a project scores quite low. Elements that score quite high are anything to do with innovation or with re-engineering business processes. That is why big IT projects are much more risky than, for example, some of the big projects that DRD would work on.
Mr D Armstrong:
If I were to give you another example of construction projects, I would discuss those that are not being carried out through the centres of procurement expertise or that are in the experience of Roads Service or Northern Ireland Water. For example, the refurbishment of the Ulster Museum was a difficult construction project in that it was designed to reshape the internal workings of the building. It also presented changes to the business in the way in which exhibitions can be brought together. That also meant that there were changes to the necessary storage arrangements. That type of construction project can fall into that category.
Mr McLaughlin:
Over the past five years, has our experience of projects in and around the budget threshold of £10 million demonstrated that those projects carried a risk that would have meant that they did not qualify for a gateway review process? There is a range of such projects, but I am more interested in those that demonstrated risk through their delivery.
Mr Priestly:
I cannot name any. However, figure 3 in the report lists two such projects. The first is the customer billing system, which was designed to create a new billing system for the introduction of water charges. Members can see that that project got a risk-potential assessment score of 47. The capital value of that project is not listed in figure 3; however, it is not as great as that of some of our other capital projects. Second is the mobile work-management system, which introduces a new computer-based system of allocating jobs to people in Northern Ireland Water. That is inherently risky, and, therefore, was subject to a gateway review. However, its capital value is probably quite low.
Mr Allister:
Figure 3 includes two road schemes, costing in and around £10 million, which are now on the ground. One is the M2 Ballee Road East, where we are extending the M2 underneath the Larne Road roundabout to bring it on to the existing dual carriageway. That cost approximately £13 million, and its risk-potential assessment score was 21. The other scheme is the Cairnshill park-and-ride site, which is being developed off the Saintfield Road. That cost £10 million, and again, it was low risk. Those are examples of what approximately £10 million can achieve and what the associated risks could be.
Mr McLaughlin:
Can you tell us whether the Department has carried out post-project evaluations on some of those schemes and whether that demonstrated that risk existed but had not been provided for under the current arrangements?
Mr Priestly:
I assure you, Mr McLaughlin, that arrangements are in place in the Department to ensure that all projects complete post-project evaluations. I cannot put my hand on my heart and say that I see and read all those evaluations. However, I have assurances that arrangements are in place to make sure that they are done. If they are not done, it is reported to me. As accounting officer, I try to make sure that post-project evaluations are done. I do not know whether Mr Allister, having seen a post-project evaluation and what was done with it, can illustrate the process.
Mr Allister:
I cannot add to that. The M1 Westlink project and the M2 are worth drawing to your attention. However, I am not referring to a formal post-project evaluation as such; that will follow. The Northern Ireland Audit Office came in subsequently and investigated those projects last year. It reported back favourably that Roads Service got a value-for-money deal. Elements of good project management were applied, sufficient governance structures were in place, and we learned lessons from the design, build, finance and operate (DBFO) model that we were following; that is, the Highways Agency’s model. Although it was pleasing to get that assessment, the important thing is that our second package is following on using the same governance arrangements. We have already learned some of the lessons from that, and we are applying them to the new projects that we are taking forward. Such projects include the A5 Derry to Aughnacloy road, on which a gateway review process will start in the next week or so, and the A6.
Mr McLaughlin:
Having learned those lessons, does it mean that there is a likelihood that the gateway review process will be applied to the component elements of those major schemes?
Mr Priestly:
Do not forget that the policy has just shifted. Any project that is worth over £20 million is now subject to the gateway review process. Therefore, such schemes are subject to the gateway review process.
Mr McLaughlin:
The Audit Office report identified a vulnerability where the gateway review process may have been applied to a major project that had a number of complex and interlinked elements. I am questioning whether those elements would now be subject to individual gateway reviews.
Mr Priestly:
Earlier in the discussion, I wanted to draw attention to such an example. I refer you back to figure 3 of the report, which states under the heading “Translink”:
“New Trains 2 — trains
New Trains 2 — Adelaide facility
New Trains 2 — Programme”
They are all the same. The programme is subject to the gateway review process, which means the procurement of the trains and of the Adelaide facility.
Mr McLaughlin:
Are they going to be separately subject to —
Mr Priestly:
No. The programme is subject to the gateway review process, which means that the procurement of the trains and the Adelaide facility are part of the programme. They are two projects within a programme; therefore, the New Trains 2 programme is subject to the gateway review process. That means that the procurement of the trains and the procurement of the Adelaide facility are covered. Are you with me?
Mr McLaughlin:
Yes; they are covered by an overall gateway review process.
Mr Priestly:
Yes.
Mr McLaughlin:
That is what concerns me. Normally, those reviews take three or four days to do. Similar to the concern that Jeffrey Donaldson teased out, my concern is whether there is the possibility that Translink would justify its own bespoke gateway review process.
Mr Priestly:
I will give you some of my experience as a gateway reviewer in doing a programme with a number of separate projects. It can be done. Do not forget that you are really seeking to increase the likelihood that the programme and the projects in it will succeed. However, it is possible to get quite a lot of work done and to do a whole programme in a single gateway review.
Mr Allister:
From a practical point of view, and from looking at individual projects — for example, our two DBFOs, which have been seen as one programme and which have had a risk-potential assessment as such carried out — it is important to examine what the gateway review process looks at in the various gateways. They look at: whether a tendering strategy reflects business requirements; whether there is an awareness of the market; whether there is good practice in procurement and changes to business needs; whether funding is available; and whether adequate financial controls are in place. Those would apply equally through the combination of the schemes that we have, for example, the PPP packages.
It also looks at whether the tendering process has met its objectives. The same tendering process is followed through. If one looks at our PPPs, they have the same governance structure, the same management team — we use the Strategic Investment Board as advisers — and the same programme steering group looks after them. There are a lot of commonalities. Therefore, it is appropriate that they are looked at as an overall programme.
Mr McLaughlin:
You have been very indulgent. However, to finish, I have a question on the post-project evaluations. Can you provide the Committee with some detail of the range and scope of the post-project evaluations over the past three years, for example, that would cover the transitional period? I am particularly interested to see whether post-project evaluations called into question whether the risk assessment — the gateway review process — could or should have been applied in circumstances where they were not. I am particularly interested in lessons learned.
Mr Priestly:
We will do that.
Mr Donaldson:
I have a query on the M1 Westlink upgrade project, which I noticed was not on the list of projects that had been considered for the gateway review process.
Mr Priestly:
That is a PPP package.
Mr Donaldson:
OK. Does that come under either PPP package 1 or 2? Clearly, the decision was made that the project was not risky enough to warrant a gateway review. Might that process have helped to identify the problems that later developed with the drainage system in the underpass at Broadway? Would the benefit of independent assessment alongside internal procedures have been of any help to the Department or to Roads Service?
Mr Priestly:
I will ask Geoff to answer that, as he has lived and breathed that project.
Mr Allister:
I have been closely involved in the review of the Broadway underpass incident, and there is nothing that I have seen in that report that would lead me to think that a gateway review would have covered the issues that arose or that were the main causation factors for the flooding. The risk assessments that we would normally carry out were done.
Mr Priestly:
I would step back and say that when Roads Service originally designed the scheme, it wanted to have an overpass, not an underpass. That was when risk was built in. The reason that it flooded — grilles were blocked — is prosaic rather than being the result of any fundamental design problem.
The Chairperson:
Would any other members like to ask a question?
Mr McLaughlin:
Yes. I understand that more than just blocked grilles caused the flooding, although that was a major factor. Some design modifications were done at the behest of the contractor, and I also understand that the contractor had to take responsibility for that. Change of design while the contract was proceeding may have been good or bad, but on this occasion, we ended up with an unplanned open-air swimming pool.
Mr Priestly:
I think that I am right in saying that there was no fundamental change in the design.
Mr McLaughlin:
It was a causation factor.
Mr Priestly:
The lead-up to the culvert was raised so that there was less likelihood of the water spilling out over the culvert in future.
Mr Allister:
Some of that was to have been done under the contract and had not been done when the incident occurred.
Mr McLaughlin:
My main point is that there was a modification that was presumably signed off on and agreed. It was bound to have affected the initial risk assessment. It was a question that someone had to answer; was it asked and answered?
Mr Priestly:
I do not follow you. My understanding is that the engineering design was to withstand a one-in-100-year flooding event. The onus is on the contractor to produce a scheme that could resist that event. If the contractor had not produced such a scheme, the risk is with him to modify it so that it meets that need. We do not pay.
Mr Allister:
In our processes, and when we saw the results of our risk management, we took advice from the Department of Agriculture and Rural Development about what the design standard should be. That was clearly documented. The design risk was passed to the contractor to deliver against.
Mr McLaughlin:
I am acutely conscious that some of my colleagues wish to speak, but presumably, at the point at which final contracts were being signed, there was agreed design criteria or specification, so everyone knew exactly what they were going to do and what they were to be paid for. My understanding is that, subsequently, the contractor came up with a different layout and design for the water run-off, which was a contributory factor to the flooding.
Mr Allister:
Are you referring to the change from taking the culvert directly over the top to the proposal to take the culvert up the side of the motorway and underneath it?
Mr McLaughlin:
The confusion might be my fault; I am not particularly interested in the detail. I am talking about the broad principle. On day one of a contract, people sign off and know what they are expected to build. There cannot be an unusual set of circumstances whereby suggestions or modifications are made, agreed and signed off. A cost factor might be involved that is agreed and signed off. In this case, I am trying to establish whether those modifications exposed the fact that the original risk assessment was void. It was of no benefit, no help with the value-for-money question and of no assistance in identifying the potential for risk.
Mr Allister:
As I said, the design change through which the culvert was moved from going straight across the junction to alongside the motorway, underneath it and down the far side did not introduce any additional risk to the project. It still had to be designed to satisfy the requirements of the Department of Agriculture’s one-in-100-year storm flood policy. During the process of identifying the risks before the project began, that design standard would have been set very clearly and the risk would have been transferred to the contractor. It is up to the contractor to take the risk on board and design and build with it in mind. It is the role of the client — Roads Service in this case — to monitor delivery.
The Chairperson:
We have the opportunity to pose further questions later. You will be glad to hear that the first session is over, Mr Priestly, but you will not be so glad to hear that you are going into another session straight away. We know that you are changing your team, so we will give you a bit of time to get ready.
Mr Jim Drennan from DFP is here today. He will be retiring from his current role after seven years. Over the years, Jim has worked closely with accounting officers to prepare them for evidence sessions in Westminster and here. On behalf of the Committee, I thank you for your time and effort over the years, Jim. Good luck.