COMMITTEE FOR REGIONAL DEVELOPMENT
OFFICIAL REPORT
(Hansard)
Inquiry into Managing Data Risks in Northern Ireland Water
17 June 2009
Members present for all or part of the proceedings:
Mr Fred Cobain (Chairperson)
Mr Jim Wells (Deputy Chairperson)
Mr Cathal Boylan
Mr Allan Bresland
Mr Willie Clarke
Mr John McCallister
Mr Alastair Ross
Witnesses:
Mr John Mills ) Former Independent Reporter to Northern Ireland Water
Mr Ewen Bradley )
Mr Chris James ) Steria/Crystal Alliance
Mr James While )
Mr Jon D’Arcy )
Mr Dominic Mudge ) KPMG Ireland
Mr Arthur O’Brien )
The Chairperson (Mr Cobain):
Good morning, John. You have 10 minutes in which to make your presentation, which will be followed by about 15 or 20 minutes of questions.
Mr John Mills (Former Independent Reporter to Northern Ireland Water):
Thank you, Chairman. Good morning, everyone. I am grateful for the opportunity to meet the Committee and provide it with evidence as part of its inquiry into managing data risks in Northern Ireland Water. I appreciate the opportunity to outline my evidence before I answer questions.
I provided the Committee with some written evidence, which I will use as a basis for my presentation. In that evidence, I touched on the role of the independent reporter. I also provided an outline of the key submissions that were audited by the reporter, and I touched on confidence grades, which we use to judge the quality of the data that we see. In addition, I provided some views of the management of key data risks in Northern Ireland Water.
First, I will touch on my background, because the reporting is an independent role that is carried out by an individual. What you see it what you get: it is really about the individual. I am a civil engineer. I have more years of experience than I care to remember of working in England, Wales, Scotland and Northern Ireland, mainly in the water industry.
The Committee will find that, by background, most reporters tend to be civil engineers. It really is a reflection of the development of the water industry and the development of the reporting role by Ofwat, which looked for people who could primarily comment on large capital programmes.
My experience includes being a design engineer and supervising construction on site. I have carried out a range of feasibility studies. I have reasonable expertise in process design, project appraisal, and in the design and management of fairly large teams delivering capital programmes.
I have also worked extensively from about 1992 in a reporting role in England, Scotland and Wales. From August 2007, I was the independent reporter for Northern Ireland Water. As you may know, I now work for the Utility Regulator, where I am the manager for network regulation. I am speaking to you in my personal capacity as a previous reporter; I am not representing necessarily the Utility Regulator’s views or those of my former employer.
What is a reporter? Reporters are billed as being independent auditors, commentators and certifiers, which is a fairly nondescript and broad title. They report to the Utility Regulator on the regulated activities of water companies. My costs were paid by Northern Ireland Water, so the role of the reporter is an additional cost for Northern Ireland Water. The reporter’s primary duty of care is to the Utility Regulator, to allow it to fulfil its statutory duties. However, the reporter also has a duty of care to Northern Ireland Water. Therefore, the reporting role is not there to do down Northern Ireland Water but to report independently and fairly on what it does and to describe what it does to the Utility Regulator to assist its understanding and decision-making.
Audit scopes provided by the Utility Regulator define fairly clearly what the reporter does. Therefore, we act on the instruction of the Utility Regulator; we are not independent in that we cannot do what we want but must act on those instructions. The work is carried out mostly with the company. Reporters meet company staff at all levels to discuss what they do and the methodologies used to put submissions together, and we carry out sample audits on the data.
One of the points that I want to make is that, although the role is broad in that we cover a very wide scope of work, we look at individual topics that are quite detailed in their content. We do that by carrying out very limited sample audits with a focus based on our own experience and knowledge of what looks and feels right.
Another point that I want to make is that we have a duty to keep things confidential. We have a duty of care to the company to keep the data that we receive from it confidential. That is to protect the company — in particular, its commercial interests — and its suppliers and customers.
I have highlighted three particular submissions to show the type of work that we do. The annual information return is a fairly comprehensive return of data setting out performance that the company makes to the Utility Regulator each year. It includes a wide range of measures from asset data to information on low-pressure, flooding, customer numbers, the water balance, and also on the company’s progress on delivering its expenditure programmes. The reporter looks at all of those elements.
We also look at the interim principal statement and scheme of charges, which is the submission that the company uses to set its tariffs. That is also used in the context of Northern Ireland to determine levels of required subsidy. In that context, the reporter looks at information on customer numbers, volumes of water taken and drainage volumes. There is a general assumption that the reporter looks at technical data; not necessarily financial data. However, there is certainly a crossover in the role to look at some of the allocation of financial information as well.
We also look at business plan submissions, which are submitted to the Utility Regulator, in this case, every three years, but, subsequently, probably every five years. The reporter’s primary role in those is to look at the investment programme to understand whether proposals are reasonable and are linked to outputs.
I have outlined some of the work that we do on confidence grades in my submission. The confidence grading of data is something that the Committee might be interested in with respect to how we judge and report the quality of data. It is something that, perhaps, could be used in the future to judge improvements in quality of data, or, at least, form part of that judgement.
I want to touch on some of the key risks that I have seen and see developing. First, there is an inherent risk in an emerging company using legacy data for new things for the first time. Things will go wrong with that data because it has never been used for that purpose before and it could be found wanting when it is applied for a new purpose. The company is currently developing new data systems, putting in place new processes and formalising existing systems. Those actions will probably continue to expose weaknesses in data, which will have to be addressed over a period of time.
Work in respect of the reapportionment of revenue triggered the investigation and other discussions. Poor-quality data will result in the inaccurate calculation of revenue of tariffs. There is also the issue that if the company does not get its numbers right, it will not retain sufficient revenue to sustain its business.
One of the key roles of the reporter is to look at the comparability of data between companies. That is used by regulators to assess and challenge investment plans and company performance. That assessment cannot be robust if the data is not robust. In relation to the process through which we are currently working, inadequate data means that the company could have difficulty with producing investment plans. There is a role of understanding the quality of the information that is put forward in investment plans and how that relates to data issues.
In relation to the overall data issues, there is a risk that the issues that are being exposed, which need to be managed, risk a loss of confidence in stakeholders and the ability of Northern Ireland Water to manage its business and deliver key services. That is a risk that we should avoid. We should be careful that we do not lose confidence in the company.
I previously touched on confidence grades. They have been used in Scotland, Wales and England, and things are not perfect there. Confidence grades have improved over time as data has developed and improved. Even after a period of 20 years of regulation in England and Wales, not everything is an A1 confidence grade. It is probable that not everything can be an A1 confidence grade.
If one looks at the history of other companies in England, Wales and Scotland, it is apparent that they have faced similar risks. Those have been identified in the past and they have been managed through the regulatory process. There are processes in place in Northern Ireland through regular annual reporting; the undertakings given by Northern Ireland Water; some quarterly reporting by the company on key data issues; the publication of annual reports by the utility regulator; and the development and analysis of the PC10 business plan, which works towards developing and managing those risks.
The Chairperson:
Thank you very much. I have a question about the quality of data that you receive and on which you base your projections. From an early stage, the Committee has been troubled by the strategic business plan that was originally drawn up. When the company moved from being a public company into the realms of Northern Ireland Water, a lot of the financial projections were based on that plan.
Mr Mills:
Yes.
The Chairperson:
After a number of evidence sessions and a number of years down the road, I think that everyone agrees that that plan was wildly inaccurate.
Mr Mills:
I suppose, like many things, that the plan is probably good in parts. There is an issue around the revenue and its allocation, which has been a key difficulty. However, in some respects, if one looks at the plan to deliver 900 km of water mains or to deliver a number of sewage-treatment works and performance improvements, those aspects of the plan are being delivered.
I would not like to comment as an expert at the moment, but, broadly, as far as I can see, the budgets were set at that time. In relation to the plan’s ability to set out a list of deliverables and to go about delivering them, it has not been that bad. It has not fallen to pieces during delivery. The issues have been around the revenue side of the plan in relation to data. If the Committee asked what went wrong with the data from the business plan, I would have difficulty in answering because I was not there. However, one of the issues is probably around data that is selected for a particular purpose in the public sector that has not been used for much in relation to numbers of customers.
That is not an issue for a public-sector organisation. The data is suddenly being applied to the allocation of revenue and billing, which is quite an important area. As data systems develop, the data that has been collected for one purpose is suddenly pulled together and used for something else, and it takes a period of time for it to become robust and reliable for that other use.
The Chairperson:
I take your point. The Committee was concerned that individuals who were involved in that had also been involved in water companies across the water that went through exactly the same process of changing from a public company into a private company.
I also take your point that it is easier to project data on capital investment than on billing and other issues. The best illustration of the data issue is that the recommended level of debt for Northern Ireland Water was 5%. We have pressed Northern Ireland Water on that on a number of occasions. There is no possibility of the company achieving that forecast. Companies across the water, which are far more sophisticated in the area of billing and addressing non-payment, run at a level of debt of 7% or 8%. Those companies have been in private hands for around a decade. Northern Ireland Water is running with data that is imperfect. How does one work from a financial perspective when one knows perfectly well that the data to which one works is inappropriate?
Mr Mills:
One should not work in such a way.
The Chairperson:
Northern Ireland Water is working that way. The Committee’s difficulty is that those stand-out issues have a direct effect on getting revenue to carry out Northern Ireland Water’s other work. The company still runs with that recommendation, and it must know perfectly well that it is not valid.
Mr Mills:
I cannot really answer that, because I have had only two days to look at the business plan to consider what it says on that specific issue. Such issues can be dealt with by looking at comparative companies. Perhaps one might assume and realise that Northern Ireland Water is at the upper range of debt because of its circumstances and the introduction of billing.
Mr McCallister:
John, you provided an overview of the functions of the independent reporter. You have huge experience of the industry. What are the limitations and constraints, if any, on the independent reporter?
Mr Mills:
Some of the limitations are related to the breadth of what we do. We cover a wide range of topics, on which a small team of people and I have to work. Each of those topics is incredibly complex, and they include masses of data. Unless one is prepared to provide an endless budget and endless time, reporters must look across topics that give breadth. It is not that we look at topics superficially, but we must look at topics at a high level to understand what is going on. We do that by using our experience, and that experience targets us towards areas where, as you said, Mr Chairman, things do not look right. We base our challenge on that to understand the company’s plan.
The timing of audits is a challenge for us. The timescale between the company having developed a realistic set of proposals to submit to the regulator and the time that it makes its submission is always limited. Reporters’ reports go in at the same time as the company’s submission. We work quite hard for a period to understand what is going on, and little time is available to do that.
Maintaining one’s experience is an issue. As I said, it is experience that allows one to do the work. Regulation is a complex subject. It is impossible to be an expert on regulation itself and about 16 other disciplines. Therefore, one must develop and maintain knowledge outside the role purely of regulation.
A loss of independence, or regulatory capture, is a challenge, because the work is carried out with people face to face. A good working relationship, independence and impartiality must be maintained.
Frankly, one of the weaknesses of the role is that reporters can expose issues and make recommendations but cannot fix things. Reporters have an audit role. It is for others in the company and for the regulator to pick up on and to fix those issues.
Mr Bresland:
What are the key submissions audited and what are their functions?
Mr Mills:
I touched in my evidence on some of those submissions.
First, there is the annual information return, which is usually submitted around July and sets out the company’s performance in the previous year. The annual information return fulfils a couple of functions. It provides the regulator with data that can be used to compare the company with others. It also allows the company to report, and the regulator to assess, the company’s performance and progress in improving levels of service and in delivering its intended investment programmes.
I touched next on the annual scheme of charges report, which is delivered around January and is used to set tariffs for the following year. For the reporter, that serves as an update on key parameters, customer numbers, water volumes and so on, that are used to determine tariffs. The reporter’s role is partly to consider whether the data is consistent with what has been said previously.
I have also commented on the company’s procurement activities. A procurement plan is submitted about every 18 months, but it is not actually about the efficiency of procurement; it is about whether the company, in its procurement policies, is acting reasonably and whether it maintains control of its assets. I suppose that the example is Welsh Water, which contracts out its entire operation and has a total workforce of 150. Ofwat was concerned about whether the company retained control of its business because of the way it has procured services.
The final matter that I touched on was the business plan submissions, which allow the company to make presentations for its investment proposals and its business over a three-year or a five-year period. The reporter’s key role in that is to examine the investment and costs and to link that investment to outputs.
Mr Ross:
What confidence grades are attributed to the reported data? Who confers those grades? What role has the reporter in assessing or commenting on the grades?
Mr Mills:
The confidence grade comprises two parts — from A to D and 1 to 6 — that form a measured judgement of reliability and accuracy. A to D defines how reliable the data is. A means that the information came from a well-established corporate data set that is well-used, well-audited and well-policed. D denotes an assumption or an educated guess. In the accuracy range 1 to 6, 1 is better than plus or minus 1%. At the other end of the scale, 6, with a margin of error of 50% to 100%, is very inaccurate. Certain combinations are not allowed. One cannot have an absolutely perfect data set that provides an accuracy rating of plus or minus 100%, because that is a nonsense. Nor can there be a guess with a range of plus or minus 1%. Therefore, certain sets of data are excluded.
The data is initially assessed by the company. The point is that it is the company’s report; it is its own submission. The company and the reporter may disagree. A key part of what the reporters do is to audit and assess those confidence grades. A company rarely disagrees with a confidence grade submitted by a reporter, partly because doing so is not in its interest.
The confidence grades sound surprisingly arithmetic and rigorous. There is an element of judgement in them, because often you are making a judgement on unknowns. You know what data has been reported and make a judgement on how much is in the data set. They are useful in that they are a process of capturing and understanding confidence, and they are a way of communicating it. They are useful in developing and reporting improvements in confidence data, but they are only one part of it, because, if you take a movement from a 3 to a 2 accuracy grade, you are talking about moving from between 5% and 10% to 1% and 5%. There is a lot of difference between moving from 6% to 4% and moving from 9% to 2%. Therefore, they are quite a blunt instrument, which is why they should not be considered just in isolation.
Mr Wells:
In your written evidence and in your introduction you outlined your views on the management of key data risks in Northern Ireland Water. Obviously, that is absolutely crucial. Will you expand on that? You had a great deal of time in your introduction to deal with it, and we really want to hear your views on it, shooting from the hip, as it were, because the issue has bugged our relationship with Northern Ireland Water and the Department since its inception.
Mr Mills:
My role as a reporter is not necessarily to criticise, but to expose the position and, when you know where you are, you can start to make improvements. There are a number of issues about managing data risk, including ensuring that they are exposed, and, if you know where you are, you can manage the associated risks. There are two aspects to the process of managing data risk. The first aspect is to manage the improvement process and to be realistic about the timescales and the expectation of that improvement. The second aspect is the level of accuracy that you need or that is possible.
Not all data will improve rapidly. Data that is subject to improvement is primarily data that is recorded and reported each year. If the right systems and management practice are put in place, things will improve quite rapidly. However, even doing those two simple steps can take up to four years. It is not instantaneous. Other sources of data will not improve for many years. For example, England and Wales benefits from about 20 years of records of flooding data. Therefore, they can see whether properties truly are at risk of flooding. If you start from a bad data set, it takes some time to understand and to build that understanding.
The second point about managing data risks is that Northern Ireland Water has a service to provide, which must continue, irrespective of the quality of today’s data. My concern about the issue of confidence in the company is that the service must continue and the PC10 process must continue, otherwise the investment plan stops, and that is not a tenable position. Therefore, my concern is that we need to expose the data risks and understand how confident we are of the data. Then we need to respond in an appropriate way to the way in which Northern Ireland Water is funded, supervised and managed.
Mr Wells:
You have been quite prolific in this field. In fact, you are the acknowledged expert in it. You have made submissions to various audits and report, including the procurement plan, the annual information return 2007-08, the scheme of charges for 2008-09 and the report on the PC10 business plan. Therefore, it is clear that you have dealt with the issue extensively for several years. Having done all that and commented on all those documents, will you outline to the Committee your observations on the quality of the data and recommendations for improvement that were made in relation to those documents? Having been through the mill of those four documents, what is your overall impression?
Mr Mills:
The overall impression is that it is good in parts. I would not, as a result of the issues that have arisen, write off Northern Ireland Water’s data. Certain parts of the data, such as the monitoring of waste-water treatment works and of water quality, are of a standard that I would expect to see anywhere else. Other parts of the data are in a rapid state of flux. Those issues have arisen as the company has emerged from being a Government service. It did not particularly need that data set then, and is now bringing that together.
We have seen major improvement in things like the collation of contract data. There has been a lot of movement in data that was set out in individual departments being brought towards central systems of management. That can be a strength and a weakness; it can be a good thing and a bad thing, because you are taking data away from the people who own and use it. The quality of that data might deteriorate because it stops being their problem; it is not their data. By bringing that data towards a central system, you impose central management and central checking on it, so you get uniformity.
The central management of the business also understands what the quality of the data is. We see the level of improvement that is happening, sometimes from very poor quality data such as flooding records. There are other areas, such as the allocation of the capital programme for regulatory accounting. That is used, not for actual accounting, but by regulators who want financial data that fall among four categories of quality-based growth enhancement. That is partly to allow us to compare the company’s investment performance data with that which was included in its business plan. It is partly to allow us to build up a long-term record so that we can benchmark the company against other companies.
It is the first time that Northern Ireland Water has produced that data and it needs to be proved. It will probably take a couple of years before it is actually working. I am not quite sure how far I need to continue with the argument.
Mr Wells:
My questions are trying to tie down your overall impression of the quality of the data. I think that you are being fair in saying that some of it is fine, some of it is in flux and some of it is inadequate. That is a fair representation of what the Committee has encountered.
Mr Boylan:
Thank you very much for your presentation. The Chairperson has touched on the business plan, and that we believe that it was inaccurate. Although you have answered that question, there is also an issue about consumer confidence.
Do you have any examples of issues around data quality in any other utilities, given your 27 years of experience in the business? Can you elaborate on the time frame in which to improve the data, and try to address those issues? I realise that Jim has touched on that point; we are not trying to nail you down.
Mr Mills:
I am smiling because every time you mention my having 27 years’ experience, I suddenly feel very old.
Mr Boylan:
I am sure that you have dealt with a lot of issues in that time.
Mr Mills:
One of the issues that I touched on is confidentiality, and I am quite nervous about speaking from experience with other companies. I have quite a lot of detailed information from other companies, which I must treat in confidence. I will talk in general terms, if I may.
Yes, I have seen similar issues over the years. I started this work in 1992, after water companies began to be privatised in England and Wales in 1989. Data was sitting out in lots of different organisations. I was looking at paper records, and at people who were trying to collate those paper records for this funny person called a regulator, who had been imposed on them. They did not quite know how to do that. Some of those data reports were of poor quality.
People also struggled with the concept of regulatory reporting, in that there were a set of strict definitions. What does a paper document actually mean to someone who is collating the information for the first time? People were just getting things wrong in the reports. Over the years, there has been a transition in the collation of data into the central new IT systems that were developed and imposed on companies by their management. Data was being sucked through into the centre, and being analysed and understood in much greater detail.
We have touched on the business plan, and I have spent quite a bit of time working on company costing systems. I have seen company costing systems improve dramatically over relatively short periods; from having scattered information of no use to anyone, whereby everyone is making their cost estimate on the basis of their judgement, to having coherent, corporate costing systems that are auditable and able to be traced back.
Those types of improvements can be made in a reasonable timescale. I am aware of issues concerning customer numbers and similar reporting issues in companies in England, Wales and Scotland. It is always surprising when a company that is billing its customers does not have a complete and accurate record of those customers. I could set the Committee a task to compile a list of Northern Ireland addresses and determine whether they are connected to the water and sewerage system; it is not easy to do accurately.
Mr Boylan:
Your answer suggests that there are companies that have already started out on this venture. Our problem concerns why the NIW has not learnt from those experiences. Obviously, it did not learn from the available reports and must now build public confidence. What is your experience of trying to build public confidence in this area?
Mr Mills:
The only way to build public confidence is to deliver on one’s plans, so that people see that delivery. The system should be managed in such a way that ensures that the NIW is clear and realistic about what it can achieve over a period of time. If it starts with unrealistic numbers and expectations, it will fail and confidence will go. Plans should be clear and should be monitored so that people can see improvement over time.
Mr W Clarke:
Thank you for your presentation. Although I appreciate that you cannot discuss specifics, are there any industry examples of data quality and risks? Who should the Committee be talking to in order to hear experiences of managing such risks?
Mr Mills:
The Committee should talk to the other regulators in England, Wales and Scotland: they have an overall view of data risks. Moreover, they experience the management of those risks, because they receive business plans and then decide how to assess companies’ future investment and targets when they know that information is not as good as it could be. Furthermore, they take action against companies to manage the data improvement or to penalise them for the quality of data that they report. Regulators are possibly most open to discussion with the Committee; people are not keen to discuss their mistakes in public.
The Chairperson:
Thank you very much. That was very interesting.
I welcome Chris James, James White and Ewan Bradley from Steria Ltd. You have 10 minutes to make a presentation, after which members will ask questions for about 20 minutes.
Mr Chris James (Steria Ltd):
My name is Chris James, and I am the client director in Steria. My responsibility is to provide Steria’s services to the NIW. My colleague James White is the services director and looks after the day-to-day operation of IT and business processing. Ewan Bradley is head of professional services and looks after projects and ongoing service improvement initiatives connected with the NIW.
I will go through our submission to the Committee. Steria provides IT and business process services to several organisations in the public and private sectors. The company is French owned; and, since January 2006, has been a partner to the NIW. The NIW awarded the contract for customer contact and billing, and mobile works management to a company called Xansa. Steria acquired Xansa in October 2007, when its operations became part of the wider Steria business.
Steria provides customer contact and billing. Therefore, we look after the IT systems that enable the billing operations of the NIW. We are also responsible for customer contacts. Anyone with an operational or billing enquiry makes contact to us by telephone or letter, and we respond on behalf of the NIW. We provide a predominantly IT-led business process and service to the NIW.
Steria also has a number of partners, on a subcontract basis, who provide various discrete, specialist functions. Echo Managed Services provides contact handling. It provides the agents who answer the phones. AMT-SYBEX provides support for the IT systems to enable the mobile works management applications. Those take the operational calls from the contact centre and route them through the IT system so that the engineer can respond to the incidents and provide the fix.
Steria has been involved with the customer contact and billing project from procurement to award of contract in January 2006. As part of that contract, we were required to implement the original billing system to enable NIW to commence its billing operations, which were planned for April 2007. Once those billing systems were implemented and operational, we were required to provide ongoing operational support for the IT and business process systems. We have moved through the implementation phase and are now practically at the operational support phase.
The customer contact and billing operation was originally built to include domestic billing. However, since its deferment, the operation has focused solely on non-domestic billing. From an operational perspective, we cover both domestic and non-domestic contacts. Approximately 200 people work in customer contacts. They are based in Belfast and answer telephone queries.
The billing system was based on a platform called Rapid, which is a software application owned and developed by our partner Echo Managed Services, the company that also provides the day-to-day agents. Rapid was originally developed in south Staffordshire, and Echo Managed Services is wholly owned by South Staffordshire Water. The Steria billing system involved the creation of the original database and the implementation of the IT systems. Twelve independent databases from nine discrete organisations were used to build the database that enabled billing to commence from April 2007. Those databases ranged from information held by Land and Property Services (LPS), formerly the Rate Collection Agency (RCA), through to the NIW’s legacy billing system, HiAffinity. Much of the work involved the merging of data, because the operation was, effectively, a greenfield site. Although there had been the HiAffinity system, that was for commercial customers only and was originally designed for the domestic and non-domestic market.
We have about 200 staff. The entire IT infrastructure is designed to enable the support of the billing system and the operation of the mobile works management system. More than 300 phone lines come into the building, and the infrastructure of around 40 servers is highly resilient to enable a system that is operational or available 24/7. A large storage area network capacity holds the data, and it is accessible by the contact agent when a customer rings.
Steria’s business is the provision of IT services and business process services to a range of private-sector and public-sector customers. Whether through Xansa or other businesses it has acquired, Steria has been providing services in the utility sector for 25 or 30 years. We have a number of customers in the water industry and in the energy utilities sector, particularly in GB and Europe. We also have a number of other customers, such as Boots and Barclaycard. We are used to dealing with large volumes of data and the reliance that companies have on that data. Our focus is on making sure that the IT systems are available to enable users to access the data when required.
We recognise that the NIW is not unique. As with a number of utility companies, there is always a challenge around data and its quality. We have found that poor data impacts on an organisation’s ability to collect bills and debt against bills. Furthermore, bills that are inaccurate due to poor data, or a misspelt name, for instance, impact on customer satisfaction, and that drives customer contact with the organisation.
We have touched on some of the typical benefits of effective data management: it is easier to collect a debt when an organisation has its customers’ correct details; customers are less likely to need to make contact with the organisation, and it will be more difficult for them to evade the debt. Then there is the utilisation of data for operational effectiveness, such as linking in with the mobile works management system and using operational data for the development of key performance indicators.
We are almost three years into Steria’s contract with the NIW. The original implementation happened from January 2006 to April 2007. The creation and implementation of that original database went through a number of distinct phases. The initial phase was to capture and identify the source systems from which one could extract information. That process began with the identification of the data required and how it needed to look within the new database structure and new billing system. The process lasted from January to October 2006. Around October 2006, the new databases went through a migration and consolidation exercise, because we were using data in different ways than before. We needed to ensure that we minimised duplicated data, as far as possible, and that we matched data from the different source systems to produce one overall record that was based on, effectively, a customer contact detail.
The process went through ongoing implementation. Once the database was available to us around November 2006, we undertook a customer contact campaign to try and ensure that the data in the database was as accurate and reflective as possible. We contacted customers to try to get them to correct the data where necessary — for example, to tell us whether Mr James was at the correct address and that his name was spelt with an “a” and not an “o” and was not, actually, Mr Jones. Similarly, we needed to know whether customers were domestic or non-domestic. That process continued until April 2007 when the new billing system went live, which was when the decision was made to defer domestic billing. We went live with an element of the measured non-domestic users, with a plan to bring in non-measured, non-domestic users in April 2008.
The contract with the NIW is now in its operational phase, and we have been working with the NIW on a number of initiatives, particularly with regard to data. The NIW set up what it calls a data integrity group. We are involved with that group, looking at any initiatives that the NIW wants to take forward with the data from an operational perspective. We are an active participant in another group, called the data co-ordination group, where we engage with the NIW and the LPS to help them with the annual reconciliation of non-domestic data. There are a number of other ongoing service improvement proposals based on work that we are doing in the utilities sector or in other sectors in GB, reflecting back potential improvements in the NIW.
The Chairperson:
How disappointed were you that the first attempt at billing non-domestic customers was so far out after all the work that you put in? It was a disaster.
Mr James:
My colleagues and I joined Steria about two years ago. I am aware of the pressures of the implementation time, but I was not there to experience that. A lot of effort was put in by many people in very concentrated timescales in order to get to 31 March 2007. We were at a point whereby the system was up and running and bills were potentially being printed when the call came to stop. I am sure that the project team was disappointed, because it had put in so much time and effort. However, everyone was aware of the wider context, and accepted that these things happen with projects. This project is no different from any other in the sense that external events happen that impact the project.
The Chairperson:
That was not my question. I ask whether you were disappointed by how wide of the mark you were with the billing.
Mr James:
I am sorry, but I do not understand the question.
The Chairperson:
A large number of bills were sent to non-domestic customers that were clearly wrong. The amounts were wrong, the addresses were wrong, the names were wrong: all the things that you talked about. You obviously did a lot of work on that, but at the end of the day the NIW was highly embarrassed by its billing system on two occasions.
Mr James:
That is endemic in an evolving billing system and the nature of the source data. It was a brand new billing system and database. The effort put in was the best that could be done at the time, and all possible steps were taken to try to ensure that the information was accurate. However, we were dependent on the underlying source data that was in the systems and on our ability to match it.
The Chairperson:
Was the original data flawed?
Mr James:
I cannot give an opinion on the data.
The Chairperson:
Is it the case that, based on the information you gathered, any bill you sent to a non-domestic customer was 80% inaccurate? Is that due to the data you received, or can it be blamed on anything else?
Mr James:
There are several reasons for an inaccurate bill. The original format had the potential to cause confusion. Also, some customers were getting bills for the first time. There was a campaign to educate customers. Part of the ability to improve bills and improve data is gained by obtaining information from customers about why they felt that their bills were inaccurate.
The Chairperson:
But the bills were inaccurate: customers were being overcharged and undercharged.
Mr Ewan Bradley (Steria Ltd):
There was one specific issue with regard to the migration of data from the legacy billing system. It related to the size of the water pipe, which determines the standing charge. Under instruction, that information was migrated into the new billing application. However, the instruction was incorrect in that the wrong size was taken across and the standing charges were calculated incorrectly in about 6,000 cases.
The Chairperson:
It was slightly more than that.
Mr E Bradley:
I am sorry: there were 6,000 overcharges and about 2,000 undercharges.
The Chairperson:
Those inaccuracies occurred in what is quite a small non-domestic base. If that was the case in a small non-domestic base, what percentage of inaccuracies would there have been had you went live with domestic customers?
Mr E Bradley:
Given that the majority of domestic customers are not metered, the pipe size does not apply. Domestic bills would have been calculated on the net annual valuation (NAV).
The Chairperson:
Yes, the bills would have been calculated on the value of the property.
Mr E Bradley:
Yes.
The Chairperson:
However, the value of the properties was also wrongly calculated. Even the dummy bills that were sent out to non-domestic customers were wrong.
Mr E Bradley:
From the NIW’s perspective and from our —
The Chairperson:
It was whoever was responsible for sending the bills out.
Mr E Bradley:
All we could do was to take the property values from, at that point, the Valuation and Lands Agency, which is now the Land and Property Services.
The Chairperson:
My point is that customers are not worried about where you get your information from: they are worried about the bill that you send to them. Even the dummy bills for domestic customers were wrong. The data that you were using to calculate bills for domestic customers was wrong.
Mr E Bradley:
It may well be, but that is completely outside the NIW’s control.
The Chairperson:
In what sense?
Mr E Bradley:
In the sense that the NIW does not carry out property valuations; it takes that data from somewhere else.
The Chairperson:
But it is the NIW’s bill: when the NIW sends me a bill, they are charging me. It is the NIW’s responsibility to ensure that the bill is right. It cannot be anyone else’s responsibility.
Mr E Bradley:
That is fair enough, but we just do what we are told when we produce the bills.
Mr W Clarke:
You are very welcome. Thank you for your contribution, Chris. The Chairperson has asked most of the questions that I wanted to ask, and you answered them in your presentation. I will ask you the same question that I asked in the previous evidence session.
Are you aware of any other organisations that have dealt with the same problems and risks with data? You have a long history in this field: are there people who have dealt with these problems successfully and could assist the Committee and the North generally?
Mr E Bradley:
All water utilities will have experienced similar issues. Indeed, some of them are our customers, and we have helped them through similar issues. When the majority of users are not metered, the data quality tends to be significantly poorer. I suggest approaching water utilities that are willing to advise the Committee of their experiences.
Mr W Clarke:
Should the Committee also speak to regulators elsewhere? Would that be worthwhile?
Mr James White (Steria Ltd):
Yes; or the Committee could approach other energy companies, such as the NIE. The NIE bills and provides a service to the same group of customers as the NIW. Furthermore, the NIE has been in the private sector for a long time and its data handling and billing procedures are more mature.
Mr Boylan:
Given your experience in the industry, what are the limitations or constraints of the current billing system, and how might those be addressed? Furthermore, Steria is used to collecting data, but how is that data substantiated? Whose job is it to get the data right? There is an obvious operational aspect, but if you are only using data collection methods how do you get it right?
Mr E Bradley:
The main billing application takes either the NAV or meter-read from the LPS. It then calculates the bill, which is sent to the customer. The billing application does what it is supposed to do. It works on the data provided.
The real issue was that the data that came down from the initial migration from the HiAffinity system, and the data that came via the VLA and the RCA, were particularly poor. Indeed, approximately 50% of the property details were missing from the HiAffinity system, which is hugely significant when one is trying to populate a new billing application.
Mr Wells:
Clearly, various attempts have been made to cleanse the system and improve the data quality. How good is the data at present?
Mr E Bradley:
Things have improved significantly over the last couple of years. We have just gone through the annual billing exercise for 2009, and the reconciliation of our data with that from the LPS was less than 1% with respect to a match. Furthermore, the annual billing has progressed extremely well, and the level of contacts from customers is significantly lower compared to last year. A lot of effort has been put in by Steria and the NIW to address some of the underlying data issues.
Mr Wells:
Therefore, you think that the data is approximately 99% accurate?
Mr E Bradley:
No. There was a 99% match between the unmeasured data that the LPS hold and the data that we hold. We go through that reconciliation on an annual basis, ensuring that a comparison is done and that the correct customers.
Mr Wells:
Do you have any idea roughly how accurate your data is?
Mr E Bradley:
It is probably 95%-plus accurate.
Mr Wells:
How did you validate that data? What were the mechanics of that validation process?
Mr E Bradley:
As part of the data integrity group, a number of initiatives were undertaken to address known problems with the data: for example, the high number of duplicate records, the incorrect classification of meters, and property valuations. A significant number of initiatives were undertaken to improve the overall quality of the data in the billing application.
Mr Wells:
Who carried that out? Was it your company or Northern Ireland Water?
Mr E Bradley:
We both did that. The data integrity group is a committee made up of department managers from Northern Ireland Water and data experts from Steria Ltd. In the case of test meters, for example, the meter readers were tasked to visit properties and undertake an investigation, and, as a result of that investigation, they passed the information back to us and we updated the underlying database. It was a joint activity.
Mr Wells:
Was any independent auditing done on the quality of the data? Was it done entirely in-house between Steria Ltd and Northern Ireland Water?
Mr E Bradley:
As part of the information returns each year, the regulator takes a view on the quality of the data. Independent audits of the initial data migration were carried out, although I do not know who did that as I was not there at the time.
Mr Wells:
The data was 95% accurate. When will it be completely accurate?
Mr E Bradley:
The data of any utility or water company will never be complete. There are always issues with data — in particular, the fact that customers move properties and issues with information from new connections means that the data will never be complete. Activities are in place to address data-quality issues.
Mr Wells:
I hate to tell you this, but thousands of people pay their rates for a water service in Northern Ireland. There are folk who have never been billed for anything and who are keeping their heads down.
Mr E Bradley:
I understand that, and, to be honest, that is true across the piece. That issue is not specific to Northern Ireland Water. I have a meeting with the company next week to address that problem.
Mr Wells:
I think that Northern Ireland people are better at that than anyone else.
Mr White:
Many programmes are ongoing. As the data improves, our focus moves to what is left and to trying to categorise that into small groups. We have void inspectors and property inspectors who go out to check whether a property at a location has been demolished and whether there is a business at that location. That gives us leads on which we can follow up.
Mr Wells:
There are little country lanes in south Down to be checked.
Mr White:
Yes, right down to cattle troughs.
Mr E Bradley:
There are agencies that hold that data, and getting the correct data-sharing protocols and ensuring that NIW or IPS adhere to data protection is an issue that hinders the sharing of complete sets of data between organisations, which would be extremely useful.
The Chairperson:
You said that your billing data is 95% accurate. What is the industry’s average for billing accuracy?
Mr E Bradley:
I am not sure what the average is.
Mr James:
We shall take that point away and respond to you.
The Chairperson:
I am concerned that 5% of 500,000 bills represents a high number of inaccurate bills.
Mr Ross:
You said that some of the data issues are not unique to Northern Ireland. Your company has experience with various utilities. What issues typically come up with data elsewhere? Do similar issues arise, or are the issues different elsewhere?
Mr James:
The issues, which were mentioned earlier, are lack of information and trying to track customers as they move so that their debt can be closed down or their details can be transferred to their new property. We must keep track of that information and identify it in the first place as people move into properties and as properties become vacant. Retaining that information and ensuring that they can identify and bill as many of their customers as possible is a challenge for all utility companies.
The key issue is more historic and involves properties that are connected to the water supply, but the existence of which NI Water is unaware. Such situations will disappear over time, as all new customers will need to be metered. However, I am meeting a customer next week who has told me that between 10% and 15% of the total population are not being billed, and they have 3·5 million customers.
Mr McCallister:
What is an acceptable percentage for accuracy? I assume that one of the benefits of addressing the shortcomings will be getting that right, but what approach are you taking to creating and maintaining the data systems and ensuring that that continues? What is an acceptable accuracy target? Is it 96%, 97%, 98% or 99%?
Mr E Bradley:
One of the key functions of the data integrity group is to ensure that all data that comes into the billing system and all data that leaves it are fit for purpose. Therefore, a lot of work has been done in analysing the business processes to ensure that the data is as it should be.
With regard to accuracy targets, one would want to get as close as possible to 100% accuracy, but that will take time. Northern Ireland Water is relatively new, and it will take time to get to the levels that one wants.
Mr McCallister:
Do you have a target for that?
Mr E Bradley:
I am sure that NI Water has its own internal targets, and we try to do everything that we can under its instruction to resolve any data issues. From an operational perspective, the call centre receives written correspondence, which is used to try to enrich the data set as much as possible and to verify that the data that is held for a particular person or organisation is as accurate as possible.
Mr White:
Proactive data cleansing steps are also taken. When we look at specific client bases or customer groups and discover something, we can check it against the LPS database and then cleanse the data proactively as well as reactively.
The Chairperson:
The metering of all properties is a long way off. It is not even on the horizon because of the amount of capital that is required. The charging of domestic properties is based on the valuation of properties, and the valuations are carried out by Land and Property Services. That is the only body that can provide you with that information. No one else does that. Are you concerned when you see Land and Property Services’s record for providing information? I am a city councillor, and I am sure that other councillors around the table will be aware that its historic record of providing property information is horrendous.
Mr E Bradley:
I agree, and I am sure that Land and Property Services will also agree. A data co-ordination group, which comprises Northern Ireland Water, Land and Property Services, the Housing Executive and Steria, is undertaking initiatives to try to improve its service. We are, obviously, selfishly looking at it from Northern Ireland Water’s perspective in respect of what we need from the LPS and how we ensure that what we receive is as accurate as possible. The data-sharing protocol that will shortly be put in place will go a long way to help us to do a more thorough reconciliation and investigation of the data that it holds.
The Chairperson:
I hope that you will be more successful than the city councils have been, because some of the reassessments of valuations are now five or six years old.
Mr White:
It is all the more important, then, that the groups work together.
The Chairperson:
Yes, but they do not have the information.
Mr James:
They are currently undertaking a complete revaluation of their commercial —
The Chairperson:
But what we are all concerned about, with respect to customer confidence in the company, is as follows. I know, you know and the company knows that the source of the data that you are getting for domestic customers — which forms the biggest part of your billing system — will not be accurate. We know that before you start. The Land and Property Services data is not accurate. That is the difficulty for the Committee. Some of the databases that have financial impacts on the company are wrong, but the company still insists on carrying on, knowing that full well. The information they arrive at from that data is wrong.
That is the position of the Committee. We have asked people about this and they have not come back to give us information. The financial forecasts and the strategic business plans are wrong, and the company know that. However, some of the financial projections are based on that data. The billing of domestic customers is wrong. The Land and Property Services data is suspect. Do you acknowledge that before you start?
Mr E Bradley:
Yes.
Mr Boylan:
To follow on from that, there is an issue of townlands and postcodes. We get the impression that the company will be sending bills into County Monaghan if it is not careful. Let us be realistic. How will the company obtain the correct addresses, never mind get the bills right? That is a realistic question. You say that you are gathering data across all bodies, but up to now that has not happened. It is all right for you to laugh, but I am serious. You need to get the postcodes and townlands right. We are talking about public confidence. Mr Wells talked about the North of Ireland —
Mr Wells:
I did not talk about “the North of Ireland”.
Mr James:
What we are trying to do —
The Chairperson:
The Committee knows that the data produces the bills.
Mr E Bradley:
There are two issues. First, there is address quality, which has been addressed on an ongoing basis. The second is the valuation on which the bills are based. That is a much more difficult question to tackle.
The Chairperson:
You can operate only on the data that you get, and we do not dispute that. However, the data is suspect. Everyone knows that, and Land and Property Services would concede that. We worry that there will be tens or maybe thousands of customers complaining about Northern Ireland Water and calling into question its credibility. Then the issue arises as to whether they will pay. If there is ever to be a charge for water, that is what the Committee is worried about. Thank you very much for your time. It was very interesting.
We turn to the evidence from KPMG. Thank you very much for coming. I apologise for keeping you late.
Mr Arthur O'Brien (KPMG):
I thank the Committee for its invitation. I am an audit partner with KPMG and I am responsible for signing off the audit opinions on Northern Ireland water. I am supported by my colleagues Jon D’Arcy, another KPMG audit partner and the firm’s head of audit practice in Belfast; and Dominic Mudge, an audit director in KPMG who does the bulk of the real work in audit that I get to come along and talk about. We have prepared a brief presentation, which should take us through the evidence paper that we submitted in advance.
I draw the Committee’s attention to a couple of points in our submission. Our audit work was conducted to enable us to form an opinion on the truth and fairness of the financial statements. Therefore, there are issues of interest to the Committee that we may not have considered during our audit. I will explain that as it becomes evident during the presentation.
KPMG supports Northern Ireland Water’s commitment to be open and transparent with all of its stakeholders, including the Committee. However, we encounter certain confidential items in the course of our work in respect of the company, its commercial activities, its employees, customers and suppliers that must remain confidential. There may be matters that we cannot stray into during this session.
KPMG was appointed as the first auditors to the company when it became a Go-co on 1 April 2007. We have done only one audit and have completed just one audit cycle, which was to the year ended 31 March 2008. We provided an unqualified opinion on the financial statements on 3 July 2008. We are in the process of working through the audit for 2009, some of which will be relevant to today’s proceedings.
KPMG’s role as auditors is primarily to report on historic financial information and related activities and transactions that have taken place. We do not look forward at transactions, nor form or report an opinion on the accuracy of the data or information or of the quality of the systems used to produce that information. We do consider the quality of the data and of the systems in order to enable us to form our approach to the audit. We like to be able to rely on systems of control to make our audit efficient and to avoid having to do lots of detailed transaction testing. Therefore, weak or poor systems do not necessarily lead to qualified audit opinions; other audit procedures can be used that enable us to get our information from elsewhere. I will expand on that later.
Implicit in our unqualified opinion that the accounts give a true and fair view is KPMG’s view that Northern Ireland water has maintained adequate books and records to enable it to produce its historical financial statements on a timely basis. That does not necessarily mean that everything is perfect; it just means that they are adequate to enable the company to produce its accounts. Other areas of the company’s books and records, which are not relevant to the production of accounts, may be inaccurate, and the Committee discussed confidence grades with the former Northern Ireland Water independent reporter earlier.
Similarly, KPMG’s opinion does not extend to forecast financial information. We do not look at the company’s detailed business plans. Doing so is not necessary for us to form our opinion. Nor do we look how costs are apportioned between household and non-household streams. That information is not presented anywhere in the accounts for audit and it is not an audit requirement for us to do that.
In one respect, KPMG’s audit of Northern Ireland Water differs from that of other private limited companies. The statutory audit is exactly the same: there must be truth and fairness in the statutory accounts that are reported to the shareholder. However, there is a separate and parallel process in reporting to the regulator on a number of items. I will now take the Committee through the items that are relevant to its investigation.
We have reported on four pieces of work. The review of the interim principal statement is a report to the regulator on six-month figures. That occurs in January each year. Management prepares a detailed submission to the regulator. We are asked to report on two figures; that the turnover for the six months until September has been properly extracted from books and records. So, the only role that we have in that process is a very simple one. We do not do any work that has an impact in relation to billing, forecasting or apportionment.
The audit of the statutory and regulatory accounts is the major piece of work that we do during the course of each year. It probably takes us a couple of months to complete it. Our opinion of the statutory accounts, which is publicly available, is reported to the shareholder, and we report our opinion of the regulatory accounts to the regulator. The statutory accounts are based on historic financial information; the regulatory accounts differ because they take that historic information and make adjustments to assets that were purchased many years ago to ascertain their current replacement costs. In simple terms, that is what regulatory accounts are. Forecasting apportionment is not included in either of those reports, but billing is. I will talk about that in a moment.
The water reporter earlier talked in detail about the annual information return. We also report on the financial tables in it. About a dozen of those tables support the regulatory accounts, so it provides a lot more detail. There are fixed assets in the regulatory accounts. The financial tables are split into a lot more detail about infrastructure assets, which is the network of clean-water and dirty-water pipes; and non-infrastructure, which are the above-ground assets. They are then split again between water and sewerage. Primarily, those are the tables at which we look in those accounts and a range of others.
At the same time, the directors provide the regulator with a certificate that explains whether they have adequate systems of control and planning in place, and whether they have traded with any associated companies. We have a minor role in that process. We study the report and state whether anything inconsistent is contained in it. In its directors’ certificate last year, management highlighted issues of data quality to the regulator. We read that and were happy that what they said was consistent with what we found.
For financial auditors such as us, auditing is governed by international standards. The process that we apply is internationally common to all auditors. We use a concept of materiality to form a view, which enables us to conduct our sampling. For Northern Ireland Water, that was something in the region of £3 million. It is mostly formulaic and is about 1% of revenue.
Our audit process goes through four phases: planning, control evaluation, substantive testing and completion. In relation to controls, we test systems to see whether we can rely on them to produce the financial information. We do not assess the quality of those systems to provide an opinion of them — we assess them to establish whether we can use management systems for the purpose of conducting our audit.
If we find issues with the quality of systems, we, as a matter of course, report those to management and the audit committee. If we cannot rely on those systems, we have to do a lot more detailed sampling in our field work, which is what we have done. We conduct all of our detailed testing and sampling, and we bring that all together and form an opinion, not of individual balances of the accounts, but of the accounts as a whole. Throughout that process, we will identify control issues in the company and report those to management. We do that with all companies.
As stated on page 7 of the briefing paper, our audit opinion was unqualified. The detailed audit did not look at the apportionment of costs between household and non-household income. That level of information is simply not provided in accounts and is not required to be reported on by us.
Similarly, we gave no opinion on forecasting. We are only required to look forward for 12 months and assess whether a company has the adequate financial resources to operate in that time. It was not necessary for us to review financial projections in any detail because DRD provides more than 80% of the revenue for the company. It is worth pointing out that we do not provide audit opinions on individual amounts or specific aspects of the accounts.
We raised a number of issues with management about revenue and, in particular, billing. Those are detailed on page 8 of the briefing paper. Issues such as cash receipts on the ledger not being allocated against appropriate invoices and the need to raise provisions against accrued income gave us some cause for concern during the audit process. The amount of accrued or unbilled income concerned us because that must be estimated in the accounts. We looked at billing cycles and found that some customers had not been billed for more than six months. We, therefore, agreed with management that adjustments needed to be made to make provisions in respect of customers who had not been billed for a significant period of time.
Another issue was management’s understanding and use of customer information. In the first year, management took its time to understand what the reports were saying and to decide what actions to take place. We highlighted the fact that they needed to do that. Another issue related to the provisions being made for older debts within the accounts.
We reported 11 specific matters to management regarding the revenue and billing cycle. We reported those concerns to management. Post July 2008, from regularly attending meetings of the audit committee, we know that management and the internal audit have followed up on those issues and reported to the audit committee on a regular basis. From our work on the audit for the year ended 31 March 2009, we know that nine of those 11 issues have been completed or addressed. Good progress has been made in addressing some of those issues.
Given the issues that we had with the revenue and billing cycle, one might reasonably ask how we were sure that the accounts presented a true and fair view and that the revenue was not materially misstated. That answer to that is covered on page 9 of our document. For the year ending 31 March 2008, 86% of Northern Ireland Water’s revenue came from DRD — DRD wrote it a big cheque — and 14% of it came from non-domestic customers. We were able to conduct other procedures to prove that the revenue amount was not materially misstated.
We were not able to rely on those systems in their entirety because of the matters that I raised, so we carried out further detailed testing. Those items of testing are set out on page 9. We applied our £3 million materiality. We identified differences in the detail of billing. We thought it imprudent for management to recognise items that had not been billed for a long time as income, because the chances of collecting that money were low.
Adjustments of £600,000 were made to the accounts in respect of that.
There was also under-billing in respect of accrued income, which was estimated on the number of days since the last bill rather than on volumes of consumption. We thought that volumes of consumption, or run rates, were much more accurate than number of days, because the latter does not reflect seasonality of billing between billing cycles. Therefore, adjustments were made. When that was done we were happy to conclude that the revenue figure in the accounts was not materially misstated, and we reported the adjustments to the audit committee.
That deals with the statutory audit. We do another piece of work for the regulator on the annual information return; the detailed financial tables that support the regulatory accounts. We identified further issues there when we got into the detail of the billing and the accuracy of billing. Those are listed in our submission. The issues had been identified and reported by management, and we confirmed and concurred with management’s view that there were issues around data quality and billing accuracy. Again, we reported those issues to the regulator. However, we were happy that, having gone through revenue and billing in quite a bit more detail than we would normally have expected, we were able to form the view that the accounts were materially correct.
Our submission also contains musings on utility companies and data quality issues in utility companies in general. It is legitimate to ask whether the experience of the NIW was normal. The quick answer is yes. In the early stages of start-up, when industries are deregulated, opened up to competition, or liberalised, when new data sets are required because there is a new method of billing or collecting income, or there is a new regulator, it is often the case that the databases established during the previous regime are no longer appropriate.
That has been the case with the NIW. It has also been the case with utility companies in GB, and we list a number of examples in the power and gas markets, and in the water sector. It might not be desirable that companies have those issues, but it is not unexpected in the early years. How long does it take to fix those things? One would expect substantive issues associated with the old databases to have been teased out in the first couple of years. Certainly, in years two, three and four, one would expect management to be putting significant effort into addressing that issue. One would expect a lot of it to have been cleaned up by the end of year five. However, there are still utility companies in GB with data-quality issues 10 or 20 years down the line.
The Chairperson:
Thank you. If the Department for Regional Development were not writing a big cheque for 86% of the NIW, would KPMG take a different view of the audit? The NIW is relying for its billing on data, which I think everyone would agree, is flawed. If a charging regime is put in place, and domestic customers are charged according to the value of their homes, would that not give you cause you concern?
Mr O’Brien:
Fortunately, or not, that is a theoretical question; I have not found myself in that position. If more than 14% was being collected from customers, it is fair to say that it would be more difficult for us to conduct our audit, and we would have had to do a lot more detailed work.
As regards the data being flawed, there has been some discussion about the property values that the VLA provided. I cannot comment on the accuracy of that information, although some assertions have been made that it is inaccurate, which creates a challenge and is unhelpful, because accurate billing then becomes much more difficult. The company will have to use the data set that it has been given and has agreed to use, and it will have to undertake a process to correct it. However, that is not unusual in new regime start-ups.
The VLA’s database, which has been discussed this morning, was not designed for use by the NIW, so there are issues with, for example, the fact that although people in Housing Executive houses do not pay rates, they use water and, therefore, will have to pay water charges. Lots of issues will have to be addressed in order to perfect the database.
The Chairperson:
The NIW’s projections suggest that its ongoing bad-debt rate will be approximately 5%. Is that an issue?
Mr O’Brien:
That may be the projection, but I cannot comment on projections. I can comment on what has happened up to March 2008, when the level of bad-debt provision was running at 7·5% of turnover.
The Chairperson:
That figure was for non-domestic users only.
Mr O’Brien:
Yes; approximately £41 million was billed to non-domestic customers —
The Chairperson:
And the debt for non-domestic customers is approximately 7·5%.
Mr O’Brien:
Yes, and provision of approximately £3 million has been set against that.
Mr W Clarke:
What are the main differences between auditing an ordinary limited company and a Go-co?
Mr O’Brien:
There is no difference, per se. In effect, the only difference between the two is who owns them. A limited company is owned by private individuals or is publicly listed, whereas a Go-co happens to be owned by the Government. Therefore, there is no difference in the statutory audit that must be undertaken or in the financial statements.
The difference with respect to the NIW is that it is a regulated entity that provides water and sewerage services under a license agreed with the regulator. Within that license, there is a requirement for different audit processes to take place, including, as I said, regulatory accounts and our report on the annual information return. Consequently, we agree with the regulator to conduct certain procedures on various tables and data sets in the annual information return. Those procedures are laid down by the local regulator, although they are the same ones that Ofwat applies. Therefore, reporting to the regulator is the main difference.
Mr Boylan:
What aspects of data quality and risk are encompassed by, or are outside, the scope of the statutory audit, the financial statements and the other documents that must be audited for the regulator; the interim principal statement and the annual information return?
Furthermore, when conducting a general audit, you normally deal with records and documentation. When planning an audit, which is one of the four distinct phases that you mentioned, you suggest that you should identify key risks. Given that you cannot look at financial projections and that you are looking at estimated bills, will you expand on how you identify the risks? You have seen the projections, and you know how they got things wrong, and that people have not received bills. I am not saying that you have written that off, but how do you reflect it in your report? You can only work with information, receipts or reports that you are given. Is that not the case?
Mr Jon D’Arcy (KPMG):
In any audit, we will always be faced with issues that do not fit the norm. In this situation, we examined the context and determined what proportion of the billings was non-governmental. Effectively, we assessed the risks attached. There are several risks, which may lead to understatements or overstatements of revenue. We also considered the impact on the balance sheet. That impact is probably more to do with the bad debt risk, that is, if bills are perceived to be overstated or if people have lost confidence in the debt that they were issued.
Thereafter, in our audit, we assess the impact of the level of bad debt and whether it could be material to the financial statements and what other work we can do to ensure that we are comfortable that the level of risk is limited. We do not say at any stage during the audit that everything is now correct. We try to reach the stage in which we can say that the likelihood of a significant error in the accounts is not going to be material to the overall picture of the financial statements. We never say that they are correct, because, given what we heard this morning, if the raw data is flawed, they will be incorrect. Although it sounds like understated revenue, one cannot say that it is understated or overstated revenue; one can never know.
Mr O’Brien:
With respect to understated revenue, it would be imprudent for management to recognise that in its accounts without providing against it. If it has not been billed for several months or over a year, the chances of collecting it and enforcing the debt is lower. We would consider those billing risks and examine under-billing and over-billing. Customers who have been over-billed tend to delay their payment. Therefore, we consider the debtors ledger at the end of the year and determine how old the debts are. Management will have difficulty collecting payment for bills that are over 90 or 180 days old or over a year old. We are happy with the profile of provisions, and additional provisions were made during the audit process. We thought that more provisions needed to be made.
Mr Ross:
Are you aware of any specific industry examples in which the data quality and risk issues that we discussed have been successfully addressed? Whose experience might be useful to the Committee’s inquiry in Northern Ireland to improve performance?
Mr O’Brien:
That is a slightly unfair question. I prefer not to list and grade our clients and say that some are better than others. A previous witness said that the Committee should talk to other regulators. I endorse that suggestion. If the Committee asks other regulators which companies have and have not worked well, they will offer a good steer. I prefer not to list good or bad examples.