Summary of Responses to the Core Market Data Cleanse Request for Information (RFI) - Version 1.00 | 17 February 2021
←
→
Page content transcription
If your browser does not render page correctly, please read the page content below
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI) Version 1.00 | 17 February 2021 Authors Luke Austin luke.austin@mosl.co.uk Matt Labrum matt.labrum@mosl.co.uk
Contents
1. Introduction ..................................................................................................................................................... 2
1.1 Background ................................................................................................................................................ 2
1.2 Key highlights............................................................................................................................................. 3
1.3 Next steps .................................................................................................................................................. 5
2. Summary of RFI Responses.............................................................................................................................. 7
2.1 Impacts of poor data quality ..................................................................................................................... 7
2.2 Trading party activities to mitigate poor data quality ............................................................................. 12
2.3 Benefits of data quality ........................................................................................................................... 17
2.4 Paid for data services .............................................................................................................................. 19
2.5 Maintaining data quality ......................................................................................................................... 20
2.6 Data outside of CMOS ............................................................................................................................. 22
2.7 New technology and end-user verification ............................................................................................. 23
2.8 Customer details definition ..................................................................................................................... 24
2.9 Meter manufacturer validation ............................................................................................................... 25
2.10 Additional feedback............................................................................................................................... 26
3. Conclusion ..................................................................................................................................................... 29
Appendix I .......................................................................................................................................................... 30
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 11. Introduction
1.1 Background
As part of the Market Performance Operating Plan (MPOP) 2020/21 (see Figure 1, Workstream A: ‘High quality
customer, premises and asset data’), MOSL committed to delivering a data cleanse plan for core market data
items.
Figure 1: Overview of MPOP for 2020/2021
We published a Request for Information (RFI) in October 2020 to inform the planning and prioritisation of the
data cleanse activity.
We sought feedback on: (i) the cost-impact and benefits of data quality; (ii) the required activities for
mitigating or maintaining data quality and (iii) the recommended next steps for resolving data quality issues.
The RFI consisted of ten questions covering the three core groups of data contained in the Central Market
Operating System (CMOS):
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 2 Customer details and premises data
Meter location data
Meter details data.
This document summarises the findings from the RFI and provides high level detail on the publication and
delivery of the data cleanse plan.
1.2 Key highlights
Overview
The responses provided 97 per cent coverage of the market based on aggregate share of supply points,
consisting of 14 wholesalers and 10 retailers
Respondents welcomed the focus on addressing core market data issues and were broadly supportive
of the approach; the focus on the key data items identified; and the proposals outlined to address
data quality issues
Some wholesalers questioned the value of ensuring completeness of unique property reference
number (UPRN) and valuation office agency (VOA) reference data; however, the responses to the RFI
have provided strong evidence that these premises data items are crucial for facilitating the fulfilment
of core market obligations and activities
The absence of occupancy status and Yearly Volume Estimates (YVE) from the scope of data cleanse
was noted by some wholesalers; however, we can confirm that they are within scope for broader
market improvement activity but have not been included here as they are already being addressed or
investigated as part of other ongoing work.
Impacts of poor data quality
80 per cent of retailers (eight) and 71 per cent of wholesalers (10) said they are adversely impacted
by poor-quality customer and premises data, aside from the impacts already identified in MOSL’s
analysis
Based on the RFI responses, we estimate the minimum annual resource cost to the market from
managing poor quality data to be £10m; with customer and premises details accounting for 39 per
cent of this cost, meter location data 42 per cent and meter details data 19 per cent
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 3 Trading parties are deploying resource to obtain and verify missing or inaccurate data items,
administer bilateral requests, manage consistency and mismatches between central and internal
systems, resolve inaccurate or incomplete supply point data, address high volumes of failed meter
reads and manage meter verification requests.
Benefits of data quality
Both retailers and wholesalers consistently highlighted that an improved customer experience was
the main benefit of improved data quality through providing a better platform for excellent service,
operational efficiency and meter reading services; thereby leading to greater customer satisfaction
and trust, shorter customer journey times, lower performance penalties and improved accuracy of
settlement charges, leakage calculations and customer billing.
Paid for data services
The majority of respondents said they are using third-party services to manage data or data quality
issues, particularly for credit searches, verifying occupancy, verifying supply point eligibility, and
obtaining or verifying customer and premises details. The types of services used include desk-based
research, site visits, investigations, and data enrichment.
Maintaining data quality
Retailers are predominantly using manual processes to maintain data, but are increasingly moving
towards automated processes, or aspire to do so, such as automated notifications following customer
switches or new supply point registration
Most wholesalers said they have proactive consistency checks to compare CMOS with their internal
systems but rely on more reactive processes to resolve issues or anomalies detected through quality
checks, site visits, customer complaints, bilateral requests, or periodic reviews.
Data outside CMOS
70 per cent of retailers (seven) said there are differences between their internal systems and CMOS,
primarily due to transactions failures, manual intervention, backlogs, system migrations and
unintentional data enrichment
57 per cent of wholesalers (eight) said there are minimal differences between their systems and CMOS
due to the automated processes in place to verify consistency; however, many said they hold
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 4additional data that could be valuable to the market, such as photos of meter locations, meter location
notes, logger identification and ownership details, more granular meter read data, non-billable
premises and premises categorisation information.
New technology and end-user verification
58 per cent of respondents (14) said they supported the use of new technology to capture end-user
input and 42 per cent (10) were not supportive. Many of those that were not supportive said they
would need more detail before agreeing, whereas others provided arguments against, including: it
would weaken data ownership responsibilities; it would negatively impact a competitive element of
the market or it would increase the cost of maintaining CMOS.
Customer details definition
The majority of trading parties said they interpreted ‘Customer Name’ (D2027) as the legal entity of
the customer that is responsible for paying for the water consumption or services provided by the
retailer; however, there was less consistency for the interpretation of ‘Customer Banner Name’
(D2050) field. In both cases there was support for providing greater clarity due to lack of consistency.
Meter manufacturer validation
67 per cent of trading parties (16) supported removing meter manufacturer from the CMOS validation
for meter read submissions, with many suggesting that it provides no benefit and hinders the
submission of legitimate reads into the market. Of those that were not supportive, there were
concerns that meters would not be uniquely identified in all cases and they argued that meter
manufacturer data item should instead be standardised.
Additional feedback
Aside from several additional comments and suggestions that were raised as part of the data cleanse
RFI (summarised in section 2.10), there was a consistent call for MOSL to facilitate further discussion
on key areas to better understand challenges and find ways to resolve issues.
1.3 Next steps
The responses to the RFI have been used to develop a series of activities as part of a programme of data
cleanse work. We will publish a plan setting out these activities at the beginning of March 2021.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 5Broadly, this plan will include details on the following activities that will be implemented over the next 12
months:
Incentivising the completeness of core data items and resolving identified data quality issues through
performance monitoring and rectification
Developing a process for sharing market data to enable end-users to provide better quality or missing
data to the market
Investigating and assessing options for incorporating user-verification for core data items to capture
accuracy
Reviewing the use of meter manufacturer in CMOS validation for meter read submissions
Refining the purpose and usage of ‘Customer Name’ and ‘Customer Banner Name’ data items in
CMOS.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 62. Summary of RFI Responses
We would like to thank the trading parties that took the time to respond to the request for information (RFI).
The responses consisted of 14 wholesalers and 10 retailers, providing good overall coverage of the market
(see Table 1 for further details).
Table 1: Overview of respondents to the data cleanse RFI
Group Number of respondents
Wholesalers 14
Retailers 10
Total Respondents 24
Aggregate supply point share 97 per cent
Wholesalers Retailers
Affinity Water Limited Business Stream Limited
Anglian Water Services Limited Castle Water Limited
Bristol Water Plc Clear Business Water Limited
Dwr Cymru Everflow Limited
Northumbrian Water Limited Pennon Water Services Limited
Portsmouth Water Limited Smarta Water Limited
Severn Trent Water Limited Veolia Water UK Limited
South East Water Limited Water Plus Limited
South West Water Limited Water2Business Limited
Southern Water Services Limited Wave Limited
Thames Water Utilities Limited
United Utilities Water Limited
Wessex Water Services Limited
Yorkshire Water
2.1 Impacts of poor data quality
To ensure that we have captured the main impacts of data quality issues in the market, we asked trading
parties if they experienced any additional impacts of poor data quality aside from those already identified.
Customer and premises data
In the RFI, we highlighted the following impacts of data quality issues for customer and premises data:
Missing property reference data hinders the verification of premises address data, occupancy status,
gap site eligibility and supply point eligibility
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 7 Unreliable customer data obstructs retailers from reliably locating customer supply points within
CMOS, creating costs and barriers for tendering and switching customers, as well as reputational
damage.
Figure 2 shows a summary of the main impact areas identified by respondents for customer and premises
data.
Of the retailers that responded, 80 per cent (eight) said they experience additional adverse impacts from poor-
quality customer and premises data. The additional impacts highlighted by retailers include the following:
Increased resourcing and administration cost due to the need to verify the accuracy of premises
data or obtain missing premises data; and a high volume of site verification and bilateral requests to
wholesalers to verify supply information when this information is unclear
Reduced ability to identify, tender and switch customers due to the lack of uniformity and general
quality of customer and premises data, particularly for national or multi-site customers
Impacted business growth due to the inconsistency of customer data impeding retailers from
effectively competing for tenders against incumbent suppliers
Poor customer experience and reputational damage due to incomplete or inaccurate supply point
data leading to supply points being switched in error; and inefficient verification of occupancy due to
difficulties accurately identifying relevant premises
Hindered debt recovery due to misaligned CMOS and billing systems data, delaying legal proceedings
to verify customer data when there are differences.
Most retailers indicated a concern with an impact on customer experience and reputation due to poor data
quality. Larger, incumbent retailers were particularly concerned by the added cost of obtaining missing
premises data (i.e. UPRN and VOA); and the need to design additional processes and workflows to manage
the inconsistent completeness and accuracy of key data items when verifying premises details or occupancy.
Smaller retailers were more likely to highlight the impact on business growth and the ability to tender or switch
customers.
Of the wholesalers that responded, 71 per cent (10) said they experience additional adverse impacts of poor-
quality customer and premises data. The additional impacts highlighted by wholesalers included the following:
Difficulty identifying and contacting customers for Guaranteed Service Standards payments or
following an unplanned event, emergency response or supply interruption – the incompleteness of
the Security and Emergency Measures Direction V (SEMDV) flag was noted, suggesting that it should
be mandatory
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 8 Water efficiency and market insight is impeded due to missing or inaccurate customer segmentation
details, such as Standard Industrial Classification (SIC) codes.
Several wholesalers said they are starting projects to improve the completeness of UPRN and VOA data in
CMOS; however, some wholesalers questioned the value and expense of maintaining these data items.
Our analysis and the responses to the RFI provide strong evidence that UPRN and VOA data are key for fulfilling
core market obligations and activities. This includes the following benefits of UPRN and VOA data:
It provides a unique reference that identifies a property throughout its lifecycle, from planning
permission through to demolition
It enhances the ability of trading parties to efficiently verify premises and customer address details
and to identify gap sites, demolished sites, occupancy, duplicate premises and erroneous supply
points
It creates a reliable link with external datasets that facilitates the verification of broader data accuracy
and better insight across the market.
Figure 2: Summary of main impact areas of poor-quality customer and premises data
Market Insight
Debt Recovery
Customer Service
Occupancy Data
Cost to Serve
Business Growth
Premises Administration
Customer Visibility
0 1 2 3 4 5 6 7 8 9 10
Wholesalers Retailers
Meter location data
In the RFI, we highlighted the following impacts of data quality issues for meter location data:
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 9 Difficulty finding and reading meters leads to increased costs for trading parties and could damage
reputation
Missing reads reduces the accuracy of settlement, leakage calculations and customer billing.
Figure 3 shows a summary of the main impact areas identified by respondents for meter location data.
Of the retailers that responded, 50 per cent (five) said they experience additional adverse impacts of poor-
quality meter location data. The additional impacts highlighted by retailers include the following:
Increased cost-to-serve due to meter reading companies struggling to find and read meters based on
the CMOS location data
Additional Market Performance Standard (MPS) charges due to issues related to data items retailers
have no control over
Estimated reads are driving non-payment of bills due to perceived over-billing where meters cannot
be read – this is leading to lost revenue as customers sometimes refuse to pay or dispute charges
based on estimated reads
Unable to identify leakage and missed water efficiency opportunities due to inaccurate
consumption data.
Of the wholesalers that responded, 57 per cent (eight) said they experience additional adverse impacts of
poor-quality meter location data. The additional impacts highlighted by wholesalers include the following:
Customer disputes and market friction due to missed meter reads contributing to inaccurate
estimates, leading to inaccurate end customer billing and inaccurate settlement
Increased cost to serve through bilateral forms, alongside additional costs for abortive visits and
meter verification
Additional costs and poor customer service due to estimation hindering the ability to identify leaks.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 10Figure 3: Summary of main impact areas of poor-quality meter location data
Consumption Accuracy
Debt Recovery
Cost to Serve
MPS Penalties
Meter Read Performance
Water Efficiency
Settlement or Billing Accuracy
0 1 2 3 4 5 6 7
Wholesalers Retailers
Meter details data
In the RFI, we highlighted the following impacts of data quality issues for meter details data:
Increased meter read rejections reduces the accuracy of settlement, leakage calculations and
customer billing
Inaccurate meter details can lead to increased unplanned settlement runs and query requests,
increasing trading party costs
Incorrect meter size details can cause the wrong read frequency to be applied, leading to wasted
effort and increased costs.
Figure 4 shows a summary of the main impact areas identified by respondents for meter details data.
Of the retailers that responded, 40 per cent (four) said they experience additional adverse impacts of poor-
quality meter details data. The additional impacts highlighted by retailers include the following:
Additional costs due to poor meter manufacturer and inaccurate meter dial data through either meter
re-reads, additional IT solutions or settlement reruns
Inaccurate billing due to incorrect meter dials data, resulting in customer disputes and under-recovery
of charges.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 11Of the wholesalers that responded 57 per cent (eight) said they experience additional adverse impacts of poor-
quality meter details data. The additional impacts highlighted by wholesalers include the following:
Increased unplanned settlement runs, increased cost to serve and inaccurate customer billing are
driven by inaccurate rollover flags
Verification of meter manufacturer details can be confusing for third parties to verify due to the lack
of uniformity of the data and this can hinder meter reading
Figure 4: Summary of main impact areas of poor-quality meter location data
Meter Verifcation
Read Rejections
Customer Service
Revenue
Cost to Serve
Inaccurate Billing or Settlement
0 1 2 3 4 5 6 7 8
Wholesalers Retailers
The responses have provided strong evidence for the impacts of poor data quality. They have both confirmed
the impacts we had already identified through our analysis and insight gained through subject matter experts;
and provided additional insight into the challenges of data quality. In particular, the detail provided by retailers
regarding the impacts of poor-quality premises data will help us to prioritise remediation activity as part of
the data cleanse work.
2.2 Trading party activities to mitigate poor data quality
We asked trading parties what activities they undertake to mitigate the impacts of poor data quality and to
estimate the cost of those activities. This information would enable us to build a case for data cleanse work
and prioritise the required activity more effectively.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 12Based on analysis of the responses, we were able to determine an assumed annual salary cost and associated
oncosts for a full-time employee (FTE) assigned to data management activity. From the responses from trading
parties regarding the number of FTEs addressing data, we were also able to determine a range of FTE cost.
We then derived an average cost per supply point of managing data and estimated the total minimum annual
cost to trading parties of data quality issues to be approximately £10m per year. Retailers account for
approximately 56 per cent (£5.6m) of this cost and wholesalers account for the remaining 44 per cent (£4.4m).
Please note that this is a strictly minimum cost as it only considers the resource cost of managing poor quality
data and does not include the cost of third-party services to assist with data management nor revenue loss
from reputation or business growth impacts.
Table 2 provides a summarised breakdown of the costs of specific data quality management activities based
on the responses to the RFI. Where possible we have indicated which of the three core data groups are
impacted by these activity costs.
Table 2: Summary of data quality management costs
Estimated Annual Market Percentage Affected Core Data
Data Management Activity
Cost (£ m) Share Group
Locating and verifying meters 3.13 31 per cent Meter location data
Verifying premises details and Customer and
2.81 23 per cent
onboarding customers premises data
Managing system mismatches 1.88 19 per cent All
Administrating bilateral requests 1.35 14 per cent All
Meter details validation 0.83 8 per cent Meter details
Customer and
Managing vacancy 0.52 5 per cent
premises data
Total 10.00 100 per cent -
By aggregating these costs, we found that data management activities to resolve data quality issues with
customer and premises data accounted for approximately 39 per cent (£3.89m) of the total cost to the market,
meter location data approximately 42 per cent (£4.2m) and meter details approximately 19 per cent (£1.91m).
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 13Customer and premises data
Of the retailers that responded 80 per cent (eight) said that mitigation activities requiring additional dedicated
FTE resource were required to manage data quality issues associated with customer and premises data.
The data mitigation activities for customer and premises data highlighted by retailers include the following:
Missing UPRN and VOA data is being obtained and maintained at a cost outside of CMOS as retailers
are not the data owner
The resolution of poor data provided at supply point registration is adding to onboarding costs: for
example, occupancy tracing is an expensive and time-consuming exercise which is being hampered by
poor premises and address data
Dedicated FTE resource and third-party services are required to manage high volumes of bilateral
verification requests and forms due to poor premises data.
Of the wholesalers that responded 100 per cent (14) said that mitigation activities requiring additional
dedicated FTE resource were required to manage data quality issues associated with customer and premises
data.
The data mitigation activities for customer and premises data highlighted by wholesalers include the following:
Management of differences between wholesaler systems and CMOS
FTE and third-party services are being used to investigate and validate occupancy
Verification and investigation of premises to maintain UPRN and VOA data
Resolution of inaccurate or incomplete supply point data at registration.
Figure 5 summarises the main resource areas for mitigating poor customer and premises data.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 14Figure 5: Summary of customer and premises data management resourcing
Premises Registration
Bilateral Forms
Premises Verification
Premises Occupation
Systems Data Misalignment
0 2 4 6 8 10 12
Wholesalers Retailers
Meter location data
Of the retailers that responded 50 per cent (five) said that mitigation activities requiring additional dedicated
FTE resource were required to manage data quality issues associated with meter location data.
The data mitigation activities for meter location data highlighted by retailers include the following:
Resource is being used to address missed meter reads, including the cost and time expended in initial
attempts to read a meter and managing additional bilateral requests to resolve issues
Skip code management for failed meter reads requires dedicated resource and heavy administration
due to high volumes
Cost is being expended on market performance charges due to poor meter location data.
Figure 6 summarises the main areas of resourcing required for data management activities.
Of the wholesalers that responded 100 per cent (14) said that mitigation activities requiring additional
dedicated FTE resource were required to manage data quality issues associated with meter location data.
The data mitigation activities for customer and premises data highlighted by wholesalers include the following:
Management of meter location data differences with CMOS
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 15 FTE resource and third-party services are being used to address long unread meters, including projects
to rectify issues with free text fields and Geographic Information System (GIS) coordinates
Resource is being expended to manage bilateral requests
Sharing of additional data to retailers that currently sits outside of CMOS such Automated Meter
Reads (AMR) and additional location data.
Figure 6: Summary of main areas of wholesaler meter location data management resourcing
MPS Penalties
Meter Verification
Meter Data Share
Bilateral Requests
GIS Data Issues
Locate Meters
Systems Data Misalignment
0 2 4 6 8 10 12
Wholesalers Retailers
Meter details data
Of the retailers that responded, 50 per cent (five) said that mitigation activities requiring additional dedicated
FTE resource were required to manage data quality issues associated with meter details data.
The data mitigation activities for meter details data highlighted by retailers include the following:
Management of mismatched meter data due to legacy data concerns that accurate asset data was
not consistently uploaded at market opening
Administration of high volumes of meter read rejections due to inconsistent approaches used by
wholesalers to maintain meter details data.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 16Figure 7 summarises the main areas of resourcing required for data management activities.
Of the wholesalers that responded 100 per cent (14) said that mitigation activities requiring additional
dedicated FTE resource were required to manage data quality issues associated with meter details data.
The data mitigation activities for meter details data highlighted by wholesalers include the following:
Resource is being assigned to audit and monitor mismatched meter details, particularly for meter
manufacturer and physical meter size – this activity tends to be driven by either bilateral requests or
following a meter exchange
FTE and third-party services are managing meter verification requests.
Figure 7: Summary of main areas of meter details data management resourcing
Read Rejections
Bilateral Requests
Meter Validation
Systems Data Misalignment
0 1 2 3 4 5 6 7 8 9 10
Wholesalers Retailers
The responses have provided a clear basis for developing a case for data cleanse and will help us to effectively
prioritise data cleanse activity. Specifically, the responses have shown that improving the quality of meter
location data is a priority, followed closely by premises data.
2.3 Benefits of data quality
We asked trading parties to detail the benefits to their organisation of good data quality to gain insight into
the desired outcomes of data cleanse work. Figure 8 summarises the main benefits highlighted by trading
parties.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 17Figure 8: Summary of main benefits of good quality data
Improved Occupancy Rates
Enhanced Market Relations
Reduced Customer Complaints
Customer Engagement
Enhanced Market Insight
Water Efficiency
Improved Market Reputation
Increased Revenues & Profit
Consumption, Billing & Settlement Accuracy
Customer Satisfaction
Improved Efficiency
0 1 2 3 4 5 6 7 8 9
Wholesalers Retailers
Both retailers and wholesalers consistently said an improved customer experience was the main potential
benefit of improved data quality. They pointed to being able to focus on providing excellent service,
significantly improved operational efficiency, more efficient and effective meter reading, a reduction in missed
or skipped meter reads and a reduction in read rejections. Trading parties suggested this would lead to more
customer satisfaction and trust, better customer experience and shorter customer journey times, lower
performance penalties and improved accuracy of settlement, leakage calculations and customer billing. This
in turn would generate greater confidence in the market through reliable and stable billing, leading to less
customer complaints and improving timely billing and payment.
Retailers said that customers would also be more efficiently and easily switched and on-boarded, which would
improve retailer reputation and increase engagement with the market. Retailers also said that the benefit
from improved operational efficiency, an enhanced ability to resolve bad data and remove unnecessary
expense would improve margins and profitability. This would improve competition and increase the scope for
providing enhanced customer services, including water efficiency support and advice.
Wholesalers said that improved data quality would provide a platform for retailers to provide better services
to customers, support vacancy programmes, identify leakages, improve accuracy on primary settlement and
less corrective settlement runs. Some respondents noted the opportunity to enhance the relationship with
retailers and reduce the number of bilateral requests due to poor data.
Wholesalers specifically highlighted the importance of being able to communicate effectively with customers
regarding planned work, incidents, disconnections/reconnections, or targeted initiatives, which is greatly
enhanced by having accurate customer data. Similarly, better quality data allows a more intelligent emergency
response based on insight into the customer and customer types.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 18Both retailers and wholesalers pointed to reducing the time and effort required to manage meter data queries
and to minimise the number of bilateral requests by having better data quality. Wholesalers noted better data
quality would also support the provision of timely and accurate responses to retailers, improving the bilateral
experience for retailers.
The responses have shown that improving customer outcomes is a key objective of data cleanse, both in terms
of customer billings but also customer satisfaction and the reputation of the market. Improved accuracy of
settlement and lower cost-to-serve has also shown to be important.
2.4 Paid for data services
We asked trading parties whether any paid for data services are used to manage or enrich data and how these
services were used, their limitations and advantages or disadvantages compared to free-to-use datasets. This
would be used to further understand the activities, costs and challenges required to manage data quality
issues. A summary of the main responses can be found in Figure 9.
Of the retailers that responded, 60 per cent (six) retailers said they use third-party services to manage data
quality issues. The types of services include desk-based research, site visits, investigations, and data
enrichment.
The uses of third-party services included confirming customer or site details (i.e. company name or occupancy
status), verification of occupancy and issues associated with the meter and to identify potential occupiers of
a property or meters that meter readers cannot locate. One retailer said they are using external resource for
analysis of transactions and data to monitor and address issues at a large-scale covering vacancy change
applications, backdated amendments and meter read submissions following meter exchanges.
Verifying change of tenancy or occupancy was a common use of paid-for-services, and respondents said this
is because they typically have better quality information; however, it was highlighted that this represents a
significant cost compared to ‘free-to-use’ external data sets, even though it ensures the accuracy and
completeness of data. A different retailer clarified that they consider this data cleanse for data items they do
not own or control, rather than data enrichment. They said data enrichment is only pursued once the data has
been cleansed at a cost. They highlighted, for example, that enhanced occupancy tracing services relies on
having accurate address data in CMOS.
Several retailers said they are using credit reference agencies (CRAs) to verify the legal entity receiving
services. One respondent again noted that the unreliability of the market data set impedes the matching
process due to the challenge of establishing the correct customer name or address, particularly for small or
medium sized businesses and sole traders.
Of the wholesalers that responded, 64 per cent (nine) said they use third-party services to rectify and manage
data.
The focus was on obtaining more accurate address details, credit searches, verifying supply point eligibility
and meter details, obtaining premises identifiers (i.e. UPRN and VOA) at supply point registration, assisting
gap site and vacancy initiates and verifying occupancy. One wholesaler noted that third-party services do not
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 19recognise the water network and the complex nature of water supply, which can significantly reduce the value
of enrichment services.
Figure 9: Summary of main uses of third-party services used to manage data quality issues
Meter Verification
Systems Data Misalignment
Consutancy
Eligibility Verification
Credit Checking
Finding Meters
Premises Verification
Occupancy tracing
0 1 2 3 4 5 6 7
Wholesalers Retailers
Respondents have highlighted a number of uses of paid-for-services and provided further insight into the costs
and challenges of managing data quality issues. This will help to inform our approach to data cleanse going
forward.
2.5 Maintaining data quality
We asked trading parties how they maintain good quality data in CMOS for which they are the data owner as
we wanted to understand the protections in place for maintaining data and whether there were any gaps or
inefficiencies. A summary of main responses can be seen in Figure 10.
The responses showed a combination of manual and automated processes were being used across the market
with varying degrees of proactiveness. Generally, all trading parties were proactively ensuring consistency
between their internal systems and CMOS but varied regarding how proactive or reactive there were to
resolve known data issues.
Most retailers indicated that they predominantly use manual processes to verify and correct data but are
moving to automated processes where possible. Most retailers are using consistency reports to identify
mismatches between their billing system and CMOS, covering occupancy, customer name and YVEs.
Differences are investigated and resolved or updated automatically using the High Volume Interface (HVI) as
appropriate. Other retailers have data cleanse teams working to ensure that customer information is
accurately captured in CMOS as quickly as possible. These teams also work to correct legacy data.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 20Many retailers have systems in place that automatically update CMOS following customer notifications,
customer switches or a new supply point registration. This ensures that customer details are captured
accurately when on-boarded and submitted automatically to the market. Some retailers said they would use
site visits to verify occupancy when a customer is unresponsive.
One retailer said they use exception reviews and audits to update meter location free descriptors. Reviews
can be triggered by skip codes indicating that a meter cannot be found or when a customer or wholesaler
notifies of a change in meter location. Another retailer said they rely on their meter reading service provider
to notify them of metering data issues. They will then either correct the data themselves or notify the relevant
wholesaler. They note that this is time consuming, expensive and suggest that it may not have been a priority
for retailers. A different retailer also said they find it “frustrating” to rely on wholesalers to update certain
items, particularly meter location data such as GIS coordinates.
Most wholesalers said they proactively compare CMOS data to internal systems to ensure consistency using
automated processes. In addition, there are reactive procedures to resolve issues or anomalies identified
through quality checks, site visits, customer complaints, bilateral requests, dashboards, or periodic reviews.
Dedicated teams are in place in most cases, as well as automated checks using external datasets.
Some wholesalers said they use technological solutions to capture GPS data when reading or exchanging a
meter, which is then provided to retailers and the market. Another wholesaler said that they can use the serial
number of newer meters to verify meter details such as number of digits and this is pre-populated on systems.
Figure 10: Summary of main activities to maintain good data quality
Reactive management
Exception Reporting
0 2 4 6 8 10 12
Wholesalers Retailers
The responses have shown strengths in several areas of data quality maintenance, especially the aspiration
for proactive improvements to data quality and automated consistency checks between internal systems and
CMOS. The responses also highlighted areas for improvement, such as: a high level of manual processes in
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 21place for some retailers, and a reliance on meter reading service providers for correcting meter data that is
often expensive and time consuming.
2.6 Data outside of CMOS
We asked trading parties if there are any known differences between the core data stored on their own
systems compared to CMOS; and, if so, to provide details and indicate whether they would be willing to share
this data with the market. This information will help us determine whether there is a need to provide
additional routes for data into the market dataset to improve completeness and accuracy. See Figure 11 for
a summary of the main reasons for differences between systems.
Of the retailers that responded, 70 per cent (seven) said there are differences between their systems and
CMOS.
Retailers pointed to misalignments in data where they are the owner due to transaction failures, manual
intervention, backlogs, system migrations and unintentional data enrichment. Many retailers indicated they
are working to correct these issues and reviewing procedures, notifications, and technical solutions to ensure
greater consistency.
Two retailers explicitly indicated a willingness to share misaligned data for which they are not the owner, such
as GIS coordinates and meter location notes; whilst one retailer explicitly said they would have no interest in
sharing misaligned data as it could impact their profitability.
Of the wholesalers that responded, 57 per cent (eight) said there are differences between their systems and
CMOS.
Most wholesalers said that there are minimal differences between core data items and their own systems due
to the automated processes in place verifying consistency. Many, however, said they do hold additional data
that could be valuable to the market. This included photos of meter locations, meter location notes, logger
identification and ownership detail and more granular meter read data (e.g. monthly fixed reads), non-billable
premises and premises categorisation information.
Most wholesalers commented that they share this additional data with retailers where possible, but many
added that it would be beneficial to retain this data directly in CMOS. One wholesaler said that in certain
instances they have customer details that are not shown in CMOS, which they provide to the retailer subject
to appropriate GDPR and Data Protection Impact Statements (DPIA) approval.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 22Figure 11: Summary of main reasons for system data differences
Customer Name Differences
Occupancy Status Differences
Additional Data
Manual Processing
0 1 2 3 4 5 6 7
Wholesalers Retailers
The responses provide evidence that data-users do not consistently have proper routes to share data into the
market where they are not the data owner. Substantial improvements to data quality are possible if processes
can be agreed to allow this data to be uploaded, particularly for key areas such as meter location data.
2.7 New technology and end-user verification
We asked trading parties whether they agreed with MOSL implementing new technology to supplement the
market dataset with end-user input. This could include input from meter reading service providers to verify
the accuracy of core market data items such as meter location and meter details data.
Overall, 58 per cent (14) of the respondents said they supported the use of new technology to capture end-
user input and 42 per cent (10) were not.
Of the retailers that responded, 60 per cent (six) were supportive. The remaining 40 per cent (four) retailers
that were not supportive provided additional feedback showing a mixture of viewpoints.
One retailer that disagreed said that responsibility for data quality resides with the data owners, and MOSL
should instead focus on incentivising and rewarding data quality performance.
Two retailers that were not supportive said they broadly agree in principle with capturing end-user input and
could be supportive if they had more information. In particular, more detail on the potential technical
challenges and limitations would need to be considered before they would support this type of approach.
They welcomed further discussion or development in this area.
Another retailer argued that retailer-owned data was a competitive element of the market and supplementing
the market dataset with this data would reduce competition. They also stated that the purpose of the market
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 23dataset and CMOS is to provide efficient market operation, calculating settlement and enabling switching.
They also noted a concern that maintaining this data could increase the cost of maintaining CMOS, thereby
reducing profitability of retailers.
Of the wholesalers that responded, 57 per cent (eight) were supportive. The remaining 43 per cent (six)
wholesalers that were not supportive provided additional comments.
Four wholesalers that disagreed said they would need to see more details but were interested in the concept
and supported further exploration, potentially through a working group. It was highlighted that there would
need to be appropriate governance to maintain data ownership. It was also suggested that it would need to
be mandatory if it was implemented to ensure it added value.
Another wholesaler raised concerns about MOSL funding new technology given that trading parties are
obligated to ensure accuracy of data. They suggested that the approach for funding new technology or
processes should be based on the “polluter pays principle”.
One wholesaler said this approach created a risk of data quality deteriorating further as it would weaken
accountabilities for ensuring accurate data. They argued that existing market processes were sufficient to
update or validate inaccurate data.
The responses have provided a strong mandate to investigate the possibilities of incorporating end-user input
to capture the accuracy of data. This would enable accuracy to be measured and incentivised, however the
challenges need to be explored and discussed with the market.
2.8 Customer details definition
During our analysis, we uncovered potential ambiguities in the interpretation of customer data fields.
Therefore, we asked trading parties to explain how they are interpreting the customer name data fields, i.e.
‘Customer Name’ (D2027) and ‘Customer Banner Name’ (D2050) as defined in CSD 0104 (‘Maintain SPID Data’,
section 4.1.1) to understand whether clarification was required.
The majority of trading parties said they interpreted Customer Name as the legal entity of the customer that
is responsible for paying for the water consumption or services provided by the retailer. Many retailers said
they apply this, as appropriate, to an individual, company or organisation, whereas some retailers only
specified a company. Some retailers specified they are using the name contained in Companies House data,
but they did not say how they treat customers that sit outside of the Companies House dataset, such as sole
traders.
There was less consistency for the interpretation of the Customer Banner Name field. Some retailers said they
provided the name of the customer’s business or trading name if it was different to the Customer Name (i.e.
legal entity); whereas another retailer pointed to the need for flexibility as retailers sometimes need to enter
a specific name requested by the customer. They suggested an additional field would be required to capture
the customer requested name, but they questioned whether this would be cost effective to maintain.
Several respondents suggested the interpretation of these data fields has not been consistent and requested
further clarity on their application.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 24Respondents identified the following usages of the customer names fields:
It identifies the legal entity with liability of charges, payment of bills or maintaining contracts
It supports meter reading and customer or premises verification by enabling the correct premises to
be identified
It provides a basis for linking customers that are part of a group with a parent company.
The responses have shown that the ambiguities for the customer name data fields, particularly Customer
Banner Name, require clarification to ensure they provide value to the market and avoid wasted effort.
2.9 Meter manufacturer validation
We asked trading parties whether they agreed with removing the CMOS validation requirement to provide a
meter manufacturer when submitting a meter read. This is due to growing evidence that it is causing legitimate
reads from being submitted into the market and we wanted trading party views on what should be done to
resolve this issue.
The majority of respondents were supportive, with 67 per cent (16) of trading parties agreeing with the
proposal. Of retailers, 70 per cent (seven) were supportive; and of wholesalers, 64 per cent (nine) were
supportive.
Trading parties that were supportive argued that the meter manufacturer validation rule provides no benefit
and hinders the submission of legitimate reads into the market due to misspellings and case-sensitivity.
The wholesalers that were not supportive said they did not believe meters could be uniquely identified
consistently without the meter manufacturer validation requirement; or else they would want assurance that
meters could be uniquely identified after removing the validation requirement. One wholesaler requested
that MOSL provide an estimate of the volume of meter read submissions that fail due to issues with meter
detail; and confirm whether this is a market-wide problem or specific to certain trading parties.
Several wholesalers argued that the manufacturer names should be standardised instead, potentially using a
drop-down option. One wholesaler noted that inclusion of meter manufacturer does not always protect
against duplication due to the lack of checks on erroneous removals of meters that are then added in back
into the market. They argued that removing the manufacturer validation check would add to this issue and
decrease confidence. They suggested standardising the meter manufacturer field and instigate a data cleanse
to migrate legacy manufacturer names to the standardised list. They also suggested introducing a unique
identifier for all meters to remove the risk of duplication.
The retailers that did not support removing this validation check argued that it was key to ensure good quality
data is maintained. There was support for reducing the case-sensitivity and improving standardisation. A
retailer that supported removing the validation rule also specified that this should not lead to a significant
change in the meter submission process or create substantial implementation costs.
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 25The responses show that a solution is needed to reduce the volume of rejections of meter read submissions
and issues associated with inconsistent or inaccurate meter manufacturer data. Further work and consultation
are required to achieve consensus, however, as there were a mixture of views on the best way forward.
2.10 Additional feedback
Many respondents provided additional, detailed comments or suggestions. All points have been taken into
consideration, with some having been directly incorporated into relevant sections above. Those not
incorporated have been summarised below:
The technology exists to improve or correct GIS coordinates, for example systems such as what3words
or Pinpoint
Wholesalers should be able to directly update the location code rather than this having to be done by
the retailer
The inclusion of skip codes would give visibility of trading party attempts to read meters and the
reason why they have failed to obtain the read
There is no way to identify known accessibility issues with a meter, therefore the volume of problem
meters cannot be tracked or easily resolved
There could be benefit creating an additional field to indicate if a meter can be read with or without
the need to contact the customer
Analysis of how current service levels and subsequent market penalties may be impacting data quality
could be beneficial
Given the work involved in addressing the quality of the data, wholesalers will need to be incentivised
with both the incentive of recognition of strong performance and performance rectification measures
The impact of COVID-19 should be considered as part of any data cleanse activity and proposed
timelines for completion
Verification of data needs to be carefully considered against competition as some data items are
competitively procured
The fairness of MOSL incurring costs for data cleanse should be considered given that some retailers
have already incurred costs to resolve data issues and would also contribute to MOSL’s costs
The question of whether the quality of data items is part of the competitive element of the market
also needs to be considered
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 26 The facility for photos of meter locations and reads to be uploaded to CMOS, or another shared system
between wholesalers, retailers and third-party meter readers would be beneficial
Reconsidering the eligibility of sites such as troughs, places of worship, public convenience and
standpipes that may be problematic for the end-customer after four years since market opening
Comparisons of data sets that cannot achieve at least a 95 per cent match should not be used to
indicate trading party data quality
All data cleanse activity should have a clearly intended purpose and outcome to ensure that resourcing
and funding is focused on improving the data items that will really make a difference
MOSL should engage with trading parties on a one-to-one basis to provide a clear explanation of the
checks which are being applied to data quality measures before any public comparisons are published
References to the timeliness of data and looking at the percentage of data that has been maintained
or validated within a certain timescale does not take account of checks made to data that is found to
be accurate and therefore would not result in a change in the market data
The process of updating VOA and UPRN data will take resource away from long unread, vacant,
unpaired SPIDs, stopped meters; and therefore, there needs to be a clear case
MOSL is encouraged to facilitate further discussion to continue capturing challenges and potential
inefficiencies caused by data in the market
The time and cost of incorporating third-party data items, such as UPRN in the market dataset, should
not be overlooked when mandating requirements to update and maintain this information
It is not clear how MOSL propose to ensure that trading parties are updating CMOS with the same
data that is held in their systems in a timely manner
Verification of data needs to be carefully considered against competition
Careful consideration of the cost of using third parties to verify data is required, especially for retailer-
owned data as any costs incurred by MOSL on this are paid for by trading parties. If not all trading
parties have invested not this is going to be “unfair”
A clear cost benefit analysis should be made before any services are procured to determine that the
additional benefits to the market outweigh the cost with clear decision points and governance
Further clarification of the definition of data items can be of benefit
The transaction to update the occupancy status of a premises should require updated customer name
details or it should be automatically set to ‘NULL’ or ‘NO CUSTOMER’
Summary of Responses to the Core Market Data Cleanse Request for Information (RFI)
Version 1.00 | 17 February 2021 27You can also read
