Dear Dame Meg Hillier,
Thank you for your letter of 26 March. Please find in the enclosed annex answers to your questions relating to Office for National Statistics (ONS) economic publications. As I committed in my letter to the Committee on 3 March, I also enclose an update on our plans for the Transformed Labour Force Survey (TLFS).
The ONS continuously improves methods and processes to ensure that our statistics are as accurate, timely and granular as possible and this includes thorough quality assurance. The
nature of statistics and our modernisation work to move off old technology and future proof delivery helps identify quality improvements. We encourage colleagues to raise issues and highlight any concerns with data. Where issues are identified, we communicate these as soon as we can, in line with the Code of Practice for Statistics. We work closely with statistics users and stakeholders on our methodological improvements; where there is not a consensus, we set this out transparently and welcome feedback.
Labour Market Statistics
Work to advance a programme of improvements to our labour market statistics continues to be our highest priority given their importance for economic decision making. We have successfully tested an experimental, shortened TLFS questionnaire and undertaken quantitative and qualitative research and methodological development, working closely with external experts and in partnership with our main users. Citizens are at the centre of the new design, which aims to address the quality issues communicated in previous updates by improving completion rates, representativeness, and data quality. The design and implementation approach have been endorsed by Professor Ray Chambers, Professor James Brown, the Labour Market Technical Group and Stakeholder Advisory Panel. The Stakeholder Advisory Panel includes representatives from the Bank of England, HM Treasury and the Office for Budget Responsibility, as well as independent experts.
The new design comprises a short longitudinal ‘Core’ labour market-focused survey, which takes on average 15 minutes to complete and is delivered online first, supplemented by targeted telephone or face-to-face contact for non-responders. This will be complemented by a separate cross-sectional ‘Plus’ survey to provide wider socioeconomic, household and local data. An article and technical report have been published today which provide the evidence for the new design.
Work is already under way to make the new TLFS design operational, with introduction planned across the second half of 2025. We will carry out a readiness assessment in collaboration with main users in July 2026, aiming for transition of our published headline labour market statistics in November 2026, though transition timing will be data-led and could be in 2027 if our assessment, or user needs, require more data to be collected and assessed.
The Labour Force Survey (LFS) will continue as the lead measure while further development of the TLFS takes place. The initiatives and improvements made to collection, methods, and communication have led to LFS quality improvements, including quarterly person data reweighted back to 2019 using updated population estimates in December 2024; achieved individual responses to the LFS (UK including imputation) have now increased to 63,069 in October to December 2024 from 44,238 in July to September 2023, with further increases expected over the coming months.
Office for Statistics Regulation Report
On 7 April, the Office for Statistics Regulation (OSR) published an interim report on their systematic review of ONS economic statistics. We recognise and share their concerns about data quality and are addressing these as a matter of urgency. Our new Strategic Business Plan published in early April includes a renewed focusing of resources on our core economic and population statistics.
I am copying this letter to Simon Hoare MP, Chair of the Public Administration and Constitutional Affairs Committee.
Yours sincerely,
Professor Sir Ian Diamond
Response to the Committee’s letter of 26 March 2025
Trade statistics
Two separate and unrelated errors in UK trade statistics were recently identified during our routine quality assurances processes, one related to trade in goods data delivered to the
ONS by HM Revenue & Customs (HMRC), and one related to trade in services data processed by the ONS. Both errors were fully corrected on 28 March 2025 as part of the postponed UK Trade release, which now resumes its regular publication schedule, as well as being incorporated into the Quarterly National Accounts and the Balance of Payments. Additional quality assurance procedures, such as further congruence checks, have been put in place to reduce risk of re-occurrence.
Trade in goods
HMRC trade data and ONS trade data are published on different methodological bases: HMRC report trade based on goods physically moving across the UK border (known as a “physical movement of goods” basis) and the ONS report trade based on how goods change ownership between residents of different countries, which may involve the goods not physically crossing a border (known as an “economic ownership” basis). This difference in methodological approach complies with international standards. As a result, the monthly data will always differ.
When HMRC delivered the annual microdata to the ONS in December 2024, this was compared with the monthly deliveries as part of our routine quality assurance. A discrepancy was identified relating to the January 2023 reference period onwards, which was escalated to HMRC. Working with HMRC to understand the cause of the discrepancy and whether an error had occurred, HMRC determined that some imports data related to a specific Customs Procedure Code had been erroneously excluded from datasets sent to the ONS. The issue was caused by a technical change being implemented in error to the long-standing process used for extracting imports information shared with the ONS.
HMRC resolved the cause of the error in the data feed and provided the ONS with the correct data on 22 January 2025 for periods January 2023 to November 2024 (and on 18 February 2025 for period December 2024). On receipt of the data, we undertook further processing to bring it onto the required methodological basis. Once the indicative impacts of the error were understood and could be shared with users, this error was communicated by a public notice on 13 February 2025, in compliance with the Code of Practice for Statistics. The Code states that users should be promptly notified of errors or revisions, and that ‘corrections that result from errors, should be explained alongside the statistics, being clear on the scale, nature, cause and impact’.
Trade is one of the components within the expenditure measure of GDP. Due to the integrated nature of the National Accounts, the earliest a correction could be made was in the Balance of Payments and Quarterly National Accounts releases which were published on 28 March 2025. In light of the trade in goods error and reference periods impacted, the ONS exceptionally updated the National Accounts revisions policy to allow revisions back to Q1 2023 in the National Accounts publications.
Trade in services
A separate error was identified in our International Trade in Services (ITIS) survey results processing system, which impacted services imports and exports estimates from 2023. The error was identified as part of our quality assurance of the trade in services data feeding into National Accounts, when large data movements were identified following results processing. Due to the scale of the revisions, we undertook a deep dive into our systems which identified that some data related to some of our quarterly survey respondents had been erroneously excluded. This was confirmed as an error on 12 March 2025, 2 days prior to the announced publication date. The system issue was promptly rectified, with further processing required to re-run all affected reference periods and bring through all relevant data. The error was announced by a notice published on 13 March 2025, which included indicative impacts of the trade in services error, as well as communicating the decision to delay the 14 March 2025 monthly UK Trade bulletin.
This enabled us to bring all corrected trade data for both goods and services together on 28 March 2025 at the same time as the Balance of Payments and Quarterly National Accounts releases, the earliest a correction could be made in line with the National Accounts revision policy.
Producer Price Index and Services PPI statistics
Several business prices statistics, including the Producer Price Index (PPI), Export Price Index (EPI), Import Price Index (IPI) and Services Producer Price Index (SPPI), collectively referred to as the Producer Price Inflation (PPI) statistics, have been affected by an error with the way certain year-to-year linking methods were applied to the underlying data. This issue was identified as part of an ongoing programme to modernise the ONS’s statistical processing systems. Consumer prices series (CPI, CPIH and RPI) are completely unaffected.
The weights for business prices series used to be updated every five years to reflect changes in the economy. Starting with the publication in November 2020, these statistics were moved to an annual chain-linking methodology, which is the method of updating weights on an annual basis and statistically linking them to produce a time series which reflects changing sales patterns more quickly than under the five-year method. This move to increase accuracy and reliability was implemented alongside several other related methodological improvements.
The ONS has recently commenced a programme focussed on modernising our legacy statistical processing systems for business prices. As part of the preparation for this work, in mid-February we identified a potential issue with the chain-linking methods used to calculate these indices, including the treatment of item level price relatives (the ratio of an item’s price at a given time to its price at another time). Additional investigation, with the support of internal and external methodological experts, confirmed that the method had not been implemented as intended in 2020 and did not align with best practice. As soon as this quality assurance was complete, the ONS issued a public notice on 21 March 2025 and made key stakeholders aware of the problem and its expected implications. We subsequently paused the publication of PPI and SPPI statistics to rectify this issue, starting with the publication due on 26 March 2025.
This work was part of a broader work programme by the ONS to improve the quality of our PPI statistics, most recently in response to a suite of OSR recommendations. These have focussed on a number of specific areas, including:
• sampling
• data validation methods
• revisions review
• non-survey data sources
• imputation
• refining published data, and
• updating the Quality and Methodology Information (QMI).
OSR identified, and the ONS agrees, that further investment in business price statistics is needed. There remains a significant element of processing on legacy technology systems, primarily focussed on the processing of the item level price data submitted by our respondents. However, constrained resources may impact the pace at which we can move production off legacy systems.
The ONS’s latest assessment of the impact on PPI and SPPI
This problem affects the years from 2008 onwards. However, investigations suggest that the main impact on annual producer price inflation rates was in 2022 and 2023, because of the large movements in relative prices during that period.
As noted in the statement of 21 March 2025, whilst it is too soon to calculate the overall impact, our current assessment is that while the effects are likely to be mixed for different products and industries, we would expect the headline output PPI level to be revised upwards. Conversely, the headline SPPI level is likely to be revised downwards.
Plans for publication to resume and quality to be restored
The ONS fully recognises the challenges some users will face from the pause in our monthly business prices publications. However, given the number of series affected, and the legacy systems described above, our current corrective action is focussed on two principal workstreams:
• In the short term, we have chosen to prioritise correcting the detailed price deflators which are needed for high priority economic statistics, a substantial proportion of which rely on underlying inputs from PPI, IPI, EPI and SPPI. These include the inputs needed for our 2025 Annual National Accounts (Blue Book and Pink Book) revisions.
• Further ahead, the ONS will look to restart publication of monthly PPI and quarterly SPPI during summer 2025. The longer time required reflects the need for a significant rebuild of a fragile processing system, and time required to do sufficient quality assurance on the outputs.
The ONS is currently consulting actively with experts from our Technical Advisory Panel on Consumer Prices and key stakeholders (both internally to the ONS and externally) and will publish further updates in due course as these two workstreams progress.
Defined benefit (DB) pension wealth
Modelling defined benefit (DB) pension wealth is a complex area of economic statistics, particularly given wider changes to the economy and the heterogeneity of public sector pensions given changes to the schemes over time. To accurately estimate current values on future pensions promises both before and after retirement we need to make assumptions and methodological choices. We keep our methods under regular review, to ensure they continue to be fit for purpose should the wider economic or pensions context change.
The ONS commissioned the Government Actuary’s Department (GAD) in 2023 to review the modelling approach for valuation of defined benefit pension wealth as used in the Wealth and Assets Survey (WAS). This review was in response to the changing economic context in the UK, along with user feedback on assumptions and inputs into the DB pensions model, including from the Institute for Fiscal Studies (IFS). As part of this review, the ONS asked GAD to make recommendations on:
• the discount rate to use in the model,
• how increases to pensions should be recognised in the model,
• what review processes would be appropriate to maintain the model going forwards,
and
• valuing defined contribution (DC) pensions in payment wealth using the DB pensions
model.
In line with government user needs, stability was the primary objective, with a secondary objective being consistency in approaches used to determine the value of other types of wealth, and further objectives of practicality, durability, and continuity. Government user need for minimising undue volatility when valuing DB pension wealth was first highlighted to the ONS by WAS consortium members following the 2008 financial crisis. The previous market-based approach for DB pension valuation used in WAS waves 1-2 led to large changes to published pension wealth estimates between survey waves, which did not wholly reflect actual growth in respondents accrued pension amounts or entitlements, and as such more stable “market immune” approaches were preferred.
GAD’s review concluded that no discount rate could give the best outcome against all of the objectives. Consideration of relative priorities led to a recommendation to use a consistent Superannuation Contributions Adjusted for Past Experience ‘SCAPE’-based discount rate in all parts of the pension model. The recommendations were discussed with a range of stakeholders, including the IFS, during the review and ahead of their report. It was not possible to achieve consensus on the methodological approach, however key government users showed consistent preference for stability. Recommendations were subsequently implemented in our 2020-2022 Household Total Wealth in Great Britain statistics, published in January 2025.
We strongly refute the characterisation of this methodological change as ‘an error’, rather that there is a difference in views on the appropriate methods to apply. Further detail around implementation of recommendations and impact on our wealth estimates were published ahead of the statistical release. In line with GAD recommendations, the ONS plans to undertake a full model review of WAS pension methodology every third round and initiate “out of cycle” reviews should major unexpected economic events occur. Meanwhile, we will continue to engage with WAS consortium members and stakeholders via relevant technical and expert user groups.