Dear Dame Meg,
As National Statistician and Chief Executive of the UK Statistics Authority, I am responding to your letter of 21 November regarding the Labour Force Survey.
The Office for National Statistics (ONS), as the UK’s National Statistical Institute (NSI) and largest producer of official statistics, delivers independent and relevant statistics and analysis, continuously responding and improving to ensure these are of high quality and meet user need. The ONS is delivering a programme of improvements to increase the quality of data from the Labour Force Survey (LFS). While we are starting to see the positive impacts of these recovery efforts, our continued work to stabilise the data in partnership with key users is our highest priority. Alongside this, we are progressing our strategic solution to the international fall in survey response rates and sizeable inflationary impacts, through the online-first Transformed Labour Force Survey (TLFS).
Recovery of the LFS
The sharp fall in survey response rates, a long-term trend that rapidly accelerated during and after the pandemic period, has been a significant challenge in the UK as well as for other NSIs around the world. There are many reasons for this decline, including increased cautiousness around the sharing of personal information, declining trust in government and public institutions, a reluctance to have interviewers inside homes and increased challenges accessing secure/gated properties.
The LFS involves five ‘waves’ of data collection – an initial survey interview, then eligible households repeat the survey each quarter on four further occasions to track changes in employment over time. The voluntary nature of the UK’s LFS as well as its significant length (around 45 minutes per wave per household on average) have meant LFS response rates have been more heavily impacted than other countries.
We are acutely aware of the significance of reliable labour market statistics as a source of evidence for economic decision making and that some indicators are only available from surveys such as the LFS. We do not underestimate the challenges involved with the use of data to inform the decision making of central banks in times of uncertainty, as the 2023 Bernanke Review set out.
Following the disruption of the pandemic and a period of substantial inflationary impacts with difficult prioritisation decisions, the ONS worked to re-establish high quality survey data collection, by re-introducing face-to-face data collection survey by survey, re-training interviewers and re-establishing the operation. By 2023, it was evident that societal behavioural changes as a result of the pandemic had influenced the data collection environment and were contributing to the survey quality not recovering at the rate anticipated. At its lowest point in 2023, the LFS response rate was 17.4% (UK, including imputed cases). This had significant implications for the quality of the statistics derived from the survey, resulting in the temporary suspension of the LFS as a source of labour market data in October 2023. Due to the dwindling response rates, we introduced the LFS Recovery Plan to restore the quality of its estimates, focusing on data collection improvements and methodological enhancements. The complexity of the survey and issues meant the interventions would take time to fully embed in the survey and subsequently improve the quality of the estimates.
As part of data collection improvements, we increased the targeted sample by over 50% in January 2024 from 16,700 to 25,800 new households each quarter, returning it to the levels adopted in the aftermath of the pandemic period. We increased face-to-face interviewing and the incentives for participation from £10 up to £50, with a particular focus on Wave 1 responses (our initial collection with a household). As a result, the achieved sample is now 15,000 higher than at its lowest point, an increase in the overall UK response rate from 17.4% in Jul-Sep 2023 to 24.6% in Jul-Sep 2024. We are currently recruiting an additional 150 interviewers to further support Waves 2-5.
Methodological enhancements have focused on the weighting of survey results to support quality. Weighting utilises population projections and estimates to ensure data collected from only a sample of the population produces outputs that are representative of the entire population. Population projections are usually provided once every two years but during this period they were impacted by the unusual migration pattern following the pandemic and the effects of changes to the immigration system following the UK’s exit from the EU. Therefore, the ONS introduced an additional partial re-weighting in February 2024 as well as for December 2024. LFS quarterly person data has been reweighted back to 2019 using updated population numbers. An article to illustrate the impact was published on 3 December 2024 and full results will be released on 17 December 2024. Reweighted two-quarter longitudinal LFS person outputs are expected in spring 2025. A more complete reweighting of all LFS data will follow during 2025/26, as well as Annual Population Survey (APS) data which makes use of LFS responses and additional sample boosts to deliver regional and local level statistics.
As noted above, the wave nature of LFS means data collection and methodological improvements take time to feed through fully into the estimates. Autumn 2024 has seen significant volatility and we have supported users to navigate the data uncertainty in this interim period. This includes being clear on our website, social media and broadcast media interviews on the data limitations and how they affect the use and interpretation of LFS estimates. Our expanded commentary recommends that, at this time, users make full use of all available data sources when assessing the labour market, such as HM Revenue & Customs (HMRC) data on the number of people on payrolls and the number of workforce jobs, which the ONS has developed and published. We highlight that these sources are currently likely to be providing a better read of recent trends in employment, particularly of employees.
Acknowledging the complexities of the challenges and the vital importance of these statistics to users, we have strengthened our engagement with stakeholders and channels for external challenge, support and expertise to inform our approach. We introduced a new monthly Technical Engagement Group in October 2023 with members from the Bank of England (BoE), HM Treasury (HMT), Office for Budget Responsibility (OBR) and the Department for Work and Pensions (DWP) to provide a forum to discuss upcoming developments and improvements in an open and transparent manner, which is providing invaluable input and feedback on our plans for both LFS and TLFS. In addition, in June 2024, we established a Stakeholder Advisory Panel on Labour Market Statistics chaired by Professor Jonathan Portes with representation from academics, think tanks, core government organisations and the Devolved Administrations, to provide independent advice and guidance on the production, publication, uses and application of labour market statistics and their technical aspects.
One focus has been on exploring how survey response challenges might be introducing potential bias in the LFS estimates, in particular for young people and ethnic minorities. This was considered at the Stakeholder Advisory Panel on 24 October 2024 and will be further addressed in a special session later this month that will also be attended by members of the Technical Engagement Group to assess whether any further methodological actions are necessary. We plan to extend our work on the coherence of key labour market data sources, with a further publication planned in the new year.
In summary, progress has been made in recovering the LFS with the achieved sample now significantly higher and the incorporation of the latest population information into the estimates. The major changes we have made to the LFS will be fully included through all five of the survey waves by the first quarter of next year, which will inform the LFS estimates for publication in May 2025.
Progress on the TLFS
Our long-term solution to falling response rates and quality challenges on the LFS is the development of an online-first evolution of the LFS called the Transformed Labour Force Survey (TLFS). The ambition of the TLFS is to improve the quality of our labour market statistics, provide a more adaptive and responsive survey to meet user needs and enhance respondent experience. While the online-first nature of the survey enables a far larger sample size than the LFS, to ensure quality of the survey and reduce bias, ONS field force still visit 3,855 addresses a week on the TLFS to encourage them to complete the survey, referred to as ‘knock-to-nudge’.
The ONS started a full parallel run of the TLFS alongside the LFS in October 2023 to determine whether the performance of the survey met operational and statistical quality criteria. Compared with the LFS, in this pilot the TLFS provides a larger achieved sample (quarterly datasets include more than double the number of people than LFS), a higher response rate at every wave (40% compared to 36% at Wave 1) and the potential for more stable weighted outputs. However, there was bias in response towards older age groups, higher levels of partial responses compared with the LFS, and quality issues with the online response to some more complex variables such as respondents’ occupation and the industry in which they work.
The analysis of the parallel run data for headline labour market outputs demonstrates differences in headline labour market outputs between the two surveys. While some of this difference is expected (e.g. the TLFS uses the latest labour market definitions and updated methods which differ from those on the LFS), we have a strand of work to further account for the differences and identify adaptations, working with users to understand the implications for outputs and manage how data are used in their own systems.
As part of our open approach and bringing in of external challenge and expertise around the survey design complexities, the ONS has enhanced the role of the Methodological Assurance Review Panel (MARP). Through MARP, we commissioned Professors Chambers and Brown to undertake a methodological and design review of the TLFS in April 2024. This review recommended new work on a shorter online survey with the capacity for designed modular additions, as well as the need to continue the parallel run for five quarters to assess how the surveys captured seasonal variations in the labour market. Based on this review and feedback from both internal and external users, in July 2024 we announced an extension of the parallel run of the LFS and TLFS and plans to test further design improvements within the TLFS.
Since July we have completed a series of discrete online design tests to assess the impact of a shorter TLFS questionnaire that aims to reduce average household completion time from 37 minutes to around 15 minutes. The test also included questionnaire changes to address bias, rates of partial response and collection of complex variables, as well as the use of a QR code to ease citizen response.
Early indications show positive outcomes from the design tests with some areas for development. We are now conducting a thorough evaluation and reviewing the content of the shorter survey to ensure it meets key labour market requirements whilst reducing respondent burden as much as possible. Pending evaluation of the test activity, an examination of our assumptions and engagement with our expert groups, there are several scenarios for when we are able to transition from using the LFS to using the TLFS, involving the implementation of the shorter survey and further periods of parallel run. Therefore, we cannot yet set out a firm timetable for transition but will layout potential timetables in quarter one 2025. While one scenario based on a shorter questionnaire is 2027, my ambition is to transition in 2026, with the timings being determined in collaboration with key users to ensure the TLFS meets their quality and system needs.
The development of the TLFS has provided a continuous opportunity for the ONS to learn and refine how it delivers this complex transformation project. In summer 2024, the ONS conducted an internal lessons learnt exercise including historical cultural issues. Our colleagues are committed and passionate about producing high quality labour market statistics and the exercise provided colleagues with the opportunity to contribute views with candour from across the totality of the project. A summary of the lessons learnt and the initial actions taken to address the issues is at Annex A. In particular, we strengthened technical and methodological leadership, including appointing two experienced senior colleagues to shore up analytical leadership capability. Additionally, the Director for Methodology is supporting oversight on the conditions for quality and the journey towards stability of the TLFS. We also raised awareness of channels that colleagues can use to escalate known issues and are promoting a culture that encourages the surfacing of risks and invites constructive challenge. While the report suggested a pause in development of the TLFS, the critical nature of the statistics and the decisions that flow from them means we are redoubling our efforts to improve and roll out the TLFS as soon as is practicable, with a continued focus on the wider issues raised.
International comparisons and further development
While some other countries do achieve higher response rates on their equivalent labour market surveys, many NSIs have also seen falling response rates in recent years. We are continually engaging with, and learning from, partner NSIs to boost response rates and reduce any response bias. The insights we receive from these NSIs are contributing to the ongoing evolution of TLFS design, specifically on the shorter survey, which harnesses the benefit of large-scale, online, self-completion.
The Independent Review of the UK Statistics Authority (the Authority) conducted by Professor Denise Lievesley earlier this year recommended that the Authority explore the consequences of mandatory completion of the LFS. Mandating would align with the census and business surveys and is akin to the civic duty of undertaking jury service. A move to mandating responses to the LFS would require legislation (as is the case for census and business surveys) and is not something the ONS can consider alone. We welcome a broader national conversation about the importance of citizens being represented in the country’s statistics and championing the value of data as critical national infrastructure.
Given the strategic challenges that surveys are facing, we are establishing a new Surveys Innovation Hub, expanding our portfolio of surveys research and are continuing to work closely with the Economic and Social Research Council (ESRC) funded Survey Futures project. On Survey Futures, the ONS is collaborating with other survey providers to explore the feasibility of using names and contact details to cross-reference with our address database to make contacting individuals easier. We are conducting a suite of research to improve our ability to reach the public, build trust and gain consent to overcome barriers to completion of our household surveys. We are also exploring the use of alternative data sources in our end-to-end survey design, for example enabling us to adapt our samples and focus operational effort on contacting those population sub-groups that are typically less likely to engage in our surveys.
The common picture across many countries, and the UK’s survey sector (public and private) as a whole, of falling response rates suggests that we can no longer rely on surveys alone. While there are some questions that only can be accurately answered by large surveys, such as the distinction between unemployment and inactivity, we have already begun publishing other data sources to build a fuller picture of the labour market. The Integrated Data Service (IDS), a functioning cross-government hub that allows full value to be extracted from data collected across the public sector and beyond, will seek to identify, share and link securely these and other data sources across government to further build our understanding of the labour market. LFS data and HMRC tax data (PAYE RTI) were linked through the IDS in November 2024 and this integration of survey data and administrative sources will be used to inform labour market quality assurance and survey methods. More effective sharing of de-identified administrative data across government will further improve the quality of statistics and evidence to support national and local decision making.
Working in partnership with our key stakeholders, and learning from other countries, we will extend our exploratory work on integrating survey and administrative data to produce a holistic picture of the UK’s labour market. This includes our plans to produce Labour Market Accounts, which aims to provide a comprehensive understanding of the UK’s labour market through the optimum balance of survey and administrative data.
Resourcing
As with other parts of the public sector, the Authority’s funding position throughout the Spending Review 2021 (2022/23 to 2024/25) has been one of constraint overall, including flatlined core funding, ring-fenced budgets, and substantial inflationary impacts. Operating within our budget in this context has led to difficult prioritisation decisions and the need to deliver efficiencies and cost savings across the organisation. Our efficiency and cost savings figure was approximately £11.4m for 2022/23 and £17.8m for 2023/24. We plan to deliver an estimated £12m in efficiencies and cost savings in 2024/25 taking the cumulative impact for the Spending Review 2021 (SR21) period to over £40m.
The work to prioritise activity and reduce costs started in 2022/23 and continued through 2023/24, impacted by the cost of increased colleague salaries and a notable inflationary effect on our cost bases, including through surveys. Significant cost saving measures to remain affordable, the need to dual run the LFS and TLFS and restrictions from ring-fenced budgets curtailed our ability to use the totality of organisational funding and dedicate the resources we would have ideally wanted to our social surveys operation.
The survey recovery plan in late 2023 set out how existing resources could be utilised to pivot to support the LFS/TLFS and other surveys to start to redress the downward trend. Given the skills and capabilities required to drive improvements and the scarcity of options, we further prioritised within the survey workforce. Simultaneously we set out our plans and a request for additional finance to HMT for the Surveys Quality Recovery Plan 2023/24, which was supported and the funding made available.
Prioritisation of surveys recovery formed a key component of our recent Spending Review 2025 (SR25) Phase 1 submission. Whilst funding for 2025/26 will remain at the same level as 2024/25 (flat cash) as part of our financial objectives we have been supported in significantly reducing the level of ring-fenced funding we receive as a proportion of the whole. This added flexibility will be vital for us in being able to pivot resources at pace in future and enables us to implement an LFS sustainability plan.
Throughout 2023 and 2024, the ONS’s three highest field data collection priorities have consistently been the labour force surveys (LFS and TLFS parallel run) and the Living Cost and Food Survey. Our decisions to not reapply for the contract to collect data for the National Survey for Wales for the Welsh Government in March 2023 and reduce interviewer resource on its financial surveys in February 2024 (including pausing the Survey of Living Conditions) reflect these priorities. Other surveys have also seen reduced support.
The additional investment allocated in the last year to recover the LFS and the ONS’s wider social surveys is enabling significant field interviewer recruitment. Our permanent face-to-face field interviewer community has grown from 477 (Dec 2023) to 544 (Nov 2024). Within the same timeframe we have also built up an agency workforce of 130 temporary field staff specifically to complete the ‘knock-to-nudge’ function on the TLFS, which releases an equivalent number of fully trained interviewers to support recovery across ONS’s social surveys. Compared with the 145 interviewers in October 2023, in the last month 275 face-to-face interviewers have worked on the LFS (across all waves). Recruitment continues, with a further 28 interviewers confirmed to start on the LFS in December to support improvements in Waves 2-5 response and recruitment ongoing to deliver the full requirement of 457 face-to-face interviewers across all LFS waves by the end of March 2025.
As an organisation, we fully comprehend the critical importance of high-quality labour market statistics and recognise the significant impact the response challenges are having on the reliability of data informing our key outputs. We are confident that by continuing to seek out internal and external challenges and expertise, progressing the improvements already made and delivering the solutions outlined above in partnership with our key stakeholders, we will be able to recover the quality issues with the LFS and continue our progress towards transition to the TLFS. Achieving a successful outcome from these programmes of work is the top priority for the ONS.
Yours sincerely,
Professor Sir Ian Diamond