Ahead of the formal Regulation Committee meeting starting, the Chair welcomed Emma Rourke and Mary Gregory from the Office for National Statistics (ONS) to provide an update on the work ONS was doing around the topic of gender identity in light of the recently published Scottish census results.
The Committee heard that further evidence from the Scottish Census would help quantify uncertainty in the England and Wales results and understand where the data collection might be improved. ONS were open to learning from the National Records of Scotland (NRS) publication and data produced by the Scottish 2022 census. ONS need to undertake further work to provide additional guidance for different types of users. The ONS is working with the Government Statistical Service and Devolved Administrations to consider future data collection.
The Committee noted the update and discussed how the issue posed challenges around harmonisation. Members suggested that ONS consider supporting users by conducting sensitivity analyses using plausible assumptions on different sources of uncertainty and bias. The Committee noted that the impact of uncertainty was particularly relevant at the local level.
2. Apologies, minutes and matters arising
The Chair welcomed everyone to the meeting. Apologies were received from Gail Rankin.
The minutes of the meeting on 6 June were approved and actions were reviewed.
3. Update from the Director General for Regulation SA(RC)(24)26
The Director General (DG) for Regulation provided the Committee with an overview of the Office for Statistics Regulation (OSR) priorities and activities and highlighted some of the key areas of focus since the last meeting.
The Committee heard that OSR secured impact during the election period through good judgements, particularly focused on evidence being used in line with the principle of intelligent transparency. OSR had also done manifesto gap analysis to look for gaps in recent and planned regulatory work.
Members discussed the use of the excellent election information material and explainers and whether these could be used in future, highlighting for example that local elections would take place the following year which could make use of the material.
The Committee agreed accredited official statistics status for the Department for Environment, Food and Rural Affairs Butterflies Statistics, and the Department for Work and Pensions Personal Independence Payments Statistics.
4. Gender Identity Statistics Final Report SA(RC)(24)27
OSR introduced a paper providing an update and draft report on the gender identity data in the 2021 England and Wales census, including the OSR draft recommended judgement on accreditation of the gender identity census data and draft recommendations for future work on the harmonised question.
The Committee heard the following points:
OSR’s interim report had been published in October 2023 on the review of statistics on gender identity based on data collected as part of the 2021 England and Wales census; and included a number of recommendations for ONS;
the interim report noted that OSR would consider the accreditation status of the gender identity data subset following publication by ONS of its research findings;
National Records of Scotland had published Scotland’s census 2022 data for sexual orientation and trans status or history on 27 June alongside a document on quality assurance;
OSR were not confident that the statistics met the standards of the Code of Practice for Statistics (the Code) and OSR were minded to recommend cancelling accreditation of the gender identity data subset. OSR noted that accreditation of the census data as a whole should remain unchanged;
OSR set out the range of options it had considered in reaching its conclusions; and in reaching the proposed recommendation OSR had considered compliance with the Code, the need to help users understand the quality both of the gender identity data and the census dataset as a whole.
Members discussed the update. The following comments were made in discussion:
the publication of the relevant Scotland Census data and analysis was an important new input in understanding the issues. The report was helpful in understanding the data subset and key issues with the census question and subsequent data, and compliance with the Code, particularly in relation to ONS’ communication of uncertainty;
the levels of uncertainties with the gender identity estimates and the need for the data to be reliable. It was suggested that OSR’s report should reflect the need for ONS to explore the impact of systematic biases by using plausible assumptions;
the lack of user trust in the gender identity census data and the lack of clear and focused guidance on how to use and interpret the data;
in terms of the overall accreditation of the England and Wales census 2021 it was considered that cancelling accreditation of this data subset would help with credibility and ensure confidence in the census as a whole is maintained;
one option considered was to cancel accreditation of the data at a local level only as this was where the most significant uncertainties and biases lay, however given the national data are built up from local areas members noted the decreased confidence in the national level data, and the potential to undermine Census statistics more widely;
the approach by OSR to accrediting a set of statistics such as census. Given that the question was new to the Census, the gender identity subset of statistics could have been labelled as a statistics in development rather than including it in the overall accreditation;
ONS’s approach to publicly addressing concerns and communicating uncertainty on this data should have been better communicated;
recommendations for future alignment across nations should be considered to make the best use of data in future; and
the need for OSR to review its Sex and Gender Identity Guidance following this discussion and the publication of NRS data and to consider the impact of this experience on OSR’s approach to accrediting statistics on new and hard to collect concepts.
In conclusion, the Committee agreed with OSR’s judgement that there was sufficient evidence that the gender identity question in the census had underperformed. It was agreed that the Committee would consider the final report in correspondence to inform a final decision on accreditation.
5. Transformed Labour Force Survey Update SA(RC)(24)28
OSR provided an update on regulatory work in relation to the ONS Transformed Labour Force Survey (TLFS). It focused on the quality and decommissioning framework for the Labour Force Survey (LFS) which would be agreed at the July Authority Board meeting.
Members discussed the update and highlighted that that data quality metrics were an important part of the framework alongside user confidence and operational readiness. It supported the intention that any major changes to the TLFS should be tested experimentally to assess the impact of different options. Implementing weighting methodology was complex and required specialist expertise. ONS would need to consider its user communication around the dual run of the LFS and TLFS surveys and around decommissioning of the LFS.
Members discussed the potential for compulsion with reference to the recommendation in the Lievesley Review that the Authority should engage with the Cabinet Office to explore the consequences of mandatory completion of the Labour Force Survey. The DG for Regulation proposed engaging with this issue with a strategic think piece on compulsion more widely, similar to the Data Sharing and Linkage Report.
Members noted the update.
6. Economic Statistics Update SA(RC)(24)29
OSR provided an update on the review of Economic Statistics produced by the ONS including an assessment of the progress ONS has made in implementing the recommendations of the Barker-Ridgeway Review and the Bean Review.
The Committee heard that OSR had engaged with ONS to discuss the scope of the review. Terms of reference for the review would be published on the OSR website in due course. The review would assess progress in a number of areas including; adoption of new data sources, technological enhancements in relation to data collection, improved granularity and coverage and responding to challenges such as reductions in the response rates to some surveys.
Members discussed plans for further user engagement, noting the importance of retaining the right level of expertise and effective resource allocation. It was noted that OSR would take a staged approach to the review, which would help identify any organisational design issues in relation to major transformations.
Members noted the updated and supported the ongoing work on the review of Economic Statistics produced by the ONS.
7. Admin-Based Population Estimates Assessment Report SA(RC)(24)30
OSR provided an update on the draft assessment report of the Admin Based Population Estimates (ABPE) produced by the ONS. OSR findings had been cross referenced with Professor Wisniowski’s findings to produce the final draft report which had been shared with the ONS and the Welsh Government.
Members discussed the update noting that Professor Wiśniowski’s findings were valuable to the process and needed to be drawn out more in OSR’s assessment report. Members discussed the potential use of a matrix box to ensure that the recommendations were clearly referenced for transparency. ONS would be asked to produce a plan of how it would implement Professor Wiśniowski’s essential recommendations. Members noted that it should be clear that ONS was tasked to do this work as part wider Government ambition, as well as part of the ONS Governance structure.
The Committee approved the report subject to the changes mentioned, which was due to be published the following week.
8. Code of Practice Review SA(RC)(24)31
OSR provided an update on recent activities relating to OSR’s refresh of the Code, outlining planned next steps for the project, including plans for the Code public consultation process in the autumn/winter 2024.
Members discussed the update noting that value for money should be referenced under value. The explanation of official statistics and accredited official statistics could be brought out more clearly, and more clarity was needed on where OSR’s guidance on intelligent transparency fits into the Code. Clarity was needed in what was meant by producers and users of statistics, particularly in the context of Ministers who may play a role under both.
Members supported the Code refresh and agreed it was an opportunity to communicate intelligent transparency and Trustworthiness, Quality and Value
9. Regulation of Transformation Activity SA(RC)(24)32
OSR provided an update on the actions OSR were taking to mitigate the strategic risk on transformation and the draft public position for regulating transformation programmes which was planned to be published later in the year following further engagement with Heads of Profession for Statistics (HoPs).
Members discussed the capacity for OSR to be involved and have oversight at all levels of a transformation process and the implications that would have on OSR resources. OSR had been prioritising its work to the areas that could add most value across the system, this had led to a focus on more systemic work and less review of individual statistics than had historically been the case. OSR were due to pilot a rapid assessment to test what a two to three hour Code review would look like.
The Committee supported the work on transformation and agreed that case studies would further support the report.
10. Developing OSR Strategy and Organisation SA(RC)(24)33
OSR provided an update setting out the proposed focus of ‘OSR 2.0’ aimed at developing the next five-year business plan, priorities, and supporting people’s development.
Members discussed how to engage more with the public and increase public awareness of OSR work. Members discussed the Public Administration and Constitutional Affairs Committee recommendation that OSR expand its remit to IT work and how OSR would embrace this.
The Committee noted the update.
11. Annual self-effectiveness review SA(RC)(24)34
Secretariat provided an update on the results of the annual self-effectiveness review for the Committee.
Members discussed the update noting that the Committee could perhaps provide more challenge for OSR in terms of drawing out the clear links to delivery of the Authority Strategy, particularly around public understanding. As part of the process for the next Strategy the OSR team would engage early at Authority level and with HoPs in order to influence progress.
The Committee noted the unambiguous appreciation for the quality of support received from the Secretariat and thanked everyone involved.
12. Horizon Scanning
The DG for Regulation raised the point that a new Government would mean new priorities and the Committee should expect more focus on Government departments, as opposed to solely ONS, in future agendas.
Members agreed that analytical understanding was important and OSR would look into the Government data and evidence base to assess gaps over the following year, in line with a recent recommendation from the Public Administration and Constitutional Affairs Select Committee.
13. Any other business
The Committee would next meet on Thursday 19 September.
