1. Quality Assurance of Public Data [SA(RC)(23)33]
Sarah Henry (Office for National Statistics (ONS) Director, Methodology and Quality) and Penny Babb introduced a discussion item on the Quality Assurance (QA) of published data. (This item was not part of the substantive agenda, but was a discussion item before the meeting started).
The Committee heard that ONS and other producers were increasingly publishing a wide range of data tables to accompany their statistical estimates. OSR had been discussing the opportunities and challenges this presents with Sarah Henry, with a focus on quality assurance. Given the volume of data published, it was not always possible for the producer of the data tables to undertake the same level of quality assurance as done for headline estimates. The Committee considered the issues arising, including how producers could signal to users the nature of quality assurance for such published data.
Members discussed the update. The following points were considered in discussion:
the importance of maintaining confidence in the statistical releases that had been fully quality assured, if subsequent issues were found with data tables;
the possibility of a group of trusted users being given early access to data as part of an extended QA process;
that if users were to undertake more QA, this relationship should be codified; and
the importance of clear communication.
The Committee welcomed the opportunity for the discussion and acknowledged that this was complex. It agreed in the value of publishing a wide range of data tables. There was support for further consideration of the issues in this area including the impact on designation of official statistics and trust in the headline data set, and communication with all users including the media. It would be helpful to be able set out the policy on QA as the work progresses.
2. Apologies, minutes and matters arising
Marianthi Dunn and Catherine Davies were welcomed to the meeting, and were shadowing the meeting.
The minutes of the meeting on 5 September were agreed, and actions were reviewed.
3. Update from the Director General for Regulation [SA(RC)(23)34]
Ed Humpherson provided the Committee with an overview of the Office for Statistics Regulation’s (OSR’s) priorities and activities and highlighted some of the key areas of focus since the last meeting.
On 9 October OSR had published its interim report as part of its review of statistics on gender identity based on data collected as part of the 2021 England and Wales Census. The report set out OSR’s expectations of what was needed from ONS to provide assurance on the gender identity data and harmonised questions in line with the Code of Practice for Statistics.
Ed Humpherson had attended the One Big Thing event on 11 October in the Newport office speaking about data quality and ethics.
Members discussed Assessment Report 351: Business Demography Statistics, Assessment Report 364: Statutory Homelessness in England Statistics and Assessment Report 379: Weekly Statistics on Accident and Emergency waiting times in Scotland to consider for National Statistics accreditation. The Committee agreed the proposal by OSR to confirm accredited official status for these statistics. The Committee also approved arrangements for the publication of the Assessment Report on Achievement of Curriculum for Excellence Levels Statistics at the end of October.
4. Update on Quality- Focused Assessment Programme [SA(RC)(23)35]
Marianthi Dunn introduced a paper setting out OSR’s progress in developing the Spotlight on Quality (SoQ) programme and the emerging findings from the latest pilot of the SoQ programme, assessing ONS’s Profitability of UK Companies and Gross Operating Surplus (GOS) of non-financial corporations statistics.
Members discussed the emerging findings and OSR’s proposed requirements, which had been shared with ONS. Members noted that OSR had identified a number of significant issues which would need to be addressed. The following points were considered in discussion:
the continued relevance of GOS and its relationship with national accounts and Gross Domestic Product (GDP);
the need for ONS to improve the quality assurance of the production process of these statistics;
the importance of effective communication with users which should not inadvertently impact negatively on perceptions of GDP; and
including an ‘effect on GDP’ section into the introduction of the report.
The Committee noted the update. A further discussion was scheduled for the December Regulation Committee meeting which would cover accreditation of the statistics.
5. Rapid Review of Revisions to Gross Domestic Product Estimates [SA(RC)(23)36]
Marianthi Dunn introduced a paper summarising the emerging findings from OSR’s rapid review of revisions to Gross Domestic Product (GDP) estimates, produced by ONS. OSR’s overall judgement was that in the context of the difficult job of measuring economic output in near real time, revisions were inevitable and that ONS should be applauded for both the speed and initial preliminary estimates of GDP. In order to better manage media coverage there was a need for ONS to improve the way that it communicated uncertainty.
Members heard about the importance of revisions to GDP data, and that OSR recommended ONS increases efforts to refine and review its current methodology and improve its sources.
Members discussed the update. The following points were considered in discussion:
not all users of revised GDP data were expert users, meaning that the need for revisions were not always well understood, hence the importance of clear communications by ONS to show how revisions were made and to provide more material on how National Accounts are constructed;
the importance of economic as well as statistical expertise in communicating revisions to National Accounts, (the Committee asked whether information could be provided on turnover in senior statisticians); and
the importance of statistics producers accompanying statistics releases with an explanation of how statistics fit into a wider economic narrative.
The Committee agreed to review the final draft report ahead of publication at the end of October or early November.
6. Office for Statistics Regulation Performance and Maturity Model Update [SA(RC)(23)40]
Rob Kent-Smith introduced a paper providing the Committee with a mid-year update on the progress against OSR’s 2022/23 business plan (Plan-on-a-Page) and five-year strategy (Maturity Model), including the current risk profile. Planning was in place for the development of OSR’s 2024/25 business plan, including how OSR will engage with the Committee at key stages. It was noted that that OSR’s financial position had improved.
Members discussed the maturity model and how future plans may be altered by a General Election, and that this would be a challenging time for statistics producers and regulators. There was a need for OSR to be flexible about different resourcing approaches, and the need for transparency regarding resourcing.
Members discussed the Strategic Risk Profile, in the context of the ‘Capability’, ‘Voice’ and ‘Independence’ risks. It was noted that the risk profile was being managed effectively with controls and mitigating actions in place.
7. Sex, Gender, and Gender Identity Data and Statistics [SA(RC)(23)37]
Gail Rankin introduced a paper updating the Committee on OSR’s progress on the sex and gender identity programme, specifically: reviewing guidance published by others; updating OSR’s existing draft guidance on collecting and reporting data about sex in official statistics, and extending the guidance to include gender identity; and OSR’s review of gender identity and the Census 2021 outputs.
Members were updated on key findings from the research undertaken by OSR, a comparative review of a selection of guidance documents of others.
Members discussed the update. The following points were made in discussion:
the research by OSR contained useful information and Members asked OSR to consider whether the findings should be published (recognising this was not the original intention);
the announcement on 3 October by the Secretary of State for Science, Innovation and Technology of the review, led by Professor Alice Sullivan, of the collection of research and statistics by all public bodies on sex and gender;
the opportunity to use the research findings to frame the resulting guidance in order to give users a sense of how terminology is currently being used and understood, recognising that this is a dynamic field; and
the need to engage all stakeholders in the production of the guidance.
The Committee commended OSR on the progress of work in this area. Members supported the inclusion of definitions into the guidance based on the findings of the research undertaken; and providing clarity for producers on data collection. As part of OSR’s development approach plans were in place for further stakeholder engagement, including with Devolved Administrations.
A further discussion was scheduled at the December Regulation Committee meeting, which would include consideration of the draft guidance, ahead of publication in late December or early January 2024.
8. Transformation of the Labour Force Survey [SA(RC)(23)38]
Vicky Stone introduced a paper providing an overview of the ONS’s Transformed Labour Force Survey (TLFS) and OSR’s views on ONS’s work to date. Members were invited to discuss and note the key challenges of the transition, agree the proposed approach, and note the challenges faced in the final publications using the existing Labour Force Survey (LFS).
Members discussed the update. The following points were made in discussion:
the value of a discussion on the LFS and TLFS by the Authority Board ahead of further consideration by Regulation Committee;
the extent to which the LFS response rate had declined, and how to use LFS to quality assure the TLFS given the impact of response rates on quality;
the financial and resource implications of extending the LFS;
current stakeholder’s concerns around LFS and whether there could be unintended consequences around cost-of-living statistics by introducing the TLFS; and
the importance of communicating discrepancies between the surveys clearly to minimise uncertainty and avoid reduction in confidence in the statistics.
The Committee noted the update. A further paper was scheduled for the December Regulation Committee meeting, which would provide a progress update of action taken by ONS against each of OSR’s recommendations. It was agreed that an update on the LFS/TLFS would be scheduled for the November Authority Board meeting.
9. Review of the Quality of Police Recorded Crime Statistics [SA(RC)(23)39]
Job de Roij introduced a paper detailing the emerging findings from OSR’s review of the quality of the police recorded crime statistics. Members were invited to comment on the findings and discuss the options available to OSR to develop recommendations.
Members heard that Police forces welcomed this review, and that while there had been significant improvement to crime statistics recording since 2014, the main gap in understanding related to national level crime statistics.
Members discussed the update. The following points were made in discussion:
the need for OSR to be confident in making recommendations to police forces if that is where the issues reside;
the onus on Police force leaders deciding to prioritise investing in crime data collection, and their reliance on legacy systems;
the role of the Home Office in monitoring the quality of recorded crime data;
how different police forces have different challenges in collecting crime statistics, dependent on size; and
the need for police forces to share best practice and learn from each other.
It was confirmed that the Committee would receive a further update on this at the next meeting in December.
10. De-Designation- Cancelling or Suspending Accreditation [SA(RC)(23)41]
Penny Babb introduced a paper describing OSR’s policy and approach for cancelling or suspending accreditation of official statistics, previously referred to as dedesignation. The Committee heard how dedesignation was rare, a result of careful consideration, and an important part of the way OSR treat accredited statistics.
Members discussed the possibility in appropriate instances of OSR presenting statistics producers with a range of issues to improve and a clear deadline before suspension. They also discussed that some statistics outlive their usefulness, and the possibility of organisations deliberately seeking dedesignation of their statistics.
The Committee noted the update and asked OSR to clarify the process prior to the suspension of statistics.
11. Horizon Scanning
Ed Humpherson facilitated a discussion on Horizon Scanning, and emphasised the need to hold producers to account in a way that leads to outcomes that serve the public good.
The Committee also referenced the upcoming General Election.
12. Any Other Business
The Committee would next meet on Thursday 14 December.
