1. Presentation – Population Statistics Transformation – ONS Dynamic Population Model
The Chair welcomed representatives from the Office for National Statistics (ONS) noting that the Regulation Committee had discussed the role of the Office for Statistics Regulation (OSR) regarding the accreditation of the future population and migration statistics at previous meetings.
The ONS delivered a presentation on the work by ONS on the transformation of population and migration statistics with a focus on Admin Based Population Estimates (ABPEs) in England and Wales – provisional and updated estimates by age, sex and year at local authority level. More timely mid-2023 estimates had been published in December 2023, as statistics in development.
The Committee heard that assurance was provided by the Methodological Assurance Review Panel (MARP), which considered all methodological work. The potential for a Technical Advisory Panel was being considered to complement expertise from MARP.
The Committee welcomed the presentation, particularly the work being done to strengthen assurance including the potential for a Technical Advisory Committee; and resilience in terms of the expertise needed for such complex methodologies. The Committee discussed the likely realistic timeline which together with an understanding of the future model would help inform OSR’s regulatory role.
2. Apologies, Minutes and Matters Arising
The Chair welcomed everyone to the meeting.
The minutes of the meeting on 14 December were agreed and actions were reviewed.
3. Declarations of Interest
The Director General for Regulation recused himself from item eight – OSR’s assessment of Personal Independent Payment given his role as Vice Chair of Motability Foundation Charity.
4. Update from the Director General for Regulation SA(RC)(24)01
The Director General for Regulation provided the Committee with an overview of the OSR’s priorities and activities and highlighted some of the key areas of focus since the last meeting.
The Committee heard that the Authority Board had considered a paper on the future of population and migration statistics on 25 January, which had provided clarity on the proposed way forward. Aside from OSR’s regulatory role there was the potential to work with the ONS from the perspective of how the Census reflects and supports social cohesion – an important and underappreciated aspect of how statistics are serving the public good.
OSR’s People Survey results were overall positive. The Senior Leadership Team (SLT) and the Wellbeing and Development team were in the process of reviewing the results with a focus on managing change, discrimination and bullying and harassment.
Members discussed the update noting the positive results and the work ongoing in areas of concern. While the bullying and harassment score did not evidence a systemic issue across OSR, the work by OSR would help provide colleagues with an opportunity to discuss and signpost what to do about it. The Committee highlighted the drop in score (although still good) in the area of ‘my organisation motivates me to achieve’ as another potential area of focus.
5. Draft Business Plan and Budget SA(RC)(24)02
OSR presented the draft OSR Business Plan for 2024/25 and the proposed associated budget.
As agreed at the December Regulation Committee meeting the business plan priorities from 2023/24 were being extended into 2024/25, the final year of the Authority’s five year strategy. Significant engagement had been undertaken since December and was ongoing, which had provided the opportunity to inform stakeholders of plans and provide comments. The rolling regulatory programme of work underpinned delivery of the plan.
With regard to the budget it was noted that the Authority’s finance team had supported the proposed approach, subject to consideration by the Committee at this meeting. OSR proposed to operate in 2024/25 within the revised baseline budget of £3.4 million agreed in 2023/24. An additional £150,000 ringfenced contingency budget for the general election was proposed, which was also supported by the Authority’s finance team.
Members discussed the draft plan. The following points were considered in discussion:
OSR’s ability to effectively manage its budget year on year;
the opportunity to consider the size of OSR given the potential breadth and complexity of OSR’s remit as part of the next strategic planning cycle;
the inclusion in the plan about OSR’s work around resilience and reference to Artificial Intelligence (AI); and
the transparency of the business plan through stakeholder engagement, and the inclusion of some evaluation of how OSR had performed in 2023/24.
The Committee supported the draft plan 2024/25, approved the associated budget and commended the draft plan for consideration and final approval by the Authority Board at its March meeting.
6. Sex and Gender Guidance SA(RC)(24)03
OSR introduced a paper which provided an update on the progress of OSR’s sex and gender identity programme of work, including the engagement undertaken to develop the guidance, an updated draft of the sex and gender identity guidance and OSR’s review of gender identity and the 2021 Census.
The Committee heard about the extensive engagement that OSR had undertaken in developing the guidance with statistics producers and wider stakeholders. The guidance aims to address common challenges producers face and provide support for statisticians approaching this topic, perhaps for the first time, by prompting areas of consideration in order to make an informed decision in a proportionate way. The focus of the guidance was about producers being clear about what they were collecting and why.
Members discussed the draft guidance in detail. The following comments were made in discussion:
The importance of OSR’s guidance for statistics producers in order to help them navigate the decisions they should consider – consistent with the level of quality needed in the data.
The need for statistics producers to provide clearly defined terminology as part of a narrative underpinning the data collection, based on users’ needs with proportionality a key consideration. Producers should also consider how the data collection would be received by respondents. (The guidance itself would not assume a particular question or set are used to collect data about an individual’s sex and gender identity).
The need for producers to consider the characteristics that in practice affect people’s experiences. It was noted that in some instances more detailed data collection may be needed, in others, it might be appropriate to live with imperfection.
The impact of the guidance, which placed a lot of responsibility with the Heads of Profession and linked to OSR’s recommendation that a discussion by the National Statistics Executive Group (NSEG) would be useful as the group that provides oversight at a strategic level across the statistical system.
The gap currently across the Government Statistical Service (GSS) and the need for more guidance by the GSS Harmonisation team, including identifying expertise across the system; taking a convening role to support HoPs; and establishing a programme of work to share best practice and establish harmonised definitions and common standards that would build on OSR’s guidance.
The Committee commended OSR on the work undertaken in reaching this point. The Committee agreed the draft guidance subject to further consideration by OSR in a small number of areas. The summary of key points should be brought forward in the document. It should be clear that it was not OSR’s role to recommend the definitions as set out in the document, which was a decision for producers. The guidance could also reference data linkage; and cover the issue in relation to producers providing separate guidance for specific questions (and the implications of it not being read). The final guidance would be circulated to members for approval by members ahead of publication.
The Committee endorsed the recommendation by OSR that a discussion is scheduled for the NSEG. Members noted that a communications plan was being finalised with the aim of publishing towards the end of February.
7. Transformation of the Labour Force Survey Update SA(RC)(24)04
OSR introduced a paper which provided an update on the ONS transformation of the Labour Force Survey (LFS) including its work to address data quality issues, and the timetable for publishing Transformed Labour Force Survey (TLFS) estimates.
Since the last update to the Committee at the December meeting ONS had been working to implement its planned improvements to address the LFS quality concerns and to subsequently reintroduce LFS based estimates in its Labour Market release. As the work had taken longer than planned the ONS had continued to suspend the use of LFS-derived data in its labour market release. ONS had informed OSR that it would be publishing an article, including LFS headline figures on 5 February.
The Committee heard that ONS had met with its steering group (earlier in the week), which included a range of stakeholders (the Bank of England (BoE), Department for Work and Pensions (DWP) and HM Treasury (HMT)), to update them on progress and the timeline of changes in relation to the transformation.
Members discussed the update and noted the importance of statistical producers, ONS in this instance, communicating effectively with users throughout the transformation, being clear on challenges being faced and the impact of a changed timeline on users. Members also highlighted concern around the potential impact of the extended dual running of the LFS and TLFS on wider ONS delivery.
The Committee would continue to remain engaged with this topic as work progressed, with the further update scheduled for the April Regulation Committee meeting.
8. Personal Independent Payment Assessment SA(RC)(24)05
OSR introduced their draft assessment report on Personal Independence Payment statistics for England and Wales, produced by the DWP. It was noted that DWP had requested an assessment of its quarterly Personal Independent Payment statistics.
The Director General for Regulation recused himself from this item due to a conflict of interest in his role as Vice Chair of Motability, the charity supporting the motability of disabled people.
OSR’s assessment highlighted the value and potential value of the statistics. One requirement was identified for DWP, which focussed on improved user engagement, in order to meet the standards of the Code of Practice for Statistics (the Code), along with Accredited Official Statistics designation.
Members approved the report noting the positive outcome. It was agreed that OSR would share the report with the Office for Budget Responsibility ahead of publication.
9. Code of Practice for Statistics Review SA(RC)(24)06
OSR introduced a paper which set out the findings from engagement activities held between September and December 2023 to gather feedback on the Code, as part of OSR’s review ‘Futureproofing the Code’.
The Committee heard that the review had provided evidence that the Code is highly regarded by those producing official statistics. The three pillars provide a clear framework – Trustworthiness, Quality and Value (TQV). Feedback highlighted areas that linked to the three pillars, e.g. integrating OSR’s concept and guidance on Intelligent Transparency as part of Trustworthiness; and the strengthening the Value pillar by providing guidance on the use of new technologies such as Artificial Intelligence.
Including feedback on the content of the Code a small number of respondents had used the call for evidence as an opportunity to comment on the early release of market sensitive statistics by the ONS. This related to the application of the Code rather than its content. The majority of respondents had asked to revert to 09.30 release time with one respondent referencing research undertaken that had concluded a release time of midday was the most effective time to release.
With regard to the proposed new badges the call for evidence had covered a number of areas: new badges for official statistics; views on moving from the National Statistics badge to accredited official statistics; and a badge for official statistics in development. There was majority support for adopting the language of ‘accredited official statistics’ in place of National Statistics. The badges proposed for official statistics and official statistics in development had produced contrasting views. As such OSR had concluded that there was not sufficient evidence to introduce badges for official statistics and official statistics in development. The call for evidence had also provided responses regarding voluntary application of the Code beyond official statistics.
Members discussed the findings from the review. The following points were considered in discussion:
The range of the feedback from the review, which had highlighted the effectiveness of the TQV framework.
Consideration of the current policy as set out in the Code, with the standard release time currently 09.30; and in circumstances when it betters serves the public good for statistics to be released at an alternative time.
The implications of extending the voluntary application of the Code and its impact on the size of OSR. A model with OSR outsourcing compliance including the potential for peer review, could create a significant workload for OSR in administering it and may not provide the assurance producers were looking for.
The ability to bring the work on badges for official statistics to a conclusion without the need for any further work on redesign.
The Committee noted the good engagement by OSR on the review of the Code and supported the proposed approach. The range of feedback on the Code could be incorporated by enhancing the three pillars. Accessibility of the Code could also be improved. Both of these areas would be considered further by OSR.
The Committee supported the approach for OSR to undertake further work to review the alternative release time for ONS market sensitive statistics, including discussions with the ONS, the BoE and HMT. It was noted that any change to the current policy would require further discussion by Regulation Committee and approval by the Authority Board.
The Committee agreed a change from the current National Statistics badge to a badge with the wording – ‘Accredited Official Statistics’; and agreed the badge design. There was no requirement for any further research on the badge design. As recommended by OSR the Committee agreed not to introduce new badges for official statistics and official statistics in development.
The Committee was supportive of the proposal to consider the voluntary application of the Code in light of the feedback provided through the review. Members highlighted the importance of accountability and the impact on OSR in terms of administering a new model. Any change to OSR’s remit could form part of the strategic direction of the Authority.
10. OSR’s Role: Transformation Programmes of Work SA(RC)(24)07
OSR introduced a paper which set out how it would mitigate its strategic risk on transformation when regulating transformation programmes across the statistical system. It was noted that there were a number of ambitious transformation programmes being undertaken across the statistical system.
OSR proposed a transformation regulatory framework that would use the Code as a foundation and take a flexible approach to regulate transformations as needed depending on the size and complexity of the programme. This approach would help statistics producers by setting out OSR’s expectations on specific programmes. OSR would continue its regulation of transformation projects while developing this framework.
The Committee supported the proposed approach. As noted in the business plan paper OSR would be undertaking a review of the transformation of the Family Resources Survey (produced by a ONS, and the National Centre for Social Research on behalf of DWP), and the revised framework could potentially provide the basis for it. A further update would be provided at the next meeting.
11. General Election Planning – Lessons Learned SA(RC)(24)08
OSR introduced a paper which summarised the lessons learned from the 2019 General Election and recent engagement around misinformation to inform OSR’s approach to supporting the statistical system during a general election.
The Committee heard about the challenges OSR might face in the pre-election period and how the lessons learned from the 2019 election and engagement since then would help inform OSR’s approach in three key areas: making sound judgements, improving the speed of interventions (in line with the policy) to address misinformation; and exploring new channels for intervening beyond formal letters.
Members discussed the update. The following points were considered in discussion:
The need for OSR to be clear about its role during this period and consideration about the extent to which it could work with partners e.g. Full Fact.
The value of the letter by the Chair of Authority to be issued to all UK political parties reminding them of their duty to use statistics accurately and fairly during the election campaign and the consequences of not meeting them.
OSR’s role in undertaking a light touch internal review of manifestos focussing on the use of data and statistics, but not providing any specific assurances.
Support for Heads of Profession including a session in March to update them of OSR’s election plans.
The Committee noted the paper and supported the proposed approach by OSR in the pre-election period.
12. Artificial Intelligence SA(RC)(24)09
OSR outlined the key aspects of AI development including how AI may be used by producers of statistics. It was noted that currently across government use cases were in the discovery phase and that the use of AI models for public facing sites was advised against because of the risk of a prompt injection attack. Taking an example of translating code from legacy systems into open source software the experience to date showed that given the time needed to test the code it would be quicker to write the code from scratch. If AI were used to link data there would be a lack of transparency that could impact on the quality of outputs and also on public trust and trustworthiness of the organisation.
Members discussed the update noting that the challenges and potential impact on the size of OSR in regulating the use of AI in statistical production given its complexity and fast pace of development. These factors would need to be considered as part of the Authority’s next strategic plan. OSR could approach this issue by prioritising the key significant risks in terms of quality and assurance of use of AI in production of statistics, as well as understanding the key opportunities.
The Committee noted the update and supported the proposed approach by OSR, who would continue to engage in cross-government networks. It was agreed that a session on AI would be of value to all members of the Authority Board.
13. Horizon Scanning
The discussions at this meeting had highlighted areas for further consideration including AI and ONS transformation programmes.
14. Any Other Business
The Committee would next meet on Thursday 11 April.
