Regulation Committee minutes: 11 February 2021

Attendees

Members Present

Other Attendees

Secretariat

Apologies

Declarations of Interest

Members were welcomed to the meeting. The minutes of the meeting on 10 December 2020 were agreed, and actions were reviewed.

The meeting received a presentation from Stephen Balchin and Lucy Vickers from the Department of Health and Social Care (DHSC) on the Department’s response to the COVID-19 pandemic. Members heard about the role of organisational structures and culture, and some of the challenges the Department had faced, including the roll-out of Test and Trace. To ensure good practice was upheld, the team had worked with the Office for Statistics Regulation (OSR) and the Best Practice and Impact team in the Office for National Statistics (ONS).
Members discussed the importance of capturing key learnings from the experience, which could be used as case studies in the future. The Committee also discussed the importance of leadership and departments working together to achieve a common goal. Members noted thanks for the presentation.

Mary Gregory presented an overview of OSR’s response to the COVID-19 pandemic, including the introduction of rapid reviews of new statistical outputs, statements and guidance for producers, and responding to a significantly increased volume of regulatory casework.
Delivering impact had been a high priority for OSR. To achieve this, OSR had built on its relationships with departmental Heads of Profession (HoPs) and the media and varied its communication methods, including greater use of statements and blogs in addition to formal letters. OSR had seen the extent of its impact through formal corrections and media mentions of the “statistics watchdog”. It had also seen improvements in the presentation of data and statistics in the public domain.
The Committee commended OSR on its response to the pandemic and delivering this impact at a crucial time, noting that much of its impact would be behind the scenes at a working level within departments. Members recommended OSR keep a record of the lessons learned, and continue to deliver a significant impact in the health statistics space.

Ed Humpherson introduced his update, which focused in particular on the review of exam results. The latest draft of the report had been reviewed by UK exam regulators and shared with the Committee for final comments. Members noted it was a fair and well-written report. The report would be published in late February or early March.
The paper had also made reference to ONS economic statistics on Gross Domestic Product (GDP) and the Labour Force Survey. The Committee was supportive of OSR’s plans in this space.

Mary Gregory introduced a paper outlining OSR’s current risk profile. The Committee had previously agreed to review OSR’s risks every six months. The meeting heard that OSR’s risks were split into four areas: Relevance, Voice, Capability, and Independence. Since the last report, the risk scores had remained broadly the same, with a slight increase in the Capability risk due to the impact of the COVID-19 pandemic on staff resources. This would be a focus for OSR over the next period and managed in line with agreed mitigations.
The Committee noted the update and would revisit in another six months.

Mary Gregory presented an early draft of the OSR business plan and work programme for the financial year 2021/2022. The paper set out the initial direction and priorities for the coming year, structured around five key themes. The work programme would remain flexible and OSR was engaging with users to inform the plan.
The Committee discussed OSR’s plans within the Health and Social Care domain, ensuring these remain strategic and are informed by, and support the needs of, users like DHSC. There would be an ongoing need for good data and statistics on the pandemic to inform policy decisions. OSR might also consider giving greater emphasis to its aims on secure data availability which was briefly mentioned in the draft plan. The draft business plan and work programme would be refined further and return to the Committee in April for approval.

Mary Gregory presented a paper outlining the two strands of OSR’s work associated with automation and Artificial Intelligence (AI). Internally, OSR was making use of web scraping and data science techniques to support its regulatory work, to help understand public concerns, and identify data gaps. Longer term, OSR was developing a series of dashboards for each domain to complement its face-to-face user engagement.
OSR’s Regulating AI workstreams had expanded over time, supported by a programme of upskilling within the team to build capability. These streams included advocating for greater use of automation including Reproducible Analytical Pipelines (RAP), guidance for users on the use of statistical models, and reviewing the Code of Practice for Statistics to ensure it remained relevant to new production techniques.
The Committee was supportive of the RAP review, which had been shared by correspondence, and OSR advocating RAP across the Government Statistical Service. OSR had been engaging with the HoPs network to understand the blockers for using RAP techniques, and would be using its assessment programme to advocate for greater use of automation. The Committee was supportive of the work to date, and OSR’s ongoing plans in this space.

Mark Pont and Iain Russell introduced the ongoing assessment of ONS Productivity statistics. Members were invited to discuss the emerging findings ahead of the formal report being presented to the Committee in April. Key points had included:
1. engagement with users to better understand and reflect their needs;
2. describing uncertainty, including uncertainty around data;
3. progress in developing new methods;
4. providing a single platform for productivity information;
5. ONS’s approach to the “productivity puzzle”; and
6. plans for international comparisons.
The Committee noted this was a large and important piece of work, particularly in relation to economic recovery after the pandemic. Members discussed the user demand for micro-level data and for international comparisons in particular. To help meet this need, ONS might consider collaborating with others such as the Organisation for Economic Co-operation and Development (OECD) to share resource. The assessment report would be shared with the Committee at its April meeting.

Four papers had been issued for comment by correspondence. Members had submitted written remarks in advance of the meeting and had no further comments on these papers.
There was no further business. The Committee would meet next on Thursday 15 April at 13:00.