• 10:30 – Introduction (Sir Bernard Silverman)
• 10:35 – Labour Market Survey methods review (Owen Abbott, James Brown, Ray Chambers)
• 10:50 – Actions Update (Secretariat)
• 10:55 – Action Update: “Right to Withdraw” question (Will Laffan, Matt Ford)
• 11:15 – Coverage Estimation using a PECADO approach (Eleanor Law, Amy Large)
• 11:45 – Paper EAP207: Admin-based Living Arrangement Estimates research to date and plans (Alison Morgan, Sally Myers) Paper to be published later in 2024
• 12:10 – Making an admin-based census available for research purposes (Alison Morgan)
• 12:30 – Break
• 13:00 – Paper EAP206: Ethnicity Harmonised Standard review (Charlie Hales, Zara Burman) Paper to be published later in 2024
• 13:35 – Paper EAP205: Methodological evaluation and quality assessment of Reference Data Management Framework (RDMF) overview (Tom Hunter, Michael Cole, Rosalind Archer)
• 14:20 – Paper EAP208: International Migration Statistics, current position and plan (Dominic Webber)
• 14:50 – Forward Agenda (Secretariat)
• 14:55 – Any other business
• 15:00 – Close (Sir Bernard Silverman)

Panel members

• Professor Sir Bernard Silverman (Chair)
• Professor Ana Basiri
• Dr Oliver Duke-Williams
• Carl Emmerson
• Professor Nik Lomax
• Professor Natalie Shlomo

External academics (only for Labour Market Methods review)

• James Brown
• Ray Chambers

Office for National Statistics

• Owen Abbott
• Sarah Henry
• Charlie Wroth-Smith
• Katie O’Farrell
• Mary Gregory
• Will Laffan (ONS presenter)
• Matt Ford (ONS presenter)
• Eleanor Law (ONS presenter)
• Amy Large (ONS presenter)
• Alison Morgan (ONS presenter)
• Sally Mylles (ONS presenter)
• Dominic Webber (ONS presenter)
• Charlie Hales (ONS presenter)
• Zara Burman (ONS presenter)
• Tom Hunter (ONS presenter)
• Michael Cole (ONS presenter)
• Rosalind Archer (ONS presenter)
• Ceejay Hammond (ONS presenter)

Observers

• Peshali Diyasena
• Steven Dunstan
• Mark Edward
• Emma Grant-Holt
• Anita Arif
• Daniel Thomas
• Charlotte Hassell
• Robyn Hunt
• Gareth James
• Paola Serafino
• Isaac Shipsey
• Tom Tarling (ONS Secretariat)
• Cristina Spoiala (ONS Secretariat)
• Susan Williams (ONS Secretariat)

1. Welcome and introductions

1. Sir Bernard welcomed the group and confirmed those in attendance.

2. Labour Market Survey methods review

1. The National Statistician, Sir Ian Diamond, has requested a review of Labour Market methodologies including the Labour Force Survey (LFS) and the proposed online Transformed Labour Force Survey (TLFS). The ONS has reached out to two academics (Ray Chambers and James Brown) to review this work.
2. The outcome of this review would be a report.
3. The external academics noted this review was independent, and they would be giving opinions and determining whether the LFS and TLFS were fit for purpose.
4. The Terms of Reference (ToR) for the Methodological Assurance Review Panel (MARP) allow for the panel to review specific topic matters at the National Statistician’s request. ONS suggested involvement of the panel during this specific review would be valuable.
5. The chair agreed, noting that MARP would review the process undertaken by the external academics rather than the content of the report. A subcommittee of the panel will be involved with this assurance. The panel agreed the external academics were appropriate for the review.
6. The chair stated that based on this review, MARP would provide advice to the National Statistician and the report would be published via MARP.

Action:

ONS to provide the subcommittee with a summary of TLFS relevant papers that have already been to MARP in the past for review.

3. Actions update

1. From the previous meeting of MARP, ONS noted that requested follow-ups and comments on previous items had been completed following that meeting in full.
2. The ONS noted a large forward agenda upcoming for the panel, driven by the Future of Population and Migration Statistics (FPMS) and proposed further meetings in August 2024, October 2024 and January 2025. The chair agreed and noted that online-only shorter meetings to discuss single topics might be needed.

4. Action Update: “Right to Withdraw” question

1. Responding to an action from MARP 33, ONS provided a more detailed answer to the legal basis and statistical impacts of the right to withdraw consent.
2. The chair pointed out that legal items are outside the scope of MARP, and that their concerns are about the impact of any withdrawal of consent such as the “right to be forgotten” on statistical production.
3. ONS confirmed that most data processing by ONS is based on both ‘public task/public interest’ and, for special category data, ‘statistical purposes’ as the legal basis, rather than relying on consent.
4. ONS confirmed that individuals have certain rights available to them, for example, the right to object and the right to be forgotten. Many of these rights have qualified/absolute exemptions for data processed for statistical purposes.
5. The panel were confident from the ONS update given that there would be no issue to ONS statistical production from the right to withdraw.

5. Coverage Estimation using a PECADO approach

1. ONS presented slides showing progress on population estimation research using an approach used by the Central Statistics Office (CSO) Ireland called Population Estimation Compiled from Administrative Data Only (PECADO). ONS showed the coverage error (both under-coverage and over-coverage) from this estimation approach when compared to mid-year estimates by age-group and sex.
2. The panel discussed the size of the coverage errors, trimming methods, signs-of-life, the model for inclusion rules and the Dual System Estimation (DSE) methodology used. The work to date is showing that the methods are not meeting the quality standards with the administrative data available.
3. ONS is constructing stricter rules that go into SPD and models, to assess the best approach to go forward.
4. An issue with undercoverage of 0-year olds shown in the data was discussed. ONS believes issues are likely caused by lags in updated child benefit data, and some data where a newborn wouldn’t be present in the dataset for a full year.
5. The panel suggested there could be under-coverage caused by instances where child benefits were not claimed, or claims were delayed, resulting in missing data for some of the youngest ages.
6. ONS asked the panel if there were any other approaches they should explore.
7. MARP suggested treating coverage similarly to the 2021 census. Multiple System Estimation could also be explored.

Action:

ONS to contact the MARP chair to follow-up on Multiple Systems and relevant papers.

6. Admin-based Living Arrangement Estimates research to date and plans

1. The ONS presented the paper and responded to the comments made.
2. The panel commented continued international collaboration on this wider problem was important, and asked if household estimates are based on the Labour Force Survey (LFS). The ONS confirmed this is the case, but the LFS data can’t go to lower-level geographies.
3. The panel recommended making use of weights and weighting. Addressing user need for unit level data was discussed.
4. The panel commented the desires for continuity and innovative methods are opposing, and noted to ONS it wouldn’t be able to fulfil every user need simultaneously.
5. The panel suggested the ONS use council tax data.
6. ONS mentioned that council data can be used at address level but with care, as data quality issues exist within council tax data. The panel commented that UPRNs were an important and valuable concept to push, and use.
7. The ONS noted it would consider bringing the research plan on this topic to the next MARP meeting.

7. Making an admin based census available for research purposes

1. ONS presented slides showing the proof of concept for a record level administrative based census.
2. The panel noted that the geography levels used were Local Authority (LA) and Lower Super Output Area (LSOA) and asked if Output Area (OA) would be better. ONS commented that currently there were more gaps in the data at this level, and the disclosure risk increased.
3. The ambition is to get to this level, but for the proof of concept the focus is on LSOA and adding further variables to the dataset.
4. The ONS stated the desire for this to be UK-wide, but the decision on Scotland and NI data is for those Devolved Authorities. The ONS target is for yearly data.
5. The panel questioned if the admin-based census will be subject to long-term archiving. The panel advised that design decisions here need to be considered from a very early stage.
6. The panel commented that published data should include coverage weights, as consistency would be valuable across researchers. ONS responded it will aim to have consistency with admin-based population estimates as the product is developed.

8. Ethnicity Harmonised Standard review

1. The ONS provided a paper to the panel for comments prior to the meeting, and presented responses to the comments in the session.
2. The panel discussed user alignment with their own datasets, the heterogeneity of user needs, the need to prioritise stakeholders, missingness not at random (MNAR) and ‘prefer not to say’ response options. The panel recommended engaging with The Runnymede Trust on the broader topic of ethnicity standardisation.
3. ONS noted that consistency and comparability were key aims and it would not mandate to data collectors how questions should be asked.
4. The panel reiterated that some of their comments were pointing towards further research ideas, rather than requirements for the existing project, and this work shouldn’t attempt to address all of them.
5. There was further discussion around the topic of “write in answers”.
6. The panel commented it’d be good to review questions over-time, rather than having a single look at a point in time. The ONS commented it planned to review questions every 5 years.

Action:

ONS to follow-up on recommended datasets mentioned by panel.

9. Methodological evaluation and quality assessment of Reference Data Management Framework (RDMF) overview

1. The ONS presented the paper and responded to the comments made.
2. The panel agreed that they are content with the QA approach but also queried what approach would be taken for user training. ONS responded that providing training is beyond the scope of methodological design, but did recognise the need for it.
3. The panel suggested doing simulations and case studies, and using machine learning approaches. ONS responded there are papers coming to future MARPs that would be linked to this.
4. ONS confirmed matching rates, precision and recall will be included for users.
5. ONS stated that other government departments, and Integrated Data Service (IDS) researchers were the main targets for the framework. IDS supports linkage for a wide set of admin data, and an indexing service for data users.
6. A discussion around differentiating “error” and “change” was had, with ONS stating its pragmatic target was for ‘acceptable noise and change’. ONS agreed this was a problem to address in the longer term, and something to investigate.

10. International Migration Statistics, current position and plan

1. ONS presented a summary of the research plan with International Migration.
2. Research is transitioning from use of the International Passenger Survey to place more use on administrative data held in Home Office Borders and Immigration Data (HOBID) and the Registration and Population Integration Database (RAPID) held in the Department for Work and Pensions.
3. The panel commented on the usage of visas within the data, noting if there are changes to visa duration this could lead to big lags and significant revisions in statistics. The panel recommended discussing with experts in Home Office to increase knowledge.
4. The panel questioned the assumption that visa holders who are not observed to having left the UK by their visa due date are considered to have left. ONS agreed that it is difficult to quantify the uncertainty around this assumption and that it is a pragmatic decision to assume they have left.
5. The panel queried if there was any scope for bilateral international statistics. ONS said it had looked at mirror statistics, but noted it would be very difficult due to differing quality.

11. Any other business and forward agenda

1. There was a discussion in setting dates for a few future meetings rather than only for the next meeting and so August and October dates 2024 will be set in advance.
2. There was no further business. The chair closed the meeting.

Actions:

105: Labour Markets: ONS to provide the subcommittee with a summary of TLFS relevant papers that have already been to MARP in the past for review.
106: Labour Markets: ONS noted a paper on weighting had been to MaRAG, a preceeding panel to MARP, and should be taken to this subcommittee.
107: PECADO: ONS to contact the chair to follow-up on Multiple Systems and relevant papers.
108: Ethnicity Harmonised Standard: ONS to follow-up on recommended datasets mentioned by panel