2. Strategic Introduction from the National Statistician
Sir Ian Diamond led a strategic discussion on the road to the 2023 Recommendation and the future for the Methodological Assurance Review Panel.
The panel confirmed that the remit of MARP will continue beyond its current focus of 2023 Recommendation assurance. The panel questioned its wider role to assure areas of research additional to the 2023 Recommendation evidence. Sir Ian Diamond elaborated on the importance of MARP in bringing an additional level of assurance to the ONS to bolster statistical quality. The suggestion was for MARP to occasionally undertake “deep dives” into other methodological areas to provide assurance for the National Statistician. The panel suggested that future deep dives could be assured by sub-committees, chaired by one of the panel members if needed. The ONS highlighted that it has a forward agenda for future quality deep dives.
Sir Ian Diamond and the panel agreed that the efficacy of MARP may be compromised if membership were expanded too much, however Sir Ian Diamond suggested the addition of one extra specialist.
The panel questioned whether an expanding role for MARP would mean the panel becoming more embedded within the ONS structure. If so, future rotation of panel members and the development of a written protocol for its future remit were suggested.
ACTION: ONS and MARP to write a working protocol for continuing MARP beyond 2023 Recommendation assurance.
3. 2023 Recommendation Evidence Roadmap
Becky Tinsley presented on the evidence roadmap to the 2023 Recommendation.
The panel commented on the bimodal distribution in the mid-year populations estimate for a Local Authority with a high concentration of student populations. Becky Tinsley informed the panel that these patterns may highlight areas of the mid-year estimates’ internal migration methodology that required further exploration.
The panel suggested that the presentation be published on the MARP UKSA site.
ACTION: The panel suggested that the presentation be published on the MARP UKSA site.
The panel highlighted the need to carefully consider the transformation of social statistics such as income and ethnicity, areas where there may be only limited information. Becky Tinsley clarified that it is possible to reproduce some topics at a local, multilevel using admin data and suggested coming back to MARP with a deep dive into these statistics.
ACTION: ONS to bring a deep dive into transformation of characteristic statistics such as ethnicity.
The panel discussed the ways that ONS will achieve coherence from the multiple hybrid methods with different strengths and the need for a developed integration framework. ONS clarified that the different approaches will feed into the Bayesian Methods for Demographic Accounts to achieve coherence.
The panel and ONS discussed the mechanism ONS will have for getting agreements for admin linkage to the Longitudinal Study.
4. Social Survey Methodology
Orlaith Fraser presented a short options paper on how MARP could be used to assure some of the social survey transformation research.
The panel agreed with the ONS to continue discussions on these options and other assurance questions.
5. EAP174 – Bayesian Methods for Demographic Estimation
Louisa Blackwell, Duncan Elliott and John Bryant presented on Bayesian Methods for Demographic Estimation.
The chair suggested that the slides from the presentation be published as an appendix along with the paper.
ACTION: Append presentation on Bayesian Methods to the original paper for publication.
The panel evaluated comments made by correspondence to confirm that all responses left in advance of the meeting had been appropriately resolved.
The panel suggested condensing some explanations of well-known concepts within the paper. For instance, making just a brief explanation and standard reference to Markov Chain Monte Carlo.
The panel was concerned that these new methods were experimental but also apparently central to the 2023 Recommendation evidence roadmap. Therefore, the panel highlighted the vulnerability of relying on Bayesian methods to bring coherence to the statistics and questioned whether the ONS had pursued alternative options in parallel for contingency and to assess quality, such as iterative proportional fitting (IPF).
ONS clarified that they had evaluated the vulnerability of the proposal and had explored alternative and more simplified approaches, for instance IPF. The panel suggested that ONS engage further with consultants on IPF methods in the future.
ONS also reiterated its commitment to be innovative with its statistics to arrive at its best-possible estimate, highlighting that the transformation journey also consisted of three options in train to provide additional contingency.
The panel was satisfied by the level of detail presented and that the ONS had explored possible alternative methods. The panel was also satisfied that the ONS had undertaken adequate assumption checking and simulation.
The panel also discussed with ONS and John Bryant on the use of priors that are strongly informative. It was agreed that for these applications including prior information was appropriate but that this needed testing.
6. Census Update
ONS provided an update on Census operations.
7. Actions Update
The panel agreed that ONS will publish its summary of quality assurance processes for address checking to the MARP UKSA site. Action 79 closed.
The panel agreed that Rachel Skentelbery and Dominick Thompson will develop a methodological briefing on the differing definitions of variables and bring to the panel in the future. Action 81 closed.
The panel agreed that Ana Basiri and Jon Wroth-Smith will produce a piece on software code assurance. Action 84 closed.
The panel agreed an action to ask ONS to bring a revised paper on Variance Estimation
ACTION: ONS to bring revised paper on Approaches to Variance Estimation.
