Executive summary

In November 2022, the nations of the UK signed a statement of agreement on cooperation on future UK population and social statistics, committing the parties to engagement across a range of areas. The statement of agreement requires that an annual report on implementation be presented to the Inter Administration Committee.

 

Over the past year, statistical offices across the United Kingdom have continued to explore the future of their population and social statistics, with nations at different stages of the process. Progress has been made in the various areas of cooperation, ranging from data sharing to allow for coherent UK population statistics, to the development of additional harmonised standards, to the maintenance of a common narrative to provide users of UK statistics with information about cross-UK cooperation.

However, activity has been uneven: some cooperation areas will grow in relevance as our respective systems continue to transform. As work advances, the parties must ensure that the enablement of fora aiding smooth information and expertise sharing and practical support on topics such as data acquisition, methodological developments and more is sufficiently prioritised, alongside effective communication.

Introduction

In November 2022, the National Statistician, the Registrars General for Scotland and Northern Ireland, and the Chief Statisticians of Scotland and Wales signed a statement of agreement on cooperation on future UK population and social statistics. It was signed in the context of a complex period of transformation, as each nation looked to make greater use of administrative and other sources of data to support the production of population and social statistics.

The statement of agreement sought to build on longstanding cross-UK cooperation, and identified ten areas for continued engagement, including to maximise the harmonisation of inputs, methods, and outputs across the UK. It committed signatories to the preparation of an annual report on its functioning for the Inter Administration Committee. This inaugural annual report provides that update and considers in particular the progress made against the ten areas for cooperation specified in the agreement.

Assessment of current UK context

Since November 2022, positions on the future of population and social statistics have continued to evolve in all nations of the UK:

In England and Wales, the Office for National Statistics (ONS) has published a range of research into the use of administrative and alternative data sources to produce more frequent and timely population, migration and social statistics. From June to October 2023, the ONS ran a consultation on proposals to transform these statistics using these new methods and data sources, with administrative data at the core. Responses to the consultation, alongside other evidence, will inform a recommendation to the UK Government on the future of these statistics in England and Wales. Regardless of the outcome of this process, it is anticipated that the newly developed Dynamic Population Model (DPM) will support the future production of population and migration estimates in England and Wales, improving their accuracy over the ten-year period.

In Scotland, the National Records of Scotland (NRS) published the first results from Scotland’s 2022 Census in September 2023. As part of ongoing research into transforming population statistics, NRS also published an updated set of Administrative data Based Population Estimates (ABPEs) in August 2023. NRS has committed to set out lessons upon conclusion of the census programme in an evaluation report, laid before the Scottish parliament, in 2024. A recommendation will be made to Scottish Ministers on future census statistics in 2024. NRS and the Scottish Government are working in partnership to undertake research on the census in the context of transformation of the wider statistical system to inform these recommendations.

In Northern Ireland, the Northern Ireland Statistics and Research Agency (NISRA) continue to work alongside the ONS to ensure that population and migration statistics remain coherent, and to ensure that any impacts of changes to estimates for England and Wales on wider UK figures are understood. NISRA are also engaging with the ONS on data and knowledge gaps with a view to improving data supply and developing a better understanding of how the ONS are moving forward with new methods. In addition, given the interdependencies between the censuses across the nations of the UK, NISRA are engaging with the ONS to understand how a decision on the future of population and migration statistics, and any future census in England and Wales, will be communicated to users.

Evolution in the delivery of population and social statistics throughout the UK presents both opportunities to improve coherence (e.g. through early discussion and alignment of additional definitions), and the risk of creating differences between what specific statistical outputs are produced for England, Wales, Scotland and Northern Ireland. The Concordat on Statistics recognises that differences may exist as “official statistics should reflect local as well as UK user needs”, but underlines the value to users of “coherent/comparable statistics at the UK and disaggregated levels”.

Final recommendations and decisions on the futures of population, migration and social statistics in the different parts of the UK lie with the relevant authorities for each nation. Nonetheless, given the interconnected nature of these statistics across the UK, clarity and transparency between nations will be critical. A shared understanding of each nation’s direction will equip all decision makers to best set a direction for population and social statistics that works in the context of each nation, and for the UK as a whole.

Progress report

Work to maximise the harmonisation of inputs, methods, and outputs has built on earlier cooperation, such as frequent engagement at UK-wide fora and the adoption of Government Statistical Service (GSS) Harmonised Standards for a range of topics. A report on further steps taken over the past year, against the ten areas for cooperation in the statement of agreement, follows.

1. The parties commit to mutual support in the acquisition from third parties of administrative data for statistical purposes.

The parties have continued to share lessons learned in data acquisition. For instance, the Scottish Government has shared data acquisition process mapping documents with the ONS, to draw on combined expertise to streamline the process and jointly identify and troubleshoot future blockers. Additionally, NRS have shared information on their process for securing data sharing agreements in their acquisition of administrative data to supplement their census data, which will feed into future developments as a lesson learned.

Onward data sharing has also advanced. For example, the ONS has secured agreement to onward-share key data with NISRA. The ONS has begun discussions about a single ‘Front Door’ for all devolved data requests, to simplify the onward data sharing process, and the Inter Administration Committee will be notified about future progress in this area. The ONS will keep partners around the UK updated. Looking ahead, parties to the agreement have noted the importance of sharing priority identifiable administrative data for the future production of population and migration statistics.

Furthermore, the ONS, NISRA and Welsh and Scottish Governments have collaborated to identify priority actions in terms of joint acquisition efforts. The ONS has developed a proposed approach, beginning with a requirements mapping exercise, which would inform engagement with data suppliers; the next step will be for the parties to amend and/or approve the proposal. Should joint negotiation be overly burdensome in this case, mapping requirements would also pave the way for onward data-sharing to fill the gap.

While next steps have been identified, challenges in future joint negotiations may arise where the data acquisition priorities and timelines of each administration do not align. Such differences stem from the fact each administration has hitherto focused on different areas of work and associated datasets.

Going forward, it will be critical for the parties to the agreement to prioritise progress in data acquisition and onward sharing, to support shared ambitions to make the best possible use of administrative and other sources of data. This should be done whilst protecting existing data access agreements and associated delivery timelines. There should also be a focus on clear communication to ensure acquisition processes run quickly and smoothly.

2. The parties will continue to work together on cross-border data-sharing required to produce population and internal migration statistics.

The ONS continues to enact the latest version of the National Population Projections (NPPs) agreement, last amended in June 2021, which covers the production and publication of official NPPs for the UK and its constituent nations. The ONS will also keep engaging with the Devolved Administrations regarding the respective production of subnational and household projections.

As set out in the statement of agreement, were any changes to requirements to cause disproportionate burden on a party, discussions could be held around cost-sharing, as described in paragraph 24 of the Concordat on Statistics.

3. Best practice relating to the efficient use of administrative data sources, including the design of a pipeline to sustain a statistical system reliant on administrative data, will be shared openly.

Design work for a pipeline to sustain a statistical system reliant on administrative data is ongoing. This commitment will remain under review as work progresses. Currently, the ONS is leading the delivery of the Integrated Data Service, which will facilitate access to key alternative data and support statistical transformation.

4. New and developing methods being considered for producing statistics in scope of the agreement should be shared openly and proactively between the parties.

The ONS has continued to develop its proposal for ABPEs, and the parties have shared expertise particularly about the ONS’s proposed DPM. For example, a presentation was given to the UK Censuses and Population Statistics Strategic Group (UKCPSSG) in November 2022. The ONS and NISRA discussed the functioning of the DPM in a workshop in July 2023, with follow-up strategic talks in August. Similarly, a presentation on the ONS’s proposal for population and migration statistics was given to the Scottish Government and NRS in June, followed by a technical workshop on ABPEs in October. The ONS, NRS and Scottish Government held strategic talks in early November.

5. Common topics will continue to be used to provide the basis for common outputs, where necessary to meet common user needs.

Since the signing of the agreement, new topics have not yet arisen that require a common approach. This commitment will remain under review as the UK’s statistical system continues to develop.

6. Coherent definitions and classifications should be used across the UK.

Differences in user needs, data collection, and legislation across UK nations mean it is not always possible to have completely harmonised data or statistics. However, harmonisation has been maximised through steps such as the continued development of harmonised standards by the GSS. New publications since the signing of the statement of agreement include:

• Ethnicity – The GSS published research, reporting the findings of discovery exercises such as an unmoderated card sort and a data collectors and processors survey, in March 2023. In April 2023, they published research about ethnicity terminology work undertaken with the Cabinet Office Equality Hub, community engagement work and findings from an interviewer survey.
• Disability – Findings from the GSS research and engagement work were published in March 2023, followed by updated guidance for existing disability standards.
• Mental health – The GSS published the findings of research and engagement work for mental health in March 2023.
• Socio-economic background – The new harmonised standard was published in July 2023.

The ONS Statistics Head of Profession Office and ONS Coherence teams run Devolution Training sessions for analysts across government, providing guidance on collaborating across the UK when producing outputs relative to other parts of the UK, and promoting the benefits of doing so. All ONS analysts are required to undertake this training. Sessions are run regularly throughout the year and the training has reached over 750 internal staff since the course started, and more colleagues across other government departments through events such Devolution Week, Analysis Function Week, and for the Cabinet Office.

The ONS’s proposed transformation of population and migration statistics raised the prospect of incorporating additional definitions for population bases, such as daytime, termtime or seasonal populations. Noting the commitment to use cross-working mechanisms to ensure a UK approach where changes to definitions are considered, the ONS shared draft questions ahead of the consultation launch, initially via an Inter Administration Committee meeting in April 2023. Subsequently, parties were informed as questions were developed in response to feedback throughout May and June, and definitive versions of the questions were shared ahead of the consultation launch. Findings from responses to the consultation will be shared in due course.

7. Where sources or methods for outputs in scope of the agreement differ, as a priority cross-working mechanisms will be used to maximise the harmonisation of outputs.

The ONS is currently consulting on proposals for a future population and migration statistics system in England and Wales. The ONS’s ambition is that the proposed methods for official annual population estimates will secure National Statistics status in 2024. It is therefore highly likely that, of the topics in scope of the statement of agreement, population and migration statistics will be the first to require the use of cross-working mechanisms to maximise the harmonisation of estimates produced using different methods. Consequently, the UKCPSSG has been identified as right forum for the most imminent discussions.

The UKCPSSG submitted a paper to the October 2023 Inter Administration Committee meeting proposing an approach to harmonised, UK-level population estimates under a transformed system in England and Wales. Such work would build on existing cooperation to produce UK-wide population estimates, including following the 2021/22 census round. The paper proposed that comparable official population statistics be maintained and put forward a minimum viable product that users could expect.

Methods for producing statistics on other topics are not yet developed to the extent that they require changes to UK-wide harmonisation efforts. A new forum (or fora) is likely to be required in due course, as new topics are added; this will be kept under review.

8. The UK Statistics Authority’s ethical principles are to be followed as changes to methods and sources for statistics in the scope of the agreement are considered. The National Statistician’s Data Ethics Advisory Committee will be consulted as appropriate. Findings and recommendations of interest should be shared via cross-working mechanisms.

The UK Statistics Authority has produced a summary of their and the ONS’s engagement with the National Statistician’s Data Ethics Advisory Committee on issues relating to the ONS’s proposed transformation of population and migration statistics. This overview also includes information on other steps taken to follow ethical principles, such as the publication of a summary of research into public attitudes in June 2023.

9. The parties will share experience of regulatory assessments and checks against the Code of Practice for outputs relevant to the agreement.

To date, no new outputs in the scope of the agreement have been designated as National Statistics. The ONS’s ambition is for improved population estimates, produced using the DPM, to achieve this designation in the future. The ONS meets with the Office for Statistics Regulation on a regular basis to discuss progress. Information will be shared between the parties about the assessment of these statistics as appropriate in due course.

10. A common narrative will be agreed and updated at a reasonable frequency to provide users of UK statistics with information about cross-UK cooperation throughout this period of transformation in population and social statistics.

In October 2022, the UKCPSSG agreed a draft UK narrative on transformation. This narrative was most recently updated by the UKCPSSG in September, and will be kept under review, particularly following the outcome of the public consultation on the future of population and migration statistics in England and Wales, and the recommendation from the UK Statistics Authority to the UK Government.

Conclusion

Progress has been made in many of the ten areas for continued engagement identified in the statement of agreement, although it has been uneven, as transformation has advanced to greater or lesser extents for different topic areas in different nations of the UK. In addition, census outputs have necessarily been prioritised. However, as the final census outputs are published, work on transforming each nation’s respective population and social statistical system will gather pace, making the year ahead a crucial one to ensure that coherence and harmonisation are maximised as early on in the transformation process as feasible.

To prevent accidental divergence (beyond what is deemed necessary to respond to local contexts) it is crucial to ensure continuous dialogue, updates and expertise sharing is facilitated by appropriate fora and processes. Redirecting course towards harmonisation at a later stage would be more costly, time consuming and, ultimately, less effective.

 

Related links

Statement of agreement: cooperation on future UK population and social statistics