On Tuesday 5 September, Professor Sir Ian Diamond, National Statistician gave evidence to the Public Administration and Constitutional Affairs Committee’s inquiry on the on transforming the UK’s evidence base.
A transcript of which has been published on the UK Parliament website.
On Tuesday 23 May, Sir Robert Chote, Chair of the UK Statistics Authority, Sir Ian Diamond, National Statistician and Ed Humpherson, Director General for Regulation, gave evidence to the Public Administration and Constitutional Affairs Committee’s inquiry on the work of the UK Statistics Authority.
A transcript of which has been published on the UK Parliament website.
On Thursday 20 October, Sarah Henry, Director for Methodology and Quality at the Office for National Statistics, gave evidence to the Public Administration and Constitutional Affairs Committee for its inquiry, ‘Civil Service People Survey’.
A transcript of which has been published on the UK Parliament website.
Dear Mr Wragg,
I write in response to the Committee’s call for evidence on the Civil Service People Survey. We have focused our evidence on questions in the Terms of Reference regarding survey design, delivery and validity of results, from the perspective of our role as the Office for National Statistics in administering high-quality national surveys including the census.
Staff surveys, and surveys in general, can adopt different strategies to protect respondents’ privacy. These can range from anonymising responses by removing any information that connects the survey to the respondents, to ensuring that analysis derived from the survey does not lead to disclosing identity.
Full anonymisation can limit analysis. For example, if different age groups have a different experience of working in an organisation, this would not by highlighted if the age field was removed to protect privacy. Therefore, best practice is to seek a compromise by using a range of measures to protect privacy. These include:
In addition, it is good practice to carry out privacy impact assessments and to make privacy notices and technical guides readily available.
To ensure the best design and delivery of a survey, you may want to be aware of the following:
Developing and delivering a survey of the scale of the People Survey is a multidisciplinary task that requires the involvement of many professionals to ensure it delivers on analytical and business objectives.
The information included in the People Survey should be based on the data user needs and the departments that will use it. As mentioned above, these can be ascertained through consultation with policy users. Comparison over time is always an important aspect of any regular survey, and we would recommend keeping question sets as comparable as possible from year to year with changes, when needed, following a transparent methodological review. Finally, some terms used within questions are subjective depending on the department – again this could be improved through consultation.
Periodically the topics covered will change and be impacted by other issues. A good example is the need to monitor the experience of working in the civil service throughout and following the pandemic. When adding or changing a metric, it is important to communicate and explain the changes, especially at the reporting stage.
Some departments may also need to consider organisational changes and how they would like them reported against previous years.
It would be difficult to quality assure the information provided via the People Survey. There are limited sources to cross-check the information, but these could be exit interviews and/or any internal departmental staff surveys. One approach to address this could be to do a quality follow-up survey with a sample of respondents – which is like what we do with the census to quality assure that data.
Non-response bias can have a huge impact. It can cause results to be distorted, and this is linked with wider issues, as typically people that don’t respond have a reason not to engage, and those reasons are particularly interesting for those collecting the data but remain unseen.
It also means that the data will not be representative and, as a result, any policy changes might not address the real issues. Methodological solutions include weighting and imputation and require comparing the population of respondents to the population of civil servants using the data that is available through HR departments. For example, if fewer people aged under 30 respond to the survey, the responses of those who have replied could be given a bigger weight. Any weighting strategy would need to be transparent and carefully considered, attempting to explain the assumption that people who have responded do indeed represent those who have not.
Strengths and weaknesses are mostly the result of trade-offs. For example, while the People Survey is relatively long, risking attrition, lower response rate and haste in completion, it does allow for more detailed analysis on many topics.
As discussed in this submission, using a consistent survey across the Civil Service enables efficiency, comparison between organisations, sharing of good practice and analysis over time while limiting bespoke design on issues that may be of interests to specific departments.
Again, as aforementioned, while the survey is not weighted, which enables quick access to the results, it does have an impact on how confident we can be that respondents represent the civil service as a whole. The survey is reported as percentages of respondents rather than a percentage of the population and users can break down the results further to compare responses from different groups. This is a pragmatic, clear approach which is clearly communicated.
A mixture of both quantitative and qualitative data collection could improve the quality of the analysis and the usefulness of the survey. The People Survey is quantitative, with a few open questions capturing free text. There are other qualitative measures such as depth interviews and focus group discussions that can be used alongside the People Survey to enhance understanding of the results. These can either be in addition to, or instead of some of the questions in the survey.
Finally, the survey is accompanied by a tool that enables quick analysis and comparisons, disseminated to all participating organisations; this is a strength.
My colleague Sarah Henry, Director of Methodology at the ONS, looks forward to discussing this further with the Committee on 13 September. Please do let us know if any questions ahead of then.
Yours sincerely,
Professor Sir Ian Diamond
Dear William,
Thank you for your letter of 14 June 2022 regarding census response rates in Scotland and the implications for the integrity of UK-wide data. I will answer your specific questions in turn, but first I wanted to emphasise the close working relationship between all UK Census offices. We have offered and provided support to National Records of Scotland (NRS) including sharing designs, seconding staff, and are now working with them to develop methods to maximise the accuracy of their Census estimates.
The Office for National Statistics (ONS) also worked with NRS to establish an international steering group, which is providing the highest quality technical expertise, advice and challenge to NRS on census matters. This group is advising NRS to focus efforts toward a good census coverage survey, particularly in regions where responses were lowest, and strengthen the use of administrative data to supplement census data sources in their statistical production, including a clear steer to prioritise the early acquisition of new administrative data sources. This oversight will offer NRS the best possible opportunity to deliver a high-quality outcome for Scotland that will, in turn, contribute to high-quality UK statistics.
The ONS has committed an internationally renowned team of experts equipped with decades of demographic experience to work alongside NRS. Through a combination of this world-leading expertise, the ambitious use of supplementary administrative data, and sophisticated estimation methods, I remain confident that together we will deliver robust UK-wide estimates of the population.
What assessment has been made of the reasons underlying the low response rate in certain areas, and what steps were taken to avoid this occurrence? To what extent is the separate delivery of the Scottish Census considered to have impacted the response rate?
The Census in Scotland is a devolved matter; our assessment of the reasons underlying a low response rate in some areas and the associated mitigations has therefore been formed through our close working partnership with NRS.
The decision by Scottish Ministers to move Scotland’s Census to March 2022 was informed by NRS analysis of the potential impact of COVID-19 on the quality of an operation in March 2021. NRS adopted a diverse and inclusive approach to public awareness around census through media and physical advertising, follow-up reminders, field staff, and focused efforts in areas of low return; approaches comparable to those of the ONS and the Northern Ireland Statistics and Research Agency (NISRA). Evidence gathered in Scotland at the end of the collection phase reported that ‘too busy’ and ‘not aware of the Census or the need to complete it’ were the more common reasons given by householders who had yet to return.
As already mentioned, in light of lower than anticipated returns, the ONS and the Registrar General for Scotland established an international steering group of census and coverage experts. Despite these challenges in the collection phase, having considered the position and the planned next steps with the census in detail, the steering group have confirmed that there is a stable foundation from which to move
from census collection onto the next stage of the census operation, namely the census coverage survey and the incorporation of administrative data into estimates. It is the combination of census returns, coverage survey, administrative data, and estimation methodology that will deliver high quality census outputs for Scotland.
What are the implications of Scotland’s lower response rate for the quality and comparability of UK-wide population statistics?
By taking the actions outlined above, through a combination of census data, supplementary administrative data, and sophisticated estimation methods, we believe that it will be possible to deliver a high-quality outcome for Scotland that will, in turn, contribute to high-quality UK-wide population statistics.
What actions will be taken to quality-assure the Scottish Census data in order to reduce the impact of the lower response rate on the standards of UK-wide population statistics?
NRS remain committed to continuing to produce the best possible population estimates for Scotland, which will be used to produce UK-level population estimates. They currently produce annual official population figures for Scotland using data from a range of sources including the Census, registration data on births and deaths, migration estimates, and a wide range of other administrative data sources. NRS continue to improve these statistics through augmentation of existing and new administrative data sources, ensuring the focus is to deliver population estimates that are accessible and valuable for users.
NRS continue to work closely with partner organisations across the UK, including the ONS and NISRA, to ensure that UK population data and analysis is coherent, comparable and understandable for all users across the UK.
The Committee might also wish to note that the Office for Statistics Regulation (OSR) is currently assessing the Scottish Census, including how NRS are responding to the current situation and the methods and quality assurance they will put in place to provide the best quality data and statistics on the population of Scotland. The OSR has, in both its preliminary findings assessment report for Censuses in the UK, and in subsequent assessment reports for England, Wales and Northern Ireland, highlighted UK data and continues to engage with all three census offices in this regard. I am sure the Director General for Regulation, Ed Humpherson, will keep the Committee informed of its findings, which it plans to publish in November.
Yours sincerely,
Professor Sir Ian Diamond
Dear William,
I am writing to draw your attention to the latest Public Confidence in Official Statistics report (2021), which has been produced by the National Centre for Social Research (NatCen) on behalf of the UK Statistics Authority. I am happy to share that the report finds that public confidence in official statistics remains high, and engagement with official statistics has increased since 2018.
Awareness of the Office for National Statistics (ONS) and the Authority has increased from 70% and 33% in 2018 to 75% and 48% in 2021 respectively. Furthermore, for the first time people were asked if they were aware of the Office for Statistics Regulation, with 41% saying that they were.
Notably, 96% of people able to express a view agreed that it is important for there to be a body such as the Authority to speak out against the misuse of statistics, and 94% agreed about the importance of there being a body to ensure that official statistics are produced without political interference.
Members might also be interested to note that a very high proportion of respondents trusted the ONS (89% of those able to express a view) and our statistics (87%). Of those able to express an opinion, trust in the ONS was highest of all institutions asked about, including the Government, the Bank of England, and the civil service as a whole. 82% of people able to express an opinion agreed that official statistics are generally accurate, up from 78% in 2018. Meanwhile 44% said they had used ONS COVID-19 statistics; they were more commonly used than any of the other statistics asked about with the exception of the census.
This report is very welcome, especially following our hard work to provide clear insights throughout the pandemic. We are proud that the public support our vision of statistics that serve the public good, which we will continue to deliver with honesty, and free from political interference.
A copy of the report will be annexed to this letter for the Committee’s information.
Yours sincerely,
Professor Sir Ian Diamond
On Tuesday 29 March 2022 Sir Robert Chote, the Government’s preferred candidate for Chair of the UK Statistics Authority, gave evidence to the Public Administration and Constitutional Affairs Committee’s pre-appointment hearing for Chair of the UK Statistics Authority.
A transcript of which has been published on the UK Parliament website.
Dear Mr Wragg,
I write in response to the Committee’s call for evidence for the inquiry ‘Coronavirus Act
2020: two years on’; specifically, the transparency surrounding the use of data in decision making relating to the renewal of the Coronavirus Act 2020.
During the pandemic, the response of statistics producers to publish close to real-time data
was remarkable – for example, daily updates on COVID-19 cases and deaths. Producers
demonstrated the ability to consider user needs when balancing timeliness against quality
and strong analytical collaboration resulted in valuable, high-quality statistics.
As noted in our written evidence to your inquiry on data transparency and accountability:
COVID-19, throughout the pandemic the Office for Statistics Regulation (OSR) has called
for greater transparency of data related to COVID-19. In May 2020 we set out our
expectations on the use of management information by government and other official
bodies, and followed up in November 2020 and July 2021.
As you know, unfortunately we had to intervene on multiple occasions regarding instances
of data being quoted publicly that were not in the public domain, which had been used to
inform decision-making. These ranged from data quoted during press briefings, testing data, and data to inform quarantine policy around red-list countries. These were key decisions and announcements made with evidence that the public, media, and Parliament should have been able to scrutinise immediately.
However, two years on, we can see that progress has been made to support the quick
publication of new statistics and data which inform decision-making. It is encouraging that,
when we have raised concerns with producers recently, often during challenging
circumstances such as the emergence of new variants, these have been resolved more
quickly. We discussed the overall improvements in our October 2021 report: ‘improving
health and social care statistics: lessons learned’ where we noted how essential
transparency is for building public trust in statistics and retaining public confidence in
government decisions.
Transparency has been a big theme of our work. In October 2021 we published a blog
setting out our own priorities: build our evidence base of good examples of transparency,
continue to intervene when necessary, and work with external organisation and
governments to make the case for transparency. We are grateful for the Committee’s
support on these issues, and we will continue to keep you updated on this work.
I hope this is a useful summary for the Committee, and please do let me know if I can be of
further assistance.
Yours sincerely
Ed Humpherson
Director General for Regulation
Dear Mr Wragg,
As National Statistician and Chief Executive of the UK Statistics Authority (the Authority), I am
writing to inform the Public Administration and Constitutional Affairs Committee about the UK
Statistics (Amendment etc.) (EU Exit) Regulations 2021, that have today been laid before
Parliament.
The statutory instrument (SI) is a continuation of The UK Statistics (Amendment etc.) (EU
Exit) Regulations 2019 (2019 SI), that came into force on implementation period (IP)
completion day. The 2019 SI revoked 293 pieces of retained EU law on statistics. The aim of
the SI laid before Parliament today is to continue this approach by revoking the remaining
retained EU law on statistics, including new legislation that has come into force since the
2019 SI was laid before Parliament. The SI revokes 137 additional pieces of retained EU law
on statistics.
In revoking retained EU law requiring the onward provision of data to Eurostat, the SI
removes substantive obligations on the UK to collect statistical data at certain times and in
certain ways. While the production of statistics will continue to be subject to domestic
regulation as UK official statistics; the UK will have greater flexibility in the production of these
statistics than under EU law.
HM Paymaster General, the Rt Hon. Michael Ellis QC MP, made the SI in the Cabinet Office’s
capacity as the Authority’s sponsoring department. The Welsh Minister for Finance and Local
Government, the Scottish Deputy First Minister and the Northern Irish Minister of Finance
have confirmed the Welsh and Scottish Government’s and Northern Ireland Executive’s
consent to this instrument respectively.
Please do let me know if any questions or if you wish to discuss further.
Yours sincerely,
Professor Sir Ian Diamond
On 21 October 2021, Sir David Norgrove, Chair, UK Statistics Authority, Professor Sir Ian Diamond, National Statistician and Ed Humpherson, Director General for Regulation, gave evidence to the Public Administration and Constitutional Affairs Committee’s inquiry on the work of the UK Statistics Authority.
A transcript of which has been published on the UK Parliament Website.
