Dear Mr Wragg,
Thank you for your letter of 15 April, in relation to your Committee’s inquiry on COVID-19 vaccine certification. I have addressed the specific questions from your letter below.
What process does the Office for National Statistics (ONS) have in place to maintain high quality survey design when working at pace?
Our Methodology Division provides advice and guidance on all aspects of survey design, which has included many changes to the methodology of existing surveys to reflect the different operating conditions caused by the pandemic, as well as central support for the rapid establishment of new surveys and questions related to measurement of the pandemic itself.
Methodological considerations include data collection mode, questionnaire design, sample design and implementation, editing and imputation and analysis, and have included social surveys and business surveys as well as the Census. The teams in Methodology collaborate closely with output areas in the ONS and provide advice for other government departments through the Methodology Advisory Service. The necessity to implement new surveys quickly, for example in measuring the pandemic, means we sometimes implement a sufficient version of a survey and improve aspects of design in future iterations. However, this also gives scope to adapt the design to meet changing needs.
For specific surveys, such as the Opinions and Lifestyle Survey (OPN) that was set up at the beginning of the pandemic, we use well established and tested processes to deliver weekly samples, questionnaires, data collection operations and analysis. This process is led by experienced researchers, methodologists and analysts across the ONS, and is all detailed publicly in the OPN Quality, Methods and Information document.
We also assured quality with the COVID-19 Infection Survey (CIS) and similar surveillance studies by collaborating with academic partners, such as the University of Oxford and University of Manchester, to ensure our methods were appropriate for the unique medical and scientific aspects of these surveys.
Furthermore, the GSS Best Practice and Impact team are located within the ONS and provide guidance for those across Government in how to maintain quality when designing surveys at pace. During the pandemic, we have adapted our way of questionnaire testing to maintain best practice methods while doing so remotely.
What approach does the ONS take to ensure quality questionnaire design?
Firstly, all our questionnaires are designed by specialised teams who have been trained in questionnaire design, development and testing. They align to best practice for questionnaire design to ensure quality, drawing upon that training and work experience.
Our data collection methodology team within Methodology Division specialise in all aspects of qualitative research. This includes giving expert advice on questionnaire design and development, focus groups and cognitive/user-experience testing of data-collection instruments. The team has provided rapid testing and development of new questions related to the pandemic for use across a range of official statistics surveys, and they have been diligent in working to improve questions over time.
For specific surveys, teams explore whether there were existing questions asked on other surveys, or within GSS harmonisation guidance, that could be utilised. If new content is required, we work with experts in questionnaire design to consider the optimum question to ask of respondents. This includes considerations around the placement of questions in the survey, and measurement error around respondent interpretation. All questions are also designed to use simple and accessible language and limited to the core questions required to minimise respondent burden.
We are also constantly seeking feedback from respondents, the interviewers and the users of the data to improve quality.
Does the ONS have a role to play in sharing best practice around survey design?
The ONS has played a leading role in the delivery of surveys, both before and during the COVID-19 pandemic. The GSS Harmonisation team are located within the ONS and share best practice in data collection on cross-cutting topics through Harmonised Standards, which are tools for improving the comparability and coherence of statistics. This includes guidance on collecting data about the impact of the coronavirus (COVID-19) pandemic. The Data Quality Hub in the ONS has also published guidance on applying a user-centred design approach to surveys.
During the pandemic, the ONS collaborated on new survey design for DHSC surveys through cross-government working.
Internationally, the ONS shares best practice as a leading National Statistical Institute (NSI) shaping the Global Statistical System by playing an active role in fora such as the UN, UNECE, and OECD. Some recent examples of this have seen the ONS taking a leading role in developing National Accounts systems and developing new methods of measuring poverty. We also collaborate with fellow NSIs and share our experience to innovate jointly, for example in the fields of AI and machine learning. The ONS actively partners with developing countries to improve the capability of their statistical systems, and these initiatives have ensured those countries response to the COVID-19 pandemic has been strengthened.
What work has been done to develop data around vaccine certification? Could you please also provide copies of any survey reports about the COVID vaccine or attitudes to COVID vaccine statistics, or if there have not yet reported, a list of the survey questions used.
Within the ONS, the CIS asks about type of vaccine, the number of doses and when the latest dose was received. The OPN collects weekly data looking at whether adults have received at least one dose of the COVID-19 vaccine, and if not, their likelihood of receiving vaccine. It also considers the barriers to vaccine, and reasons for not being vaccinated.
The latest CIS release on antibody and vaccination data estimated that in England, 7 in 10 of the adult population would have tested positive for antibodies against COVID-19 in the week ending 11 April 2021. We noted across all four countries of the UK there is a clear pattern between vaccination and testing positive for COVID-19 antibodies, although the detection of antibodies alone is not a precise measure of the immunity protection acquired from vaccination.
Moreover, ONS Infection Survey data has been used to show the impact of vaccines on infection. It found that COVID-19 infections fell significantly (by 65%) after a first dose of the Oxford-AstraZeneca or Pfizer-BioNTech vaccines. Two doses of the Pfizer-BioNTech vaccine offered similar levels of protection against COVID-19 as previous COVID-19 infection. Too few people have yet received two doses of Oxford-AstraZeneca to assess this.
Of specific cohorts, the COVID High Risk Group Insights Study reports survey-based data on vaccine uptake among clinically extremely vulnerable (CEV) people, and their behaviour following vaccination. Other studies of interest include an over 80s Vaccines Insights Study, which investigated whether being vaccinated against COVID-19 changed behaviour and adherence to guidance or attitudes to risk in the over 80s population. The Students Coronavirus Insights Survey reports likelihood of having the vaccine among students.
Regular data on vaccine sentiment using the OPN are published as part of our publication “Coronavirus and the social impacts on Great Britain”. We published two standalone releases in March and April looking at the characteristics of adults experiencing vaccine hesitancy. Further, we published analysis linking vaccination data from the National Immunisation Management System (NIMS) to the ONS Public Health Data Asset (PHDA), based on NHS number, to look at vaccination rates in people aged 70 years and over by socio-demographic characteristics. We are also planning a future qualitative study exploring attitudes to COVID-19 vaccines and vaccination roll-out among groups who refuse, are unsure about, or experience barriers to accessing the vaccine; this is due for publication on 7 May.
The OPN does not collect data on attitudes to vaccine certificates presently. We work with stakeholders to develop questionnaires over time, and keep the content of our surveys, particularly the OPN, under continual review to ensure they meet user needs. We would be happy to consider the feasibility of adding questions on vaccine certificates to the OPN if the Committee would like us to.
I hope this is helpful for the Committee’s inquiry and please do let me know if I can assist further.
Yours sincerely,