Sir David Norgrove response to Matt Hancock regarding the Government’s COVID-19 testing data

Dear Secretary of State,

Thank you for your letter of 27 May, in which you described some welcome, though limited, additions to the official data on COVID-19 tests, including a proposed note on methods (not yet published at the time of writing). I am afraid though that the figures are still far from complete and comprehensible.

Statistics on testing perhaps serve two main purposes.

The first is to help us understand the epidemic, alongside the ONS survey, showing us how many people are infected, or not, and their relevant characteristics.

The second purpose is to help manage the test programme, to ensure there are enough tests, that they are carried out or sent where they are needed and that they are being used as effectively as possible. The data should tell the public how effectively the testing programme is being managed.

The way the data are analysed and presented currently gives them limited value for the first purpose. The aim seems to be to show the largest possible number of tests, even at the expense of understanding. It is also hard to believe the statistics work to support the testing programme itself. The statistics and analysis serve neither purpose well.

To mention just a few issues in relation to the data as currently presented:

the headline total of tests adds together tests carried out with tests posted out. This distinction is too often elided during the presentation at the daily press conference, where the relevant figure may misleadingly be described simply as the number of tests carried out. There are no data on how many of the tests posted out are in fact then successfully completed. The slides used in the daily press conference do not show the date when the tests were carried out;
the notes to the daily slides rightly say that some people may be tested more than once and it has been widely reported that swabs carried out simultaneously on a single patient are counted as multiple tests. But it is not clear from the published data how often that is the case. Figures for the overall number of people being tested have previously been published but are not available in the published time series;
the top summary presents the number of positive results from diagnostic tests (pillars 1 and 2) alongside the total number of tests across all pillars. This presentation gives an artificially low impression of the proportion of tests returning a positive diagnosis;
more generally the testing figures are presented in a way that is difficult to understand. Many of the key numbers make little sense without recourse to the technical notes which are themselves sometimes hard to follow. This includes the supporting spreadsheets, which, while welcome, make it difficult to extract even basic trends.

With regard to new data that are not currently made available:

test results should include for example key types of employment (e.g. medical staff, care staff), age, sex and location (by geography and place, such as care homes). How many people in what circumstances are infected? Where do they live?
for Test and Trace it is important that a statement of the key metrics to measure its success should be developed systematically, and published, to avoid the situation that has arisen in relation to the testing programme. The statistics will need to be capable of being related to the wider testing data and readily understood by the public, through for example population adjusted maps of hotspots.

I warmly welcome of course your support for the Code of Practice for Statistics. But the testing statistics still fall well short of its expectations. It is not surprising that given their inadequacy data on testing are so widely criticised and often mistrusted.

I also welcome the Department’s willingness to work with colleagues from the Office for Statistics Regulation (OSR) and I know they have been in touch to discuss how the data and their presentation could be improved and gaps addressed. OSR will be happy to help further in any way they can.

It would be useful to develop a published timetable for the changes that need to be made and for the development of the metrics for the vital new programme of Test and Trace.

I do understand the pressures that those concerned have faced and still face. But I am sure you would agree that good evidence, trusted by the public, is essential to success in containing the virus.

Yours sincerely,

Sir David Norgrove