Breach Report – Ofqual – Access arrangements for GCSE, AS and A Level: 2019 to 2020 academic year

Information neededResponse
Title and link to statistical outputAccess arrangements for GCSE, AS and A Level: 2019 to 2020 academic year
Name of producer organisationOfqual
Name and contact details of person
dealing with report
Ben Cuff – ben.cuff@ofqual.gov.uk
Name and contact details of Head of
Profession for Statistics or Lead
Official
Vikas Dhawan – vikas.dhawan@ofqual.gov.uk
Link to published statement about the
breach (if relevant)
Date of breach report16/12/2020

Information neededResponse
Relevant principle(s) and practice(s)Pre-release
Date of occurrence of breach16/12/2020

Impact of the breach

Provide details of the impact of the breach both inside the producer body and externally:

The pre-release was sent out this morning 9.30am. An approved pre-release recipient from the Department for Education (DfE) informed us in the afternoon that they had mistakenly sent a summary of main trends to an individual at DfE who was not on the pre-release list (along with a few others who were on the pre-release access list).

We were also informed that the individual who should not have received the summary have confirmed that they have deleted the email.

Because this person was notified and deleted the email soon after receiving it, and because only a summary of main trends was sent, there should be minimal negative impact of the breach.

Corrective actions (taken or planned) to prevent re-occurrence

Describe the short-term actions made to redress the situation and the longer-term changes to procedures:

The person who mistakenly sent the summary has been reminded of the need to maintain confidentiality as per the pre-release conditions in the future. We will also look to strengthen the text that we send out as part of the pre-release process.

 

Breach Report – Office for Students – Developing an understanding of ‘start to success’ outcomes

Information neededResponse
Title and link to statistical output“Developing an understanding of ‘start to success’ outcomes"
Name of producer organisationOffice for Students (OfS)
Name and contact details of person
dealing with report
Alison Brunt (alison.brunt@officeforstudents.org.uk, 0117 931 7166)
Name and contact details of Head of
Profession for Statistics or Lead
Official
Mark Gittoes
(mark.gittoes@officeforstudents.org.uk, 0117 931 7052)
Link to published statement about the
breach (if relevant)
Date of breach report16/11/2020

Information neededResponse
Relevant principle(s) and practice(s)Trustworthiness: T3.4 - The circulation of statistics in their final form ahead of their publication should
be restricted to eligible recipients, in line with the rules and principles on pre-release access set out in legislation for the UK and devolved
administrations. No indication of the statistics should be made public and the statistics should not
be given to any other party without prior permission for access.
Date of occurrence of breach08/11/2020

Please give an account of what happened including roles of persons involved, dates and times:

The Office for Students (OfS) have recently introduced a new experimental statistic that looks at employment rates for students in higher education. This publication includes data on these rates at subject-level (i.e. based on the subject of the degree the student undertook), provider-level, and provider by subject level. The outputs have been published in anonymised form as experimental ‘ad hoc’ statistics.

Upon constructing these statistics, it had been the intention that the publication would include provider names alongside the corresponding provider-level and provider by subject level statistics. Because the publication intent involved information about the performance of individual higher education providers, as a public authority, the OfS considers that it has a duty of confidence to commercial entities such that providers are given an opportunity to make representations to the OfS before we make a final decision about whether to publish information which risks adversely impacting on their organisation. Representations were invited ahead of the publication of these new performance statistics, where providers were able to make representations in terms of both the accuracy of the proposed content and format of publication.

The statistics were shared on 26 October 2020 under strict conditions making clear to the provider the reasons for pre-publication sharing, and stating that the information must not be shared more widely.

In some cases, providers will take advice on methodology and principles with other organisations before making their representations to us. Providers are aware that they should not share any statistics or indicate direction of travel of the statistics to anyone else, and that they may only discuss methodology and principles.

On this occasion, one of the sector-representative organisations, with whom providers have consulted for the purposes of securing advice, has seen fit to submit their own response to the OfS about the proposed publication. In contacting us with their feedback, it came to light that one of the providers who had sought the advice of that organisation had shared one of the statistics with them (the employment rate for nursing) on 8 November 2020 prior to the deadline for representations and prior to publication. The identity of the provider who shared the statistic with the sector representative organisation in question has not been disclosed to the OfS.

Impact of the breach

Provide details of the impact of the breach both inside the producer body and externally:

Having explored the breach with the organisation, the OfS has learned that the statistic was shared within a broader commentary from a provider on a particular argument, and we are assured that the breach did not contain the fuller set of statistics in their proposed publication format. One of the purposes of publishing these rates as experimental statistics is to encourage an open debate among providers, students and other stakeholders about the method, thereby facilitating a transparent and collaborative approach to their further development. A research report published alongside the statistics intends to highlight the limitations and advantages of our proposed method, as well as next steps in its development.

Having decided to go ahead with publication, following the representations process the OfS implemented a number of refinements to the statistics (including the anonymisation of the providers within them) to address issues and further limitations that had been identified through the representations process. This means that the statistic that was shared was not one in its final form and so the impact of the breach is reduced.

Had this not been the case, it is possible that the affected organisation would have been in a privileged position to respond to the publication more quickly than other commentators. In light of the stated aims of this publication to encourage debate, whether this consequence would have impacted in a negative or positive way would depend on how this particular organisation has perceived the wider publication and research report. If it were negative, an early and detrimental commentary entering the public domain might have set an unhelpful initial tone for that debate.

Corrective actions (taken or planned) to prevent re-occurrence

Describe the short-term actions made to redress the situation and the longer-term changes to procedures:

As an immediate response, the OfS contacted the organisation involved to clarify the breach and why this should not have occurred, in order to raise awareness of the Code of Practice protocols and avoid future reoccurrence.

We will consider the need for further strengthening of the instructions given to providers about access to statistics in advance of publication, especially in relation to new or novel statistics on which we might reasonably expect a provider to seek advice on methodology and principles from other organisations.

Alerting providers to this breach as an example of a failure to adhere to the protocols, by referencing it within these instructions in future, may be helpful to improve understanding of activities that are and are not permitted.

The range of statistical outputs that the OfS produces can require a range of pre-publication approaches, and there are a limited set of circumstances in which we consider it is appropriate to share statistics in advance of their publication. We will continue to determine and adapt the prepublication approach on the merits of each case. In cases where the pre-publication approach involves sharing the statistics, it nonetheless remains possible – as in this case with these experimental statistics – that our publication intentions will evolve thereafter. We will remain open to adapting our course of action in response to any legitimate considerations identified between constructing and publishing statistics.

Breach Report – National Records of Scotland (NRS) – Deaths involving coronavirus

Information neededResponse
Title and link to statistical outputDeaths involving coronavirus (COVID-19), news
release for week 40 (28th September to 4th October).
Link to publication
Name of producer organisationNational Records of Scotland (NRS)
Name and contact details of person
dealing with report
Alan Ferrier
alan.ferrier@nrscotland.gov.uk
Name and contact details of Head of
Profession for Statistics or Lead
Official
Peter Whitehouse
peter.whitehouse@nrscotland.gov.uk
Link to published statement about the
breach (if relevant)
Date of breach report12 November 2020

Information neededResponse
Relevant principle(s) and practice(s)Accidental or wrongful early release (Practices T3.3, T3.4)
Date of occurrence of breach7 October 2020

Please give an account of what happened including roles of persons involved, dates and times:

The statistical news release relating to the publication of the weekly ‘Deaths involving COVID-19′ data on
Wednesday 7 October 2020 was mistakenly live on the NRS website prior to the scheduled publication time.

NRS have been publishing the COVID-19 death statistics on a weekly basis from early April 2020.  In August we moved to a more comprehensive monthly publication, while still making the headline weekly
numbers available via the Scottish Government (SG) COVID-19 dashboard. However, the figures for this
week (week 40, 28 September to 4 October) showed an increase that we considered warranted providing
additional awareness to users.

A proportionate and quick solution was to do a statistical news release that briefly explained this week’s rise and highlight again to users where the data was available (SG dashboard). We had previously made users aware that the weekly data would be made available on the SG dashboard, Given the tight turnaround time, we did not pre-announce the fact that NRS would issue a statistical news release to supplement the publication of the data on the SG dashboard.

This news release was supposed to be published at 12 noon on 7 October 2020, however due to a
misunderstanding between the NRS webmasters office and our statistical web team it was published around 5pm on 6 October 2020.

NRS were alerted to this error around 10.40 am 7 October 2020, by a Scottish Government statistician who was part of the Pre-Release Access (PRA) list. The web team were notified and the news release was
removed from the site shortly before 10.50am. It was then published at the scheduled time of release at 12 noon. The SG dashboard was updated at the usual time of 12 noon.

NRS statisticians alerted all those on our PRA list immediately of this issue so that they were prepared for any potential early use of the unpublished figures, including communications colleagues perhaps having to deal with any unexpected requests or queries. We also advised our colleagues to continue to treat the statistics as unpublished and make no comment on them prior to the official scheduled publication at 12 noon.

Impact of the breach

Provide details of the impact of the breach both inside the producer body and externally:

NRS have investigated this issue, and although the web analytics show that 47 different users accessed the news release while it was online before its scheduled publication time, NRS believe that the impact has been very minimal and are not aware of any of the figures or commentary from the news release having been used in the public domain prior to release at 12 noon.

Following a review of news outlets and social media, there was no evidence of any early reporting of the new weekly total for COVID-19 deaths in Scotland. We also received no customer requests and no media
requests relating to these figures ahead of publication.

Corrective actions (taken or planned) to prevent re-occurrence

Describe the short-term actions made to redress the situation and the longer-term changes to procedures:

We have held discussions with the statistics web team and colleagues in the NRS webmasters team (IT) to review what happened, and have updated the web publication process guidance to ensure there is clarity on the process for issuing statistical news releases.

The decision to publish a statistical news release was taken at fairly short notice in relatively unusual
circumstances and this was a major factor in the miscommunication. We were looking to provide additional context and awareness to users, at a time when the key trend was changing after a sustained period of very low numbers.

The relevant colleagues in the NRS statistical and web teams have reviewed the web approvals process to
ensure clarity and awareness of the process for quality assurance checks made on the timings of publications and associated outputs (like statistical news releases).

NRS continue to monitor the COVID-19 death numbers closely and are putting plans together to be ready to reinstate more comprehensive weekly reporting in the near future. This would also mitigate against any potential for miscommunication over publication of outputs.