Information needed | Response |
---|---|
Title and link to statistical output | “Developing an understanding of ‘start to success’ outcomes" |
Name of producer organisation | Office for Students (OfS) |
Name and contact details of person dealing with report |
Alison Brunt (alison.brunt@officeforstudents.org.uk, 0117 931 7166) |
Name and contact details of Head of Profession for Statistics or Lead Official |
Mark Gittoes (mark.gittoes@officeforstudents.org.uk, 0117 931 7052) |
Link to published statement about the breach (if relevant) |
|
Date of breach report | 16/11/2020 |
Information needed | Response |
---|---|
Relevant principle(s) and practice(s) | Trustworthiness: T3.4 - The circulation of statistics in their final form ahead of their publication should be restricted to eligible recipients, in line with the rules and principles on pre-release access set out in legislation for the UK and devolved administrations. No indication of the statistics should be made public and the statistics should not be given to any other party without prior permission for access. |
Date of occurrence of breach | 08/11/2020 |
Please give an account of what happened including roles of persons involved, dates and times:
The Office for Students (OfS) have recently introduced a new experimental statistic that looks at employment rates for students in higher education. This publication includes data on these rates at subject-level (i.e. based on the subject of the degree the student undertook), provider-level, and provider by subject level. The outputs have been published in anonymised form as experimental ‘ad hoc’ statistics.
Upon constructing these statistics, it had been the intention that the publication would include provider names alongside the corresponding provider-level and provider by subject level statistics. Because the publication intent involved information about the performance of individual higher education providers, as a public authority, the OfS considers that it has a duty of confidence to commercial entities such that providers are given an opportunity to make representations to the OfS before we make a final decision about whether to publish information which risks adversely impacting on their organisation. Representations were invited ahead of the publication of these new performance statistics, where providers were able to make representations in terms of both the accuracy of the proposed content and format of publication.
The statistics were shared on 26 October 2020 under strict conditions making clear to the provider the reasons for pre-publication sharing, and stating that the information must not be shared more widely.
In some cases, providers will take advice on methodology and principles with other organisations before making their representations to us. Providers are aware that they should not share any statistics or indicate direction of travel of the statistics to anyone else, and that they may only discuss methodology and principles.
On this occasion, one of the sector-representative organisations, with whom providers have consulted for the purposes of securing advice, has seen fit to submit their own response to the OfS about the proposed publication. In contacting us with their feedback, it came to light that one of the providers who had sought the advice of that organisation had shared one of the statistics with them (the employment rate for nursing) on 8 November 2020 prior to the deadline for representations and prior to publication. The identity of the provider who shared the statistic with the sector representative organisation in question has not been disclosed to the OfS.
Impact of the breach
Provide details of the impact of the breach both inside the producer body and externally:
Having explored the breach with the organisation, the OfS has learned that the statistic was shared within a broader commentary from a provider on a particular argument, and we are assured that the breach did not contain the fuller set of statistics in their proposed publication format. One of the purposes of publishing these rates as experimental statistics is to encourage an open debate among providers, students and other stakeholders about the method, thereby facilitating a transparent and collaborative approach to their further development. A research report published alongside the statistics intends to highlight the limitations and advantages of our proposed method, as well as next steps in its development.
Having decided to go ahead with publication, following the representations process the OfS implemented a number of refinements to the statistics (including the anonymisation of the providers within them) to address issues and further limitations that had been identified through the representations process. This means that the statistic that was shared was not one in its final form and so the impact of the breach is reduced.
Had this not been the case, it is possible that the affected organisation would have been in a privileged position to respond to the publication more quickly than other commentators. In light of the stated aims of this publication to encourage debate, whether this consequence would have impacted in a negative or positive way would depend on how this particular organisation has perceived the wider publication and research report. If it were negative, an early and detrimental commentary entering the public domain might have set an unhelpful initial tone for that debate.
Corrective actions (taken or planned) to prevent re-occurrence
Describe the short-term actions made to redress the situation and the longer-term changes to procedures:
As an immediate response, the OfS contacted the organisation involved to clarify the breach and why this should not have occurred, in order to raise awareness of the Code of Practice protocols and avoid future reoccurrence.
We will consider the need for further strengthening of the instructions given to providers about access to statistics in advance of publication, especially in relation to new or novel statistics on which we might reasonably expect a provider to seek advice on methodology and principles from other organisations.
Alerting providers to this breach as an example of a failure to adhere to the protocols, by referencing it within these instructions in future, may be helpful to improve understanding of activities that are and are not permitted.
The range of statistical outputs that the OfS produces can require a range of pre-publication approaches, and there are a limited set of circumstances in which we consider it is appropriate to share statistics in advance of their publication. We will continue to determine and adapt the prepublication approach on the merits of each case. In cases where the pre-publication approach involves sharing the statistics, it nonetheless remains possible – as in this case with these experimental statistics – that our publication intentions will evolve thereafter. We will remain open to adapting our course of action in response to any legitimate considerations identified between constructing and publishing statistics.