UK Statistics Authority
- The Chair welcomed the members to the twenty-fourth meeting of the Research Accreditation Panel (RAP).
- Members approved the minutes of the meeting held on 6 December 2021.
- Sophie Gwillym updated the meeting with progress on actions from previous meetings. All actions were complete or otherwise in progress.
2. Processor Accreditation: Sub-Committee Plans
- Sophie Gwillym presented the Panel with a proposal for the establishment of a Processor Accreditation Sub-Committee. This follows an action from the December 2021 RAP meeting, where panel members requested the implementation of a strategy to better equip panel members to assess Processor Accreditation reports. The preferred course of action was the creation of the sub-committee.
- The role of the sub-committee will be to:
- Advise the RAP on initial accreditation and annual accreditation reviews of processors to ensure compliance with DEA accreditation standards. The sub-committee will provide an accreditation recommendation to the RAP based on the expert advice provided. The RAP will still be responsible for the final accreditation decision made relating to secure data processing environments as per the RAP’s Terms of Reference.
- Advise on the ongoing maturity of accreditation processes and conditions that the UK Statistics Authority uses to assess processing environments against to provide a recommendation on accreditation to the RAP.
- The Panel welcomed the establishment of this as a sub-committee of the RAP and agreed the proposed Terms of Reference for this group. The RAP agreed that, once established, the sub-committee could helpfully support the RAP in considering the following points:
- Alignment of the standards used by the UK Statistics Authority to assess suitability for processor accreditation under the DEA with those of other processor accreditation processes across the UK through sharing of good practice and learning;
- The development of training around processor accreditation processes and standards to improve and maintain good understanding of requirements among newly incoming members of the RAP and the wider research community; and,
- Potential revision of the processor accreditation annual review format, to understand whether reducing the number of yearly assessments needed to assess maintenance of accreditation standard across the course of an environment’s five-year accreditation period is feasible.
- Sophie Gwillym invited Panel members to register their interest or suggest candidates for the role of the Processor Accreditation Sub-Committee Chair or as members of the sub-committee.
ACTION: The Secretariat to establish the Research Accreditation Panel Processor Accreditation Sub-Committee and provide an update to the RAP on the progress of this at the next RAP meeting.
ACTION: Panel members to contact the Secretariat to register their interest or suggest candidates for the Processor Accreditation Sub-Committee.
3. Project Accreditation: Digital Economy Act Research Accreditation Assurance Function
- Lily O’Flynn and Grazia Ragone presented the Panel with an overview of plans to provide assurance on research projects that are accredited under the framework overseen by the RAP to strengthen the RAP’s oversight of the wider operations of the DEA.
- The purpose of the assurance function is to ensure that accredited research projects are operating within the scope of their accredited research project when data is accessed within accredited research environments. Assurance checks will relate to each of the areas that projects are required to comply with as per the Research Code of Practice and Accreditation Criteria. The assurance function will need to be developed further alongside accredited processing environments to ensure feasibility before implementation.
- The RAP was content with this approach and welcomed this initiative by UKSA. However, the RAP stressed that this is likely to be a resource intensive function for the UKSA which requires appropriate resourcing to enable robust governance.
- Emma Gordon suggests the UKRI’s work on assessing researcher integrity would be useful to help frame measures of the assurance function.
- The Panel agreed for a mix of risk-based sampling and random sampling for the selection of projects to be assurance checked and welcomed a focus on research integrity to be included in the process. Mindful of resource, the RAP suggested a staggered approach to implementation which includes incremental increases to the sampling frame, limiting the number of open-ended questions asked by researchers and ensuring accredited processors have the capacity to support UKSA in the provision of this assurance.
ACTION: The Secretariat to liaise with Emma Gordon regarding the UKRI’s researcher integrity framework.
ACTION: The Secretariat to work with accredited processing environments to refine the assurance framework ahead of implementation and ensure that accredited processors have the capacity to support the UKSA in providing this assurance.
ACTION: The Secretariat to feed the Panel’s comments into the refinement of the assurance framework ahead of its implementation, and report back to the RAP at a future meeting on progress refining and implementing this framework.
4. Discussion: RAP’s Involvement in Identifying Research Biases
- Andrew Garrett and Lily O’Flynn presented a discussion around the paper ‘Protecting against researcher bias in secondary data analysis: challenges and potential solutions’ by J. R. Baldwin, J. B. Pingault, T. Schoeler, H. M. Sallis & M. R. Munafo. The paper was circulated to Panel members by Andrew Garrett via correspondence and discusses how research biases can lead to questionable research practises in secondary data analysis.
- The presentation focused on potential sources of bias and how the suggestions in this paper to support the research community in identifying and minimising this bias can translate into feasible suggestions to improve UKSA’s and RAP’s practises as enabling secondary data analysis. These include:
- The creation of more space for exploratory analysis to take place under the DEA gateway through a review of the project application process, to facilitate exploratory access to data under certain conditions for those researchers that are unable to refine research project applications without prior knowledge of data quality;
- Enhanced assurance to the RAP that accredited processors are content that the data requested in research project applications is suitable for the research aims set out within the application to minimise the RAP’s concerns around the extent to which pre-registered analyses will be appropriate for data requested; and,
- Improved searchability of the UKSA’s public register to make it searchable by researcher name to support records of all data an individual has accessed under the DEA. This has the potential to support the research community in replicating, challenging and validating existing statistical information available, and minimises the need for the RAP to directly identify any personal biases an accredited researcher might have when applying for data access under the DEA.
- Overall, the presenters welcomed a discussion from Panel members around the wider role of the RAP on researchers’ bias, and the RAP’s role in ensuring project accreditation is undertaken in a way that balances the need to enable projects to go ahead at pace, against the want to ensure the highest possible methodological standards of research at the point of application and accreditation.
- Overall, the Panel agreed on the suggestions made during the presentation and supported the view that it is not the Panel’s place to identify and mitigate against researcher biases when reviewing projects for accreditation against the Research Code of Practice and Accreditation Criteria. It was agreed that the Research Code of Practice does not require methodological review to take place at the point of project accreditation.
- The RAP agreed that the Panel should reconsider the way in which research project methodology is currently assessed as part of the accreditation process, to as to ensure research has methodological rigour, but is enabled at pace.
- The Panel agreed that lessening its focus on methodological rigour enables the RAP to prioritise other areas of DEA oversight. The Panel agreed that the RAP could provide additional value by considering and advising on strategic decisions relating to use of the DEA across government, and aligning governance standards across the wider research landscape. The RAP agreed for the Secretariat to bring a paper on this to the June RAP meeting to facilitate a discussion that allows the RAP to agree a revised process for research accreditation and a renewed primary strategic focus for the Panel moving forward.
ACTION: The Secretariat to bring a paper to the June meeting which sets out revised options for project accreditation, particularly relating to methods.
ACTION: The Secretariat to report back to the next RAP meeting with a paper on a new proposed way forward for the Panel, which focuses on a revised project accreditation approach and renewed strategic focus of the RAP which provides additional value to the wider research community.
5. Any Other Business
- Sophie Gwillym informed the Panel that UKSA is restructuring the way in which processor accreditation is carried out in the immediate future. Processor accreditation assesses both capacity and security controls, however the UKSA is unable to undertake any further capability reviews in the short term due to limited resource. During this period, processor accreditation annual reviews will continue to go ahead with a sole focus on security controls. Given that the majority of the controls relate to security practices, the RAP was content for the review of capability controls to be paused in the short-term.
- The RAP agreed that the review of capability controls relating to processor accreditation is an important statutory function that requires adequate resourcing to ensure required reviews can be routinely carried out in coming years. The Secretariat agreed to report back to the RAP at a future meeting on progress in securing required funding to resume annual review of capability controls to validate processors’ accreditation under the DEA.
- The Chair noted the ‘for information’ reports provided. These included:
- A paper informing the RAP of a processor change that will allow the NISRA Research Support Unit (RSU) staff to access Digital Economy Act 2017 (DEA) data remotely.
- The usual reports of accreditation processes undertaken by the UKSA and overseen by the RAP.
- The next RAP meeting is on 7 June 2022.
ACTION: The Secretariat to report back to a future RAP meeting on progress in securing required funding for Digital Economy Act processor accreditation capability control reviews.