The case studies below serve as examples of organisations that are applying the Code pillars to their data, statistics and analysis (that are not official statistics), to enhance public confidence. They were written by the producer organisation and explain why and how the organisation voluntarily applied Trustworthiness, Quality and Value.
UCAS publishes a wide range of statistics and analysis reports about higher education, as well as providing a variety of data products and services, which can be accessed via the Data and analysis section of our website.
Why voluntarily adopt the Code of Practice for Statistics?
We publish more than two and a half million data points a year throughout the UK higher education admissions cycle. Being able to provide our users with the confidence conveyed by compliance with the Code is important. A commitment to Trustworthiness, Quality and Value offers our users this confidence, and gives us the opportunity for us to compare our processes, methods and outputs against the recognised standards that the Code requires of official statistics.
Formal voluntary adoption of the Code demonstrates this commitment to customers, stakeholders and the public, thereby strengthening our reputation in the production of analytical outputs that are high quality, useful for supporting decisions, and well respected.
What was involved?
UCAS used the Office for Statistics Regulation’s (OSR) four stage voluntary adoption model: Understand, Review, Consider, Publish, in deciding to voluntarily adopt the Code.
- Initial visit to UCAS from Ed Humpherson (Director General for Regulation, OSR) to promote voluntary adoption of the Code.
- A paper was submitted to the UCAS Executive Board, to recommend voluntary adoption of the Code, and detailing the benefits and main changes required.
- A Head of Profession for Statistics was nominated to lead on the project.
- A small working group was convened to explore feasibility and identify the key changes necessary. The group comprised Communications and Policy staff, as well as analytical teams responsible for publications and public reporting.
- Required changes were identified, to establish more formal processes around pre-release access and ensuring all analytical outputs are brought into the scope of the existing formal UCAS release process.
- Follow-up meetings were held with the working group to implement the outstanding changes ahead of launch, including developing stakeholder engagement and communications plans, a press release to accompany the launch, and internal communications to UCAS staff.
- A two-page statement of compliance (PDF, 0.12MB) was drafted and submitted with a follow-up paper to the UCAS Executive Board to secure final approval for voluntary adoption of the Code.
- Launch on 2 September 2019 with the publication of the Statement of Compliance on the UCAS website, together with an updated timetable for data releases.
What are the outcomes?
Voluntary adoption of the Code publicly demonstrates UCAS’s commitment to enabling users to access and understand our data more easily, and helps provide the organisational impetus to support these aims. We aspire to further our commitment to the Code by publishing more details on our analytical methods, and by continuing to innovate in visualising our data, to expand its accessibility.
Why does the Department for Work and Pensions (DWP) voluntarily apply the Code?
DWP doesn’t just publish official statistics. We also produce a wide range of “other” data and analysis that are important in designing and evaluating policy. Core activities in DWP involve using data to: make decisions, track policies, forecast spending, and much more.
Our data are collected, reused, and analysed by many other groups and organisations outside government – for example, our interactive tool Stat-Xplore enables users to delve into the data and produce bespoke analysis.
The mix of data we use, produce and publish provides a rich range of data that contribute to the evidence base available to individuals and decision makers – information that can ultimately affect each person in Great Britain. So, we want to ensure that any use of statistics in our communications, from a tweet to writing a press notice or creating a presentation, should align with the Code, to ensure messages are clear, measured and appropriate for the story.
What does voluntary application of the Code mean?
It means that we have publicly committed to demonstrating trustworthiness, reassuring users about the quality of our information, and ensuring that the information serves the intended purposes.
Confidence and trust in our data and statistics are fundamental in how the DWP Strategy is delivered:
- From understanding and managing the performance of the business and helping inform the direction of economic and commercial activities;
- Allowing the formulation of better public policy and the effective measurement of those policies, including anticipation of risks and pressures on the system;
- Enabling the public to hold to account all organisations that spend public money, and informing democratic debate and public understanding of our society.
Ensuring that our data and statistics reflect user needs, support better decisions and gain public trust is a shared responsibility for everyone working in DWP. It is essential that those of us who are actively engaged in this area understand our roles and responsibilities.
We have an aspiration to further embed voluntary application of the Code, with the aim of having a shared aspiration across all analytical areas in the Department around when and how we demonstrate our commitment to trustworthiness, quality and public value.
How do you do this – and is it easy?
We apply the four voluntary application stages that aim to help produce analytical outputs that are high quality, useful for supporting decisions, and well respected. The stages are Understand, Review, Consider and Publish. The commit stage (Publish) is publishing the statement about why users can be reassured that the product achieves Trustworthiness, Quality and Value (TQV).
In leading up to that decision, it is helpful to understand the three pillars (TQV), review the approach to producing and publishing statistics/analysis in relation to the pillars, and consider if there are ways of improving practice. By publishing this statement, we are being transparent.
Our key is to think through each of the pillars:
- How are we trustworthy? Should people trust what we provide?
- What is the quality of the data? Is what we are doing the best estimate? Is it misleading? Could people make the wrong decision?
- How do we provide value? Is it helpful? Can users make decisions using the information?
We have had conversations across all analytical areas to raise awareness. We shared guidance and given advice to analysts who have voluntarily adopted TQV. We also obtained senior backing for applying TQV in the department – our chief analyst said:
“I am really keen for us to be as ambitious as possible in applying the ‘Voluntary Compliance …’ approach to as many outputs as appropriate from the Analytical Community.”
Here are some examples of where we have applied the Code beyond official statistics
Benefit Expenditure tables: We have added a statement of voluntary compliance with the Code to our release of the expenditure tables showing historical and forecast benefit spend. Ministers see these tables well in advance of publication as they’re part of Budget and Spending Review negotiations – so we can’t achieve all the requirements for official statistics. Instead, our statement sets out why we think this is high quality analysis, why it is trustworthy and how it adds value to the debate.
Social Research Reports: Voluntary application has started to be used in DWP research reports, as agreed with the Head of Professional for Social Research, for example, the GSR Automatic Enrolment Evaluation Report (see page 116). The report brings together the latest evidence and new analysis to show what has happened to workplace pension membership and contributions since automatic enrolment began.
Management Information (MI) is used for operational delivery and the running of the business. We have now begun to apply the Code pillars in our preparation and release of the MI, for example ESA Underpayments and Support for Mortgage Interest MI.
What do you think are the benefits of applying TQV?
It gives you an opportunity to highlight your commitment to demonstrating trustworthiness, reassuring users about the quality of your information, and ensuring that the information serves the intended purposes.
It provides a quick win to improve users’ understanding of the data presented. It also demonstrates a common-sense approach to sharing numerical information and prompts you to review your own processes.
“I found the voluntary Code helpful. The headings (TQV) made sense and it was a useful structure, in fact it reassured us as the producers as much as anything. We also enlisted an analyst from another team to quality assure the peer review of our analysis, which again proved valuable”.
Ultimately, it builds confidence and trustworthiness.
The Scottish Fiscal Commission is responsible for producing independent forecasts of tax revenues, expenditure on social security and on-shore Gross Domestic Product (GDP) for Scotland. It is not considered an official statistics producer; however, it has made an active choice to apply the Code wherever possible.
It reviewed its approaches to producing and publishing analysis against the three pillars of Trustworthiness, Quality and Value (TQV) and considered how it was already meeting the principles within the Code and what could be improved upon.
The majority of Code principles were already being followed within the Commission, in line with its responsibilities as an independent fiscal institution. However, setting out its commitment to voluntarily applying the Code in a clear statement on its website, using the TQV framework, allowed it to demonstrate its independence. Furthermore, by being transparent about its methods and quality assurance, as well as the limitations and the judgements made within its forecasts, it has encouraged trust in the value of its outputs. Building this trust is essential to meet its users’ need for independent scrutiny of the Government’s budget and to inform debate. You can find out more about the Commission’s approach in this blog.
The Commission included a commitment to continue to develop its approaches in line with the best practice outlined in the Code. It has already taken additional steps through starting to pre-announce forthcoming publications up to a year in advance, ensuring all data are released in a reusable format, and by publishing its approach to corrections and revisions.