Dear Dr Gulhane,

Thank you for your letter of 3 July regarding publications about the impact of minimum unit pricing (MUP) for alcohol in Scotland.

You raised concerns about the communication of Public Health Scotland’s (PHS) evaluation of the MUP policy, which concluded in a final report published in June (that synthesized evidence from a number of studies). You also raised concerns about a Scottish Government press release welcoming that report and an earlier health impact study published in March by authors from PHS, the University of Glasgow and the University of Queensland. We have also looked at an ‘at a glance’ document produced by PHS.

Our remit covers the production and use of official statistics and does not extend to research or policy evaluation. As such, we have not conducted a full investigation of the content or methodology of the PHS reports. Instead, we have focused on how statistical evidence has been communicated and we consider that the findings in the final PHS report are communicated clearly and impartially.

Communication of the PHS evaluation report (published June 2023)

The original version of the Scottish Government press release stated that:

“In their final report of a series, researchers said that ‘robust, independent evaluation’ and the best available, wide-ranging evidence drawing on 40 independent research publications, showed that the MUP has been effective in its main goal of reducing alcohol harm with the reduction in deaths and hospital admissions specific to the timing of MUP implementation”.

This wording might suggest to many readers that most or all of the studies referred to examined the health impact of MUP. But the evaluation report explains that of the 40 papers included, only eight provided evidence on alcohol-related health outcomes. The remaining 32 examined other potential effects of the policy such as on alcohol consumption, social outcomes, compliance by retailers and product prices. Of the eight papers which studied health outcomes, one looked at deaths and hospitalisations and found a beneficial quantitative impact on these outcomes. Based on the other seven papers, the report concluded that there was “no consistent evidence that MUP impacted on other alcohol-related health outcomes such as ambulance callouts, emergency department attendances and prescribing of medication for alcohol dependence”.

Communication of the PHS/Glasgow/Queensland study (published March 2023)

The Scottish Government press release and the PHS ‘at a glance’ document both referred to the results of the PHS/Glasgow/Queensland study. However, information about the level of uncertainty associated with the reduction in hospitalisations and deaths was not included in either output, despite being emphasised in the study. For example, the figures are estimates based on statistical modelling and the reduction in hospital admissions was not found to be statistically significant.

Summarising technical data, especially for a public audience, is challenging. Press releases, factsheets, tweets and other communications require condensed information, but it still serves users best to include caveats about the uncertainty or limitations of statistical evidence. In this case, caveats did not carry through from the final PHS report to the press release and ‘at a glance’ document.

The Office for Statistics Regulation has discussed these issues, and its broader guidance on communicating uncertainty, with PHS and the Scottish Government. It is good to see that, as a result, PHS has updated its at-a-glance summary and the Scottish Government has updated its press release to ensure that the uncertainty around the estimates is more clearly communicated. I am also pleased to report that both have committed to improving the communication of uncertainty in future outputs.

 

Yours sincerely,

Sir Robert Chote
Chair

 

Related links

Letter from Sandesh Gulhane MSP to Sir Robert Chote – minimum unit pricing