Response from Sir Robert Chote to Sandesh Gulhane MSP – minimum unit pricing

Dear Dr Gulhane,

Thank you for your letter of 3 July regarding publications about the impact of minimum unit pricing (MUP) for alcohol in Scotland.

You raised concerns about the communication of Public Health Scotland’s (PHS) evaluation of the MUP policy, which concluded in a final report published in June (that synthesized evidence from a number of studies). You also raised concerns about a Scottish Government press release welcoming that report and an earlier health impact study published in March by authors from PHS, the University of Glasgow and the University of Queensland. We have also looked at an ‘at a glance’ document produced by PHS.

Our remit covers the production and use of official statistics and does not extend to research or policy evaluation. As such, we have not conducted a full investigation of the content or methodology of the PHS reports. Instead, we have focused on how statistical evidence has been communicated and we consider that the findings in the final PHS report are communicated clearly and impartially.

Communication of the PHS evaluation report (published June 2023)

The original version of the Scottish Government press release stated that:

“In their final report of a series, researchers said that ‘robust, independent evaluation’ and the best available, wide-ranging evidence drawing on 40 independent research publications, showed that the MUP has been effective in its main goal of reducing alcohol harm with the reduction in deaths and hospital admissions specific to the timing of MUP implementation”.

This wording might suggest to many readers that most or all of the studies referred to examined the health impact of MUP. But the evaluation report explains that of the 40 papers included, only eight provided evidence on alcohol-related health outcomes. The remaining 32 examined other potential effects of the policy such as on alcohol consumption, social outcomes, compliance by retailers and product prices. Of the eight papers which studied health outcomes, one looked at deaths and hospitalisations and found a beneficial quantitative impact on these outcomes. Based on the other seven papers, the report concluded that there was “no consistent evidence that MUP impacted on other alcohol-related health outcomes such as ambulance callouts, emergency department attendances and prescribing of medication for alcohol dependence”.

Communication of the PHS/Glasgow/Queensland study (published March 2023)

The Scottish Government press release and the PHS ‘at a glance’ document both referred to the results of the PHS/Glasgow/Queensland study. However, information about the level of uncertainty associated with the reduction in hospitalisations and deaths was not included in either output, despite being emphasised in the study. For example, the figures are estimates based on statistical modelling and the reduction in hospital admissions was not found to be statistically significant.

Summarising technical data, especially for a public audience, is challenging. Press releases, factsheets, tweets and other communications require condensed information, but it still serves users best to include caveats about the uncertainty or limitations of statistical evidence. In this case, caveats did not carry through from the final PHS report to the press release and ‘at a glance’ document.

The Office for Statistics Regulation has discussed these issues, and its broader guidance on communicating uncertainty, with PHS and the Scottish Government. It is good to see that, as a result, PHS has updated its at-a-glance summary and the Scottish Government has updated its press release to ensure that the uncertainty around the estimates is more clearly communicated. I am also pleased to report that both have committed to improving the communication of uncertainty in future outputs.


Yours sincerely,

Sir Robert Chote


Related links

Letter from Sandesh Gulhane MSP to Sir Robert Chote – minimum unit pricing

Letter from Sandesh Gulhane MSP to Sir Robert Chote – minimum unit pricing

Dear Sir Robert,

I am writing to request a review of the Public Health Scotland report ‘Evaluating the impact of minimum unit pricing for alcohol in Scotland: A synthesis of the evidence’ and the associated publicity and ministerial statements.

It purports to be “the final report from the PHS evaluation of minimum unit pricing for alcohol in Scotland”.

It is likely to be used in Scottish Government decision making on whether to continue with MUP and whether to raise the minimum unit price of alcohol.

However, I am concerned the report and associated publicity and ministerial statements significantly overstate the health impact of MUP, and under-represent the significant uncertainty in the wider body of research and among the scientific community.

I will outline my concerns in turn.

The press release

On 27 June 2023 the Scottish Government distributed the following press release.

It states the conclusion that MUP “has saved lives, reduced hospital admissions and had a positive impact on health” was drawn from “robust, independent evaluation’ and the best-available, wide-ranging evidence drawing on 40 independent research publications”.

It also states: “This follows a study published in March by PHS and University of Glasgow showing MUP reduced alcohol consumption by 3%, deaths directly caused by alcohol consumption by 13.4% and hospital admissions by 4.1%. compared to what would have happened if MUP had not been in place.”

However, this conclusion is not drawn from 40 publications. 32 of these publications referenced in the “final report” are silent on health impacts and focus on other issues such as consumption.

Of the eight publications that do address health impacts, seven of them are inconclusive.

Only one study concluded MUP had reduced deaths — the PHS and University of Glasgow study mentioned in the press release.

This study was led by Grant MA Wyper, public health adviser to PHS.

This “final report” does not “follow the PHS and University of Glasgow study”. It merely restates its findings.

Furthermore, it was not “independent”. It was commissioned by PHS and led by a PHS adviser.

The PHS/Glasgow University report was itself a retread of a report that appeared in The Lancet the previous day.

These reports are presented as two distinct studies in the latest PHS “final report” as Wyper et at (2023a) and Wyper et al (2023b).

This “final report” does not build on the Wyper study. The seven other studies addressing health impacts are inconclusive.

The conclusion that MUP has reduced deaths was not robust, nor drawn from wide-ranging independent evidence. It was drawn from a single PHS report.

Also, the report itself states that the 4.1 per cent reduction in hospitalisations was not statistically significant.

Therefore, the Scottish Government cannot definitively say “MUP reduced…hospital admissions by 4.1%”.

The Lancet study

The assertion that there were 13.4% fewer deaths “compared with what would have been observed in the absence of MUP legislation” overstates the uncertainty in statistical modelling.

No statistical model can say definitively what “would have” happened, as The Lancet study acknowledges in its methodology.

Indeed, “would have” becomes “might have” in the discussion section of the study.

“Study outcomes were assessed using a controlled interrupted time series study design, allowing us to determine the difference between outcomes we observed and our best representation of what might have happened under the counterfactual situation that MUP legislation was not enacted in Scotland.”

It concludes deaths rose faster in England in the absence of MUP therefore MUP “averted…an average of 156 deaths” each year in Scotland.

An additional 156 deaths a year would be a significant acceleration of the trend seen in the preceding 20 years, which was generally downward in the first decade and plateaued in the low thousands before the pandemic.

An acceleration this size was not witnessed in any of the regions of England with a similar population and demographic to Scotland, for example North West England, which saw a post-pandemic increase of a similar magnitude to Scotland despite the cheaper alcohol.

Criticisms of the Lancet study

Dr Adam Jacobs, Senior Director, Biostatistical Sciences at Premier Research, challenged the methodology behind the 13.4 per cent increase.

He said: “It is plausible that the MUP policy would bring down deaths and hospitalisations due to alcohol consumption, but I don’t think this paper shows it convincingly.”

Prof Kevin McConway, Emeritus Professor of Applied Statistics, The Open University, rightly took issue with the “causal interpretation” in The Lancet study.

He said: “This is an observational study, and no matter how well other factors are controlled for, it can never prove conclusively that the changes observed in deaths were due to the minimum unit pricing policy. In my view there hasn’t been enough caution given around assuming this relationship is causal…”

“We can’t say that MUP definitely led to a 13.4% reduction in deaths, though that does clearly remain an important possibility…”

“While it’s possible that the deaths or hospitalisations would have decreased enough to be detectable in the follow-up period here of 32 months after MUP, it’s also possible…that they aren’t clearly detectable on that time scale, though (if they really exist) the effect should show up, and indeed be much larger, later. And given what the time lag specifications look like in the Holmes paper, in another 7 or 8 years the reductions in deaths would be immense, implausibly immense indeed, given the size of the estimate after just over 2.5 years.

“Or it’s possible that what is being picked up in the new study is an effect of a change in alcohol consumption that occurred considerably earlier than MUP, so couldn’t have been caused directly by MUP…”

“So overall, in my view, there remains some doubt about whether MUP definitely caused the alcohol consumption change and therefore whether it is responsible for reductions in deaths.”

The “final report” at a glance

The at a glance conclusion states:

“Overall, the evidence supports that MUP has had a positive impact on health outcomes, including alcohol-related health inequalities.”

However, the finding for health above states:

“MUP reduced deaths directly caused by alcohol consumption by 13.4% and hospital admissions by 4.1%.

“Reductions were greatest for men and those living in the most deprived areas of Scotland.

“There is no consistent evidence of impact, positive or negative, on other health outcomes.”

This is not “overall” evidence. It’s a single study.

The conclusion should have stated:

“One study supports that MUP has had a positive impact on deaths and there is no consistent evidence of impact, positive or negative, on other health outcomes.”

The “final report” briefing

The briefing concludes:

“Taken together, the evidence supports that MUP has had a positive impact on health outcomes.”

Taken together, the evidence does not support this. A single questionable study estimated the reductions of deaths. The rest of the evidence was inconclusive.

The “final report”

Section 3.3 on page 33 confirms evidence relating to alcohol-related health outcomes was drawn from eight papers, not the 40 papers that the press release suggests.

It confirms the 13.4 per cent death reduction figure was drawn solely from the Wyper paper (Item 25 in the bibliography) and confirms the 4.1 per cent reduction in hospitalisations in Wyper was “non-significant”.

It cites Wyper at length but gives short shrift to the other inconclusive papers.

It states:

“The five other papers that contributed relevant quantitative evidence found no evidence of impacts in alcohol-related health outcomes, either positive or negative: there appears to have been no effect at a population level on alcohol-related ambulance callouts, (Manca 2022) prescriptions for treatment of alcohol dependence (Manca 2023) emergency department attendance (So 2021) or the level of alcohol dependence or self-reported health status in drinkers recruited through alcohol treatment services in Scotland, relative to England. (Holmes 2022).”

Ministerial statements

On June 27, Humza Yousaf, Scotland’s first minister tweeted:

“When @ScotGov proposed Minimum Unit Pricing over a decade ago, it was a pioneering approach to tackling alcohol harm and some had their doubts.

“Increasing evidence is now vindicating our approach. It’s saving over 150 lives a year.”

Analysis: There is no increasing evidence. There is one consistently rehashed and questionable PHS paper and about half a dozen inconclusive papers.

On June 27, The SNP tweeted MUP has led to “a major reduction in alcohol related deaths”

Analysis: Alcohol related deaths have risen since MUP was imposed. The SNP omitted the crucial caveat that the reduction was based on a hypothetical model.

Please investigate the matters raised in this correspondence and advise.


Yours faithfully,

Dr Sandesh Gulhane
MSP, Glasgow Region
Shadow Cabinet Secretary for Health and Social Care
Scottish Conservative and Unionist Party


Related links

Response from Sir Robert Chote to Sandesh Gulhane MSP – minimum unit pricing

Response from Humza Yousaf to Sir Robert Chote

Dear Sir Robert Chote,

Thank you for your letter of 1 November 2022 and the work of the Office for Statistics Regulation (OSR) to review and share lessons for health and social care statistics from the COVID-19 pandemic.

As noted in your report, the role of analysts in providing high quality evidence and timely insights has been critical during the pandemic to inform our understanding and decision making and I share your appreciation and thanks to all of our analysts. It was fantastic to see the efforts of the COVID and Flu National Vaccination Programme Team rewarded by winning the Top Team Award at this year’s Scottish Health Awards. This was very much a collaborative effort involving numerous colleagues including analysts within the Scottish Government, Public Health Scotland, the wider NHS and beyond.

Whilst we continue to closely monitor COVID-19, our analysts play a key role in providing evidence across the full range of health and care programmes, including supporting NHS recovery and the introduction of the National Care Service. The development of Scotland’s first Data Strategy for health and social care is progressing and this will set out the vision and ambitions for how we use data to deliver better services, greater innovation, and ensure the people of Scotland have greater access to, and greater control over, their health and social care information. We will of course ensure the Data Strategy reflects the important use of data to produce statistics and research to serve the public good.

The Scottish Government remains committed to learning lessons and we acknowledge the importance of delivering in the areas highlighted in your report – horizon scanning to understand users needs, making data and statistics available in an accessible, transparent and timely way, collaboration and communication – to provide high quality, trusted and valued statistics.

My officials from Health and Social Care Analysis (HSCA) division regularly meet with OSR’s Health and Social Care team and will keep them updated on our work and continual improvements across our statistics.

Yours sincerely,

Humza Yousaf


Related links

Sir Robert Chote to Humza Yousaf MSP – COVID-19 lessons learnt

Sir Robert Chote to Humza Yousaf MSP – COVID-19 lessons learnt

Dear Cabinet Secretary,

I am pleased to write to you commending the Office for Statistics Regulation’s report that was published today. 2022 update: lessons learned for health and social care statistics from the COVID-19 pandemic is an insightful analysis of the last year of the pandemic.

In its 2021 report, the Office for Statistics Regulation (OSR) highlighted the vital role that data and statistics played during the pandemic in supporting individuals to reach informed decisions. It has been possible to meet the increased analytical demands of the pandemic thanks to the extraordinary, dedicated efforts of the statisticians and analysts in your department. For that, I thank them all.

In its 2022 update, much like the UK’s health and care services, OSR has found a statistical system which is transitioning from pandemic crisis mode to living with COVID-19. While the intense analytical demands of the pandemic have eased, a real challenge is maintaining some level of analysis relating to COVID-19 while publishing new statistics to measure emerging issues, such as new disease outbreaks, and re-invigorating ‘business as usual’ statistics, such as those on NHS performance and the health and social care workforces. In the context of the increased demands on producers, we have identified four key areas which support the production of statistics which serve the public good:

  • Horizon scanning – understanding what information people need
  • Availability – ensuring data and statistics are made available in an accessible and transparent way
  • Collaboration – sharing data, plugging gaps and improving coherence and comparability
  • Communication – continuing to innovate and communicate statistics in a clear and engaging way to a wide range of users

We have seen many improvements to health and social care statistics over the last year. We have found a better understanding of the need for transparency of numbers used publicly by governments and are encouraged by new statistics published on topics of high public interest and ongoing efforts to fill existing data gaps. However, there is still progress to be made against several recommendations in our 2021 report, such as improving, where possible, the comparability of statistics across the UK and overcoming barriers to data sharing.

There are several developments relating to health and social care data in Scotland which we consider will support producers in their ability to provide timely information and analysis to support future decision-making. The Scottish Government is currently developing its Data Strategy for health and social care. Given that the pandemic demonstrated the importance of providing aggregated statistics to inform a wide range of people, we hope to see this purpose of health and social care data included in the final strategy. I look forward to the successful implementation of this data strategy, and trust that sufficient resource will be made available to turn new operational data into published statistics.

Yours sincerely

Sir Robert Chote


Related links

Sir Robert Chote to Steve Barclay MP – COVID-19 lessons learnt

Sir Robert Chote to Peter May – COVID-19 Lessons learnt

Sir Robert Chote to Eluned Morgan MS – COVID-19 Lessons learnt

Response from Humza Yousaf to Sir Robert Chote

Jackie Baillie MSP to Sir David Norgrove: Test and Protect Statistics

Dear Sir David,

Misleading statistics used by the Scottish Government for Test and Protect

I write to you regarding the above matter.

Following media reports this weekend, I am deeply concerned to learn that the Scottish Government has been publishing Test and Protect figures that significantly overestimate its performance in meeting the World Health Organisation’s (WHO) target, which states that “at least 80% of new cases” should “have their close contacts traced and in quarantine within 72 hours of case confirmation.”

The figures that have been published to date are misleading as they do not include “failed” cases. When the 72-hour success rate calculation is done with the failed cases added in, the 80% WHO target has actually been missed by Test and Protect over the period since records began (week ending Aug 9, 2020, to week ending September 12, 2021).

The correct use of statistics is vital, particularly when we are encouraging everyone to take regular tests to stop the spread of the virus. It is imperative that the public have faith in the accuracy and truthfulness of the data that is published by the Scottish Government. As such, I would be grateful if you could investigate this matter and take appropriate action as you see fit.

Yours sincerely,

Jackie Baillie MSP


Related links:

Response from Sir David Norgrove to Jackie Baillie MSP: Test and Protect Statistics

Ed Humpherson to Scott Heald: Test and Protect statistics


Response from Sir David Norgrove to Jackie Baillie MSP: Test and Protect Statistics


Dear Ms Baillie,


Thank you for your letter of 27 September regarding your concerns over the use of Test and Protect statistics by Scottish Government.

Scottish Government should be clearer about the limitations in comparing figures for Scotland (published by Public Health Scotland) with the World Health Organization target for contact tracing.

Ed Humpherson, Director General for Regulation, has today written to Public Health Scotland, copied to the Chief Statistician in Scottish Government, welcoming improvements to the presentation of these statistics. I enclose a copy of his letter.


Yours sincerely,
Sir David Norgrove


Related links:

Ed Humpherson to Scott Heald: Test and Protect statistics

Jackie Baillie MSP to Sir David Norgrove: Test and Protect Statistics



Response on estimates of inbound visitors

Dear Ms Pow, Ms Hyslop and Lord Elis-Thomas,

Thank you for your letter of 13 August in which you highlight concerns about the quality of estimates from the International Passenger Survey for inbound visitors. You pointed to the importance of these estimates in enabling you to monitor and evaluate existing policy and develop new strategies.

Clearly you must be able to rely on these estimates. The Office for Statistics Regulation has committed to bringing forward its planned review of Overseas Travel and Tourism statistics. The review will include consideration of the issues you have raised in your letter with a view to publishing findings by the end of September 2019. This review will build on the compliance check published in October 2018 which looked at the transparency and communication of International Passenger Survey methods changes, delays and quality information.

OSR will engage with your officials to ensure the review reflects the concerns you have identified.

I am copying this letter to Iain Bell, Deputy National Statistician for Population and Public Policy.

Yours sincerely,

Sir David Norgrove


Related Links:

Letter from Rebecca Pow MP, Fiona Hyslop MSP and Lord Elis-Thomas AM to Sir David Norgrove (August 2019)

Income tax reconciliations for Scotland

Dear Finance Secretary,

The Scottish Government recently asked the UK Statistics Authority to consider a tweet and press article by the then Chief Secretary to the Treasury on the subject of income tax reconciliations for Scotland,(1) in which Ms Truss described a “£941m shortfall in Scottish tax revenues due to lower growth in Scotland.”. You argued that this was inaccurate.(2)

The Authority has examined Ms Truss’s article and the HM Treasury press statement on which it relied.(3) The statements in the press release, subsequently repeated by the Chief Secretary, clearly link the whole of the reconciliation in tax receipts and block grant adjustment to slower than expected economic growth in Scotland. We agree with you that this is incorrect. The principal reasons for the block grant adjustment were in fact an initial overestimate of the Scottish tax base and faster growth of tax receipts than expected in the rest of the UK.

The Authority’s Deputy Director for Regulation has today written to the Treasury’s Head of Profession for Statistics. I enclose a copy of her letter, in which she urges the need to improve the presentation of the Scottish fiscal framework in line with our Code of Practice for Statistics.

I am copying this letter to the former Chief Secretary and to the Heads of Profession for Statistics at the Treasury and at HM Revenue and Customs.

Yours sincerely,

Sir David Norgrove



Related Links:

Mary Gregory to Tom Orford (August 2019)

Code of Practice Consultation

Sir David Norgrove to Derek Mackay, Cabinet Secretary for Finance and the Constitution on Code of Practice consultation.