Scottish Parliament

Letter from Sir Robert Chote to Pam Duncan-Glancy MSP – Scottish attainment gap

Dear Ms Duncan-Glancy,

Thank you for your letter of 21 March outlining concerns around the First Minister of Scotland’s use of statistics about the poverty-related educational attainment gap. At First Minister’s Questions on 27 February, he said: “the overall poverty-related attainment gap [in Scotland] has reduced by 60 per cent since 2009-10”.

The Scottish Government monitors the attainment gap through a series of thirteen key measures as part of the National Improvement Framework (NIF). Taken together, these are the data by which the Government measures the poverty-related attainment gap and the impact of its education policies.

Scottish Government officials confirmed to us that the 60 per cent figure used by the First Minister was taken from statistics on follow-up leaver destinations. Chart 4 of the supplementary tables shows that the difference in the percentage of school leavers in a positive follow-up destination (nine months after leaving school) between the most deprived and the least deprived has reduced by 60 per cent since 2009-10.

These statistics are not one of the thirteen measures included within the NIF. The Framework does include a similar measure – initial positive destinations of school leavers – which also shows a reduction in the gap between the most and least deprived students since 2009/10. However, both of these statistics measure outcomes once students have left school, not their educational attainment whilst in school.

When making claims about a reduction in the ‘overall’ attainment gap, we would expect the Government to use the NIF to help evidence this, and that this assertion would reflect a reduction in more than one measure relating to attainment. The First Minister should have been clearer that he was referring to a specific set of statistics to evidence his claim, particularly as it was based on data which is not part of the NIF.

To maintain trust and confidence in their statements, Ministers should take care that when they claim progress of their policies it is by their own defined measures, or otherwise clearly explain the source of separate data so that it is readily accessible for policy experts and the public to understand. We have raised these matters with the First Minister’s office.

Yours sincerely,

Sir Robert Chote
Chair

Letter from Sir Robert Chote to Jeremy Balfour MSP – Health and Wellbeing Census Scotland

Dear Mr Balfour,

Thank you for your letter of 27 February 2025.

We are pleased that the Scottish Government has now published the Health and Wellbeing Census Scotland: Background Report. We consider that this document provides valuable information to users and demonstrates that the Scottish Government has reviewed its approach to developing questions for the Health and Wellbeing Census as the OSR requested in its letter of 27 July 2022.

As you set out, a thorough review of the legal and ethical status of the survey lies more so within the remit of the Information Commissioner’s Office (ICO) to decide on and undertake. From our discussions with ICO, we understand that they presently have no plans for a further review, beyond their engagement and action to date. It is also worth clarifying that because ICO could only act in relation to compliance with data protection law, some of the issues you have raised, such as the ethical use of opt-outs, are outside the regulatory remit of both ICO and OSR.

However, if they judge that a joint review is necessary, we are willing to help. I am grateful that your scrutiny of the Health and Wellbeing Census has encouraged closer working links between OSR and ICO.

The concerns that you and your constituents raised have clearly been justified by the lessons learned by all parties in this case. If the Scottish Government or other statistics producers decide to run another survey of this nature, we will work closely with ICO to ensure that it is done to the highest standards.

Yours sincerely,

Sir Robert Chote
Chair

Pam Duncan-Glancy MSP to Sir Robert Chote – Scottish attainment gap

Dear Sir Robert,

Scottish Government Attainment Gap Statistics

I am writing to seek the assistance of the UK Statistics Authority in establishing the accuracy of claims made by the First Minister of Scotland, John Swinney MSP, regarding the Scottish Government’s progress on closing Scotland’s poverty-related attainment gap. These claims appear to be at odds with the statistical evidence available and now risk misleading the Parliament and general public.

At First Minister’s Questions on 27 February 2025, Mr Swinney stated that “the overall poverty-related attainment gap has reduced by 60 per cent since 2009-10”. The statement has drawn attention from the public and the media and gives the impression that overall educational inequality has significantly improved — an assertion which has since been publicly challenged.

The Ferret Fact Service has investigated this claim and concluded it to be “mostly false”, noting that there is no agreed overall measure of the attainment gap in Scotland and that the First Minister’s claim is based solely on the measure of “positive destinations” — a metric which refers to the number of school leavers going on to further education, training, or commencing employment. This means the claim fails to account for progress, or lack thereof, in other key areas of educational attainment, such as National 5 and Higher exam results, achievement against Curriculum for Excellence levels, and performance against key literacy and numeracy benchmarks.

In many of these other measures, the gap between pupils from the most and least deprived backgrounds has either narrowed only modestly or remained largely unchanged. As such, I believe it to be highly misleading to describe a 60% reduction in one narrow metric as reflecting an “overall” closure of the attainment gap.

When statistics are used to justify policy decisions that directly affect the lives of children and young people, they must be presented with accuracy and honesty. Misrepresenting such data — intentionally or through omission — undermines public confidence, risks eroding trust in our public institutions, and undermines the serious work required to tackle long-standing inequality in our education system.

I would therefore welcome your views on:

Whether the First Minister’s claim that the “overall” attainment gap has reduced by 60% is a misleading use of official statistics; and
Whether the Scottish Government has sufficiently justified its use of this figure, especially given its decision to use a single outcome measure rather than a comprehensive overview of educational attainment.

I would also welcome any guidance you may wish to offer to ensure that statistics relating to sensitive and multi-dimensional issues, such as the poverty-related attainment gap — where public trust in government data is paramount — are not presented in a misleading way.

I look forward to your response.

Yours sincerely

Pam Duncan-Glancy MSP
Member of the Scottish Parliament for Glasgow Region (Scottish Labour Party)
Shadow Cabinet Secretary for Education

Letter from Sir Robert Chote to Mark Griffin MSP – figures on Scottish Planning Approvals

Dear Mr Griffin,

Thank you for your letter regarding the claim made by the Minister for Public Finance on 12 November 2024 that

“164,000 homes across Scotland have planning permission but are not yet built”.

As noted in your letter, the supporting data was subsequently published in response to your Parliamentary Questions, but this should not be used as a standard approach for releasing data. According to the principles of intelligent transparency, data should be available publicly at the same time that officials make statements based on it, with an explanation of context, sources, and limitations. The Office for Statistics Regulation (OSR) has raised this issue with the Scottish Government’s Chief Statistician, who has explained that he is currently raising the profile of intelligent transparency beyond the statistics profession within the Scottish Government to ensure that these principles are embedded across Government. The OSR will be working closely with the Chief Statistician to support him in these efforts.

The data for this claim are based on Housing Land Audits data which is sourced from local authorities. We agree that the quality of this data remains unclear based on the limited information included in the response to your Parliamentary Question. Future uses of this claim should clearly indicate that this is a high-level estimate.

We understand that there is a long-term plan to improve the data and statistics on housing in Scotland. The OSR will continue to engage with the Scottish Government as this development work progresses, to ensure that the standards set out in the Code of Practice for Statistics are adhered to.

Yours sincerely,

Sir Robert Chote
Chair

Jeremy Balfour MSP to Sir Robert Chote – Health and Wellbeing Census Scotland

Dear Sir Robert

Thank you for your response to my letter of 19 Dec 2024. My constituents are grateful that after 30 months, the Office for Statistics Regulation (OSR) is following up its original request to the Scottish government to publish a review around the legal and ethical governance arrangements relating to the Scottish Health and Wellbeing Census for school children.

In your letter, you clarify that the original requested review relates only to the questions for each age range, rather than a “review of the survey and data governance processes as a whole”. A more thorough review would, obviously, cover elements that are within the remit of the Information Commissioner’s Office (ICO).

My constituents appreciate you are only able to act within your remit. However, whether the questions for each age range are legal or ethical, depends on the entire context of the survey and the data governance processes. Prof Lindsay Paterson covered this in his ethics briefing on the census. Therefore, a review with the limited scope around questions and age range, could not properly establish the legal and ethical position.

Other factors affecting the legal and ethical position, include the extent to which legal and ethical failings in various areas were accidental or otherwise, and why the research implementation was continued without pausing, when valid concerns were raised by credible organisations and parents. For example, an article in The Times on 6 Dec 21, reported that a Scottish Government spokeswoman’s response to concerns was that it would be “irresponsible to withdraw a census which focusses on children and young people’s health and wellbeing”.

This project has been characterised by misleading statements from the beginning, for example, the same article in The Times, reports that the Scottish Government spokeswoman also said: “Health and wellbeing surveys like this one are not new and play a crucial role in ensuring children and young people have access to the help, advice and services they need”, when of course, no previous survey was directly comparable with this, and it was new for the Scottish Government to gather such data with identity numbers for storing indefinitely for continued cross linking, and attempting to do so with every single child in the country.

Therefore, my constituents are concerned that at this stage, if a siloed legal and ethics review is published that only focuses on the questions and ages they were asked, and it does not address the whole context of the survey and data governance in which the questions were asked, response to valid concerns and what has happened since, then this has the potential to further mislead the public.

To prevent this, and to be of any value, they believe the scope of this review must widen to encompass all concerns that have been identified, including the involvement of the ICO with local authorities in April 2022 and with the Scottish Government since the OSR’s original letter.

They believe the review must also address why the original internal ethics peer review carried out by the Scottish Government was not adequate, and why some advice offered by the Scottish Government’s internal legal and data protection staff appears not to have been followed.

Therefore, I will write to the ICO about this. Can I ask for your support in putting these concerns about the legality and ethics relating to the overall survey and data protection governance to the ICO, and for the need to consider this issue in its entirety, to ensure that the review published by the Scottish Government does not further mislead the public by being incomplete?

Yours sincerely

Jeremy Balfour MSP

Letter from Sir Robert Chote to Jeremy Balfour MSP – Health and Wellbeing Census Scotland

Dear Mr Balfour

Thank you for your letter raising your concerns about the Scottish Health and Wellbeing Census (HWBC).

Many of the issues about data governance and sharing procedures discussed by your constituents are out of remit for the Office for Statistics Regulation (OSR) and fall within the remit of the Information Commissioner’s Office (ICO). However, I have addressed your comments relating to the OSR’s letter to the Scottish Government, published in July 2022.

This letter set out that the Scottish Government needed to review the approach it took to question development including the “legal and ethical governance arrangements that are in place for asking questions of each age range”. Please note that the OSR did not request the Scottish Government to undertake a review of the survey and data governance processes as a whole.

It is regrettable that the outcomes of this review have not yet been made public and the OSR has now written to the Scottish Government to set the expectation that this is rectified within the next 30 days.

Yours sincerely,

Sir Robert Chote
Chair

Jeremy Balfour MSP to Ed Humpherson – Health and Wellbeing Census Scotland

Dear Sir/Madam

Please find attached a letter from two of my constituents who are writing in their capacity as Chair of [redacted] School Parent Council and an individual parent of a child at [redacted] School, both schools being located within the [redacted] Council school catchment area.

Following some serious concerns regarding the collection and use of Health & Wellbeing Census data captured in schools, they are writing to each of the public bodies you represent, asking for a response to their requests within one calendar month, as per the Information Commissioners Office policy of engagement.

I request that you respond directly to me and I will discuss your with replies with these constituents, as I have been following this case with great concern over recent months since the issues with the data collection and use came to light.

Yours sincerely

Jeremy Balfour MSP

Mark Griffin MSP to Ed Humpherson – figures on Scottish Planning Approvals

Dear Ed,

RE: Misleading figures on Scottish Planning Approvals

I am concerned that a figure that is being used by the Scottish Government, which claims that 164,000 homes across Scotland have planning permission but are not yet built is not based on accurate data and so should not be used as a verifiable statistic in the course of government business.

The Scottish Government has used this figure on a number of occasions including in a statement to the Scottish Parliament made by Minister for Public Finance Ivan McKee on 12 November 2024, and in its publication of the same date: “Planning and the housing emergency: delivery plan”. The number was picked up and widely reported in national press.

I submitted a series of questions to the Government exploring how this number had been calculated. From the Government’s responses, it was clear that it considered the number to be arrived at through aggregation of local authority housing land audits to create “a broad national picture of the scale of land available across Scotland” (Question S6W-31670). Further questions clearly indicated that the Government had little or no idea of how this number had been calculated. It also indicated in its response that it had little confidence that the land audits used gave an accurate estimate, going on to say that it intended to publish new guidance on housing land data audits to build a clearer picture of available land in Scotland.

Those figures have been questioned by various bodies, including Homes for Scotland. It had called for robust interrogation of the “stalled sites”.

Although the number is described as an estimate, it is clear to me that when a government puts a figure in the official record it implies official validation of the data as accurate. That is why it is important that, even when using estimates, those estimates should be built on robust data sources which have been properly interrogated to ensure the figure is as accurate as possible. It is of particular importance when that figure goes on to be used as the basis for development of government policy. The correct use of statistics and data is vital to ensure public confidence.

The public must have faith in the accuracy and truthfulness of statistics that are cited by Government ministers. As such, I would be grateful if you could investigate and provide guidance on the matter.

I look forward to hearing from you.

Yours sincerely

Mark Griffin MSP
Member of the Scottish Parliament for Central Scotland Region

Letter from Sir Robert Chote to Paul O’Kane MSP – Scottish Government child poverty statistics

Dear Mr O’Kane,

Thank you for your letter expressing concern about statements made by both the current and previous Scottish First Minister that Scottish National Party (SNP) policies are “lifting” an estimated 100,000 children out of poverty.

Under the principles of intelligent transparency, it is always advisable to think how an average person would interpret a high-profile quantitative claim of this sort so as to minimise the danger of them being misled and thereby trust in similar statements being undermined.

In this instance, the First Ministers were referring to a modelled estimate of the difference between the level of relative child poverty expected in 2024/25 and the level that we would have seen in the absence of a number of policy measures (as set out in footnote 4 of the updated Cumulative Impact Assessment). The Scottish Government set out the methodology behind this comparison in an annex to its Cumulative Impact Assessment in 2022 and the 100,000 estimate comes from the update published in 2024.

This kind of analysis is a reasonable way to estimate the impact of Scottish Government policies on child poverty, even though, just like alternative estimates, the calculations are bound to be uncertain and dependent to some degree on methodological choices. But the average person hearing such a statement might well assume that the First Ministers were claiming that child poverty is 100,000 lower than when the SNP took office. And, as you point out, the Scottish Government’s official statistics on Poverty and Income Inequality in Scotland conclude that the proportion of children in Scotland living in relative and absolute poverty remains broadly stable. Comparing the number of children living in relative poverty from 2004-07 (pre-SNP) to the latest datapoint of 2020-23 shows a decrease of 10,000 children (250,000 to 240,000). For children living in absolute poverty the decrease is 40,000 children (250,000 to 210,000).

Given this potential confusion, Ministers would be well advised from time to time to accompany this type of claim with a reminder of the methodology underpinning it so that they are not suspected of making an unduly flattering comparison.

Yours sincerely,
Sir Robert Chote
Chair

Paul O’Kane MSP to Sir Robert Chote – Scottish Government child poverty statistics

Dear Sir Robert

Re: Scottish Government Child Poverty Statistics

I am writing to seek the assistance of the UK Statistics Authority in establishing both the accuracy of statements made by the First Minister and former First Minister, and the methodology used by the Scottish Government, to claim that they have already, or they will lift 100,000 children out of poverty, as per the Child Poverty Cumulative Impact Assessment published by the Scottish Government on the 28th of February 2024.

On the 9th of May 2024, John Swinney claimed at First Minister’s Questions (FMQs) in the Scottish Parliament, that the SNP had ‘delivered measures such as the Scottish child payment, which is taking 100,000 children out of poverty today’. This was the second time Mr. Swinney had made this claim on that day alone.

The former First Minister, Humza Yousaf, has also made similar claims on a number of occasions, including in Parliament on the 2nd of May. Meanwhile, social media graphics and videos advertised by the SNP have promoted this claim as fact repeatedly.

According to the most recent statistics, however, child poverty rates have broadly remained static in Scotland, with approximately 240,000 Scottish children (24%) living in relative poverty. Indeed, single year statistic estimates actually suggest an increase in the number of children in poverty in 2022-23 to 260,000 children.

Given these statistics do not seem to reflect real progress on reducing child poverty, and do not correlate with claims made by SNP politicians as outlined above, Scottish Labour put in a Freedom of Information (FOI) request into the methodology and modelling used by the Scottish Government as evidence for their claims. I have attached the FOI response we received for ease of reference.

As you can see, the modelling used by the Scottish Government uses a counterfactual scenario that presumes a number of policies were not in place. Yet, the levels forecast for this counterfactual scenario are comparable to existing poverty levels, indicating that it cannot be an accurate reflection of any alternative scenario, and therefore could give an inaccurate and misleading account of the effect of Scottish Government policies.

The guidance also makes clear, contrary to some statements by SNP politicians, that this figure is an unconfirmed forecast to be reached 2024-25 and is not lifting children out of poverty but just keeping more Scottish children from going into poverty.

The guidance given to the Scottish Government explicitly states that ‘the estimated impact under this approach [the modelling used by the Scottish Government] does not equate to observable changes in poverty over time, or even… to the contribution of Scottish Government policies to those changes. For this reason, we refer to the impact of the policy package as ‘keeping’ children out of poverty rather than ‘lifting’ them out of poverty.’

You will note, as outlined in the instances mentioned above, that this advice seems to have been ignored repeatably by the First Minister and former First Minister in Parliament, and by the SNP on social media more widely.

Given both recent poverty statistics and official advice to Scottish Ministers appearing to contradict the claims of the First Minister about ‘lifting 100,000 children out of poverty’, I would welcome your view on both the language used by Scottish Government politicians, as well as the model used by the Scottish Government, which seems misleading given they compare the benefits of the policies they have implemented to an entirely unrealistic and abstract world.

I look forward to your response.

Yours sincerely,
Paul O’Kane MSP
West Scotland Region
Scottish Labour Shadow Cabinet Secretary for Social Justice and Social Security, and Equalities

Letter from Sir Robert Chote to Pam Duncan-Glancy MSP – Scottish attainment gap

Related links

Letter from Sir Robert Chote to Jeremy Balfour MSP – Health and Wellbeing Census Scotland

Related links

Pam Duncan-Glancy MSP to Sir Robert Chote – Scottish attainment gap

Scottish Government Attainment Gap Statistics

Related links

Letter from Sir Robert Chote to Mark Griffin MSP – figures on Scottish Planning Approvals

Related links

Jeremy Balfour MSP to Sir Robert Chote – Health and Wellbeing Census Scotland

Related links

Letter from Sir Robert Chote to Jeremy Balfour MSP – Health and Wellbeing Census Scotland

Related links

Jeremy Balfour MSP to Ed Humpherson – Health and Wellbeing Census Scotland

Related links

Mark Griffin MSP to Ed Humpherson – figures on Scottish Planning Approvals

RE: Misleading figures on Scottish Planning Approvals

Related links

Letter from Sir Robert Chote to Paul O’Kane MSP – Scottish Government child poverty statistics

Related links

Paul O’Kane MSP to Sir Robert Chote – Scottish Government child poverty statistics

Related links