APCP(25)01 HPI Imputation

Advisory Panel on Consumer Prices Technical Panel 31 January 2025 presentation slides

Members in attendance

• Mr Stephen Burgess (ONS, Chair)
• Mr Grant Fitzner (ONS)
• Professor Paul Smith
• Professor Ian Crawford
• Dr Jens Mehrhoff
• Mr Peter Levell
• Dr Martin Weale
• Professor Bert Balk
• Mr Rupert de Vincent-Humphreys

Guest presenters

• Professor Huw Dixon
• Ms Monica George Michail

Secretariat

• Ms Aimee North (ONS)
• Ms Rifah Abdullah (ONS)

ONS attendees

• Mr Chris Payne (ONS)
• Mr Chris Jenkins (ONS)
• Mr Peter Gatabaki (ONS)
• Mr Liam Greenhough (ONS)

Apologies

• Professor Rebecca Killick
• Mr Mike Hardie (ONS)

1. Introduction and apologies

1. Mr Burgess opened the meeting and passed on apologies from members unable to attend.

2. Proposed method improvement: HPI monthly imputation

1. Ms North presented a paper outlining the known observation that the UK House Price Index (HPI) generally observes downwards revisions to provisional estimates of new build prices which drive downwards revisions in headline UK HPI statistics. Ms North outlined a potential methodology improvement proposed by ONS. Preliminary analysis indicated that proposed improvements to the monthly imputation for the floor area and number of rooms variables in the Great Britain (GB) HPI model would improve the accuracy of provisional estimates of new build prices. This would be expected to reduce the size of overall revisions (1^st estimate to 13^th estimate of a given month’s price) in the UK HPI.
2. The analysis focused on floor area and number of rooms variables, which have higher missingness rates than other variables, particularly for new builds. The analysis showed that missingness in floor area and number of rooms tends to decrease for new builds during the months following a property transaction as that property’s property attributes information becomes available following the sale. The analysis focused on England and Wales, where this effect is most notable.
3. Ms North proposed using median/mode or k-nearest neighbour approaches to impute missing values for at least the floor area and number of rooms variables, and informed the Panel that ONS intended to investigate the impact of imputing missing values in all variables. Ms North proposed that the basic monthly imputation currently used in the UK HPI be improved in 2025 following full impact assessment, and to undergo a more comprehensive imputation methodology review after the ongoing UK HPI replatforming project (replatforming from legacy systems into Python) has been completed.
4. Floor area data is not currently used in Scotland in the UK HPI model due to it being measured differently. A panel member suggested the ONS consider either to have one floor variable for England and Wales and another for Scotland in the regression model, or to run separate regression models for England and Wales, and for Scotland, which would permit exploring the potential use of floor area data for Scotland in the HPI model. The panel member also suggested using the natural logarithm for floor area. Ms North mentioned that this is something ONS is considering as a potential future UK HPI improvement to explore as part of a fuller methodology review, and was included in the April 2023 APCP-T paper. Panel members were in support of separating Scotland to have its own regression model in UK HPI.
5. A panel member queried the use of k=5 in the monthly imputation using K-nearest neighbour imputation. Ms North explained this was chosen arbitrarily for the preliminary investigation and expected k=10 to be used in the anticipated full impact analysis since this is what is currently used in HPI’s existing annual imputation (using CANCEIS K-nearest neighbour).
6. Another panel member suggested to investigate why missingness occurs and whether including property age in the model could be appropriate in the future.
7. A panel member asked if the ONS had engaged in the RPI protocol with Bank of England yet, since this proposed methodology improvement to the UK HPI would impact the data used to calculate the RPI. Mr Burgess confirmed that ONS will engage with the Bank of England to confirm any implications from the RPI protocol arising from improvements to the UK HPI monthly imputation approach.

3. Update on Seasonally adjusted CPI

1. Professor Dixon provided an update on the project and timeline as well as changes made in response to comments raised at the previous meeting. He then outlined the recommendations from the research for the ONS. The first would be to consider the time series in 3 sections, which would allow the CPI team to only update the last section, 2015 to present, going forward. Professor Dixon also suggested using the direct approach on the dataset. Their analysis found that the indirect approach leads to drift in the series however this can be mitigated by seasonally adjusting the data in segments.
2. Another recommendation was to run the model each month and revise the entire series each month, with revisions reported as minimal. Professor Dixon additionally provided recommendations on how the figure might be reported in the CPI bulletins going forward, such as creating a new section in the text. Ms Michail provided more information regarding the seasonal adjustment software and other technical details. She also recommends to regularly review the model to ensure any new seasonal series are appropriately adjusted.
3. Panel members were happy with the recommendations provided.
4. A panel member asked whether the Eat-Out-To-Help-Out scheme contributed to the prominent downward spike in some of the charts shown. Professor Dixon and Ms Michail explain that they would need to confer with the underlying data regarding the spike but that the Eat-Out-To-Help-Out scheme is an outlier and written about as a case study in the report.
5. Another panel member questioned using only statistical tests to identify seasonality, and highlighted that economic significance should also be considered when choosing series to seasonally adjust. They also queried what had been done on outliers and calendar effects, as well as if the seasonal adjustment code would be more widely available. Professor Dixon and Ms Michail were happy to share the code, with ONS’ approval. Professor Dixon further explained that the sensitivity of the statistical test could be changed based on what is appropriate and needed and that they focused on clearly identifiable outliers such as the April budget effects, the Eat-Out-To-Help-Out scheme and VAT changes.
6. Another panel member asked for clarification on the trading day effects and the use of a 5% significance level. Ms Michail explained that the effects were seen in expected series, confirming the model was working as they wanted. The panel member also asked whether the recommendation to apply direct adjustment was in line with ONS’s standard practice. ONS tends to use indirect adjustment when aggregating low-level series, but this is subject to judgement and user needs depending on the situation.

4. GEKS-T video

1. Mr Payne provided an overview on the communications the ONS has planned to inform users as they prepare to implement scanner data.
2. A panel member praised the clarity and accessibility of the video, however raised whether the video did enough to explain the GEKS-T method (its stated purpose). Mr. Payne acknowledged this but noted that there is a limit to how much content can be covered whilst keeping the video to a suitable length and states that the other articles ONS have prepared would provide further detail which would hopefully address the panel members concern.
3. It was agreed for the panel to provide further feedback by correspondence

5. Publication status of papers

1. The seasonal adjustment papers and presentation materials to be published once report is published.

6. Any other business and date of next meeting

1. Ms North acknowledged that the Panel had raised no objections via correspondence to publishing the previously-redacted rents development papers, so ONS would seek permission from the data owners to publish the previously-redacted APCP rents development documents.
2. Other actions were decided to be actioned via correspondence or at the next meeting.