RMR is a preliminary data cleaning strategy implemented prior to any of the Census statistical adjustment processes.  While the two primary objectives of the Census program would, in principle, be met by a database containing accurate and discrete information about each HH or communal establishment (CE) and the individuals therein, collected Census data will inevitably contain multiple and/or duplicate responses associated with a particular HH, CE, and/or individual.  Multiple and duplicate responses can undermine both primary Census objectives, contributing to overcount and inconsistencies in the Census data. RMR is designed to help minimise the impact of these potential problems.

It is important to note here that the first and foremost objective for the RMR strategy is the resolution of multiple and duplicate responses received from a discrete enumeration address assigned a Unique Property Reference Number (UPRN) through the ONS address resolution strategy.  Consequently, RMR assumes that this has been assigned correctly. There is also a secondary objective to explore the possibility of extending RMR in 2021 to include a relatively local level of geography such as postcode or Lower Layer Super Output Area (LSOA).  The main point here is that the resolution of multiple responses or duplicates beyond these very local geographic boundaries is out of scope for the RMR strategy. Consequently, any incidental reference to ‘Census data’ from this point forward should be considered as always referring to data within these constraints.

Multiple responses are, of course, not always errors, sometimes occurring quite legitimately in the Census data associated with a discrete enumeration address. Typically, this would be associated with a design decision. For example, requests for additional household continuation (HC) questionnaires needed by larger HHs responding through paper format can lead to receiving two or more responses for that address. Also, to meet changing social norms, in 2021 individuals are being encouraged to complete and return an individual response (referred to as an iForm) should they choose not to disclose accurate but personal information on a primary HH questionnaire.  Consequently, in this case, the Census data could contain two or more responses from the same individual. While iForms are likely to contain different answers to some Census questions, fundamentally, this type of multiple response is a duplicate at the address in question.

Multiple responses and duplicates, however, can also emerge through unintentional (and perhaps unavoidable) error.  For instance, there may be errors in the underlying enumeration address frame leading to several questionnaires being sent to the same address that are subsequently completed and returned. Enumerators in the field may leave a paper questionnaire at an address that is completed and returned in addition to the householder completing an electronic questionnaire. In 2011 there was clear evidence that some individuals misunderstood how they should respond to the Census, entering their personal information more than once on a discrete HH questionnaire. All these examples manifest as duplicates in the Census data.  In contrast, receiving several responses from the same enumeration address could also be indicative of errors where the address frame has failed to recognise that there is more than one HH at the address.

Regardless of whether occurring through design or error, multiple responses and duplicates are problematic with respect to primary census objectives and can serve as contributing factors to a wide range of statistical errors in Census outputs.  For example, and amongst many other reasons, poor integration of HC questionnaires can lead to undercount of larger HHs.  Duplicate HH and individual responses can lead to overcount of the general population. Duplicate individual responses can also lead to potential overcount of larger HHs. Errors in the underlying address frame can lead to several different issues associated with both undercount and overcount.

The overarching role of RMR is to address these problems and minimise any associated risk to the quality of Census Outputs by resolving multiple responses and duplicates, not by simply removing them from the data, but by careful integration of the information provided through all associated responses.  As multiple responses and duplicates can also have a direct and negative impact on the performance and accuracy of statistical Matching, Estimation, Adjustment, and Edit & Imputation methodology, the RMR strategy for 2021 has been designed and developed in close conjunction with these methods through the RMR Working Group.  All in all, RMR is designed to support the production of accurate local authority level population estimates and a representative statistical Census database; the two primary Census objectives.

First implemented in the 2001 Census and again, successfully implemented in 2011, the RMR strategy has undergone an extensive review for 2021.  The primary aim of the review was to ensure that the 2021 RMR strategy was built on a comprehensive set of design principles that ensure a consistent approach and justifiable foundation in line with Census objectives.  Consequently, the review itself, and the decision-making processes leading to that design has been an iterative process where elements previously agreed were often, and necessarily, revisited a second and even a third time to incorporate emerging principles. While considerably time-consuming, we believe that this had led to a far more robust, accurate and consistent end to end design for the 2021 RMR strategy tied more tightly to the statistical and analytical aims of the Census program than RMR has in the past.

In this Section we present an overview of the principles that have emerged to underpin the basic design of the 2021 RMR strategy. Where appropriate we indicate how the RMR strategy has changed since 2011, how we have incorporated lessons learned from 2011, and how we have linked the 2021 strategy closely to subsequent statistical adjustment methods and Census objectives.  The Section should also guide an understanding of the decision-making processes that led to the detail of how RMR functions, presented in Sections 5 and 6.

4.1 General principles

• Following what was generally a positive assessment of how the 2011 RMR strategy performed in the last Census it was agreed that the 2021 RMR strategy would build on that success. Consequently, the emerging design of the 2021 RMR strategy was guided by holding the effectiveness and proficiency of the 2011 RMR specification to account and, where appropriate, removing, adjusting, or adding functionality. Key drivers of the review included:
  • lessons learned in 2011 that identified potential improvements to the RMR process
  • changes to overarching Census design objectives such as the modernisation of the collection strategy
  • changes to the analytical aims of the Census question set to meet the demands of a modern society
  • advances in statistical methodology implemented both within RMR and the methods it aims to support
  • advances in the computational power available on the new ONS Cloudera processing platform.
• While the 2011 Census RMR strategy was quite successful in meeting its objectives, post Census evaluation indicated that there were areas of the design that could be improved. For example, the statistical matching methodology in the 2011 RMR strategy left at least some unexpected duplicates in the data.  While ultimately resolved prior to outputs this was something unexpected by both Statistical Matching and Estimation leading to delays and necessary revisions to approved methods. It is worth noting here that any RMR strategy will have limitations and it is important that these are well understood and addressed in aggregate adjustments to Census estimates through Statistical Estimation. The RMR Working Group was established to ensure that all stakeholders with a significant interest in the way RMR functions were included in the development of its design.  Table 1 outlines the related and relevant topic areas covered by members of the Working Group.

 Table 1. Membership & roles of the RMR Working Group

4.2 Overarching Design

• Building on the 2011 design the 2021 RMR strategy adopts a progressive and modular approach to the problem of resolving multiple and duplicate responses. Each module, or in some cases, set of modules, focuses on one aspect of the potential problem space.  For example, Module 1 looks to resolve multiple and duplicate CE’s; Modules 2 & 3 at HHs; Module 4 at Dummy forms; Module 5 at iForms, and so on. Section 5 provides a functional overview of all RMR Modules while Section 6 delves further into the detail.  Each module builds on the outcome of one or more of the previous modules.  Consequently, sequencing formed a significant part of the review and several adjustments have been made to the 2011 ordering, not only to optimise accuracy and efficiency but also to accommodate new Modules for 2021 such as the resolution of duplicate iForms (Module 5b).  We return to the sequencing of Modules in Section 5.
• In general, as in previous Censuses the resolution of multiple and duplicate responses relative to a topic area (i.e., in CEs, HHs, Dummy Forms, and so on) will be addressed through a 2-stage process:

1. 1. In Stage 1 multiple and duplicate responses need to be identified correctly. This is often achieved relatively easily through information provided on each types of Census questionnaire and or through data from the Census Field Work Management Tool (FWMT) such as the UPRN.  However, in cases where this information is not available, not suitable, or in the presence of uncertainty, RMR relies on statistical matching methods.  Typically, statistical matching/linkage methods are used to identify people who, for some reason, are represented more than once in the data.
   2. In Stage 2, where multiple and duplicate data sources have been identified, they need to be resolved. This is not about throwing data away but instead by careful integration of the alternative data sources. Typically, this will be driven by decision-making logic designed, for example, to decide where to put an iForm that could in principle, belong to two or more HHs at the UPRN. Alternatively, to decide which set of responses from a set of duplicate individual records will serve best as a baseline set from which to build a unique and discrete observation.  Resolution is usually achieved through a hierarchy of rule-based business rules followed by rules based more on statistical probability. This general strategy serves to prioritise options and manage resolution as we move progressively towards and into uncertainty.

4.3 Identifying duplicates through Statistical matching

• We mentioned earlier that during live Census processing in 2011 the RMR strategy left at least some unexpected duplicates in the data indicating that this should be one of the most significant areas to explore for 2021. Lessons learned from 2011 noted that the matching methodology implemented in the 2011 RMR strategy was likely to be one of the most salient reasons for this. For example, 2011 RMR relied quite heavily on Soundex based matching methods which we now understand will underperform with some cultural name sets.  However, one of the most interesting aspects of the review was that the performance of 2011 RMR matching strategy was evaluated by comparing results using the far more sophisticated Census to CCS and Census to Census Matching methodologies.  From this, one of the primary guiding principles for the design and development of the 2021 RMR strategy was that the matching methods implemented in the 2021 RMR strategy should be designed by the ONS Methodology Matching Team in conjunction with the development of the Census to CCS and Census to Census Matching methods. This not only ensures compatibility and consistency between methods but also ensures that the RMR matching methods meet the high-quality criteria one would expect from the ONS Methodology Division.

Currently, there are two ONS internal technical papers available on the work of the Methodology Matching Team: Plachta and Shipsey, 2019; Plachta, 2020.  Both papers can be made available on request.  Here we provide a brief overview:

• For the 2021 Census RMR strategy, a deterministic linkage method will be implemented to identify duplicate people within an enumeration address (UPRN) prior to resolution. The linkage method uses 17 match-keys developed using 2011 Census data to capture as many duplicate matches as possible with a very low tolerance of incorrect matches. Research into the optimal combinations of match-keys was conducted by testing a variety of strengths of match-key on 2011 data and conducting a clerical review to identify those that made false positive matches (records that were not in truth duplicates). The remaining keys are loose enough to let through true matches that contained errors.

The match-keys allow for exactly matching records, records where date of birth is missing or contains slight error, records with missing gender, and records containing a variety of name errors. Name matching is implemented using two string comparators; Levenshtein Edit Distance and Jaro-Winkler; to identify matching names with spelling, handwriting or scanning errors. The benefit of using two comparators is that although both are very high quality and find a high volume of correct matches, each can identify matches that the other misses, making the RMR match-keys more resilient compared to the Soundex phonetic comparator method used in 2011.

In 2021, the Methodology Linkage team will also run a probabilistic matching exercise alongside the deterministic RMR process to independently identify duplicate responses. This will use the Fellegi-Sunter algorithm using parameters calculated using 2011 data. This exercise is not designed to add further duplicates to RMR’s list of duplicates, but as a quality assurance process to run alongside. If the probabilistic algorithm and the deterministic methods return similar matches, we can be confident that the match-keys are working well, despite the differences between 2011 and 2021 data. Using probabilistic matching in RMR is out of scope due to the resources required firstly to conduct it to the required quality, and secondly to deal with any potential matching conflicts between deterministic and probabilistic methods.

In 2011, 237,200 records were identified as duplicates in RMR, although in subsequent matching stages many more duplicate records were found. The new match-keys for 2021 have able to improve this number to 288,468 2011 duplicates identified with a precision of over 99.99% when testing on 2011 data. Although we expect the 2021 Census data to differ from the 2011 data due in part to changes in collection methods, we are confident that the improvements we have made to the method will mean that it is flexible and robust enough to perform well with 2021 Census data, outperform the 2011 strategy, and provide the required consistency with other matching methods also used to support Statistical Estimation & Adjustment.

4.4 Resolving multiple and duplicate responses through rule-based and statistical decision-making

• As outlined earlier, the resolution of multiple or duplicate responses within each topic area (i.e., CEs, HHs, Dummy Forms, and so on) are addressed in RMR through a hierarchy of rule-based business rules followed by rules driven more by statistical probability to help manage uncertainty. Two overarching principles were agreed to guide the review of the 2011 and development of the 2021 RMR strategy. In all cases where a resolution process was implemented in the 2021 RMR strategy:
  • it should not inadvertently introduce bias into the Census data.
  • It should look to retain as much supplied information as possible.
• An initial review of the 2011 RMR decision-making strategies revealed that in many cases the business rules implemented to resolve some multiple or duplicate responses were no longer valid relative to the general Census design principles laid out in Section 4.1. For instance, business rules driven by a predominantly paper-based Collection strategy in 2011 represented a particularly salient point of revision considering the transition to a predominantly electronic questionnaire in 2021.  It was also noted that in many of the 2011 modules, the resolution of residual cases that remained unresolved at the end of a sequence of deterministic rules were still being addressed through propositional (micro) business rules where a stochastic approach would have been a more appropriate way to manage uncertainty.  Retention of these rules would clearly mean breaking the principle of not inadvertently introducing bias into the Census data.

To ensure this overarching principle was always adhered to, a second general principle was applied to the process of reviewing and rebuilding the decision-making hierarchy in each of 2021 RMR modules, and that was to always start from a position where the general and default resolution strategy would be to distribute multiples or duplicates randomly amongst all available options.  During discussion and review of each module this baseline strategy could only be preceded by deterministic business rules or by other strategies if the members of the working group could justify their inclusion and why they should take priority.  In truth, the decision-making logic of all 2021 RMR modules ended up as a sequence of deterministic business rules with the uncertainty associated with any residuals at the end of the decision-making sequence being managed by statistical resolution (see Table 2 for a comparison).  However, this principled approach ensured that each of the 2021 RMR modules were examined and developed in a structured and systematic way while maintaining the overarching aim of avoiding the introduction, or for that matter retention, of unintentional bias.

• In 2011, across the entire suite of RMR modules, deterministic business rules driven the overarching design of the 2011 Census, particularly the 2011 Census Collection strategy, were used quite frequently in the RMR decision-making hierarchy. While there was no reason to move away from a similar approach in 2021, all in all, significant changes to several aspects of the overarching Census design for 2021, including the Collection strategy, led to many revisions of business rules for 2021 RMR.  Details of all these changes can be found in Section 6. Here, as examples, we present just a few of the design-based business rules considered in the review that led to the most significant changes.  As we worked through each of the RMR modules the list of business rules was referred to and implementation amended where necessary.  Again, this principle ensured that we maintained a consistent approach to the review.  These changes ensured that the 2021 RMR strategy was consistent with the reasoning behind changes to the overarching Census design between 2011 to 2021.
• EQ first. One of the most significant changes in the design of the 2021 Census compared to 2011 was the shift in the Collection strategy from predominantly paper questionnaires (PQ) to electronic questionnaires (EQ).   Driven by this overarching design principle, in some of the 2011 RMR modules PQ responses were prioritised in favour of EQ responses.  For 2021, the change in this overarching design principle led to similar rules being reconfigured where appropriate to prioritise EQ responses over PQ.
• iForms first. While prioritising the information collected through individual forms is not new for 2021 more emphasis has been placed on encouraging and facilitating the use of iForms in the overarching Census design than in 2011. This shift is designed to meet the demand for more accurate statistics on changing society norms and allows individuals to provide personal information about themselves that they may not have disclosed on a standard HH form. This change was considered throughout the RMR review.
• Non-response as valid ‘prefer not to say’. In 2011 RMR often used completion rate to determine which of a set of multiple questionnaires returned by the same individual would be considered the baseline for integration. Typically, the most completed record is selected for this purpose before ongoing integration. However, as an extension of the demand for more accurate statistics on changing society norms, the Working Group were advised that it was a legal obligation to count missing data in an iForm associated with voluntary questions such as gender identity as a valid ‘prefer not to say’ response.  This was factored into all decision-making logic that fell into this category.
• Receipt date. In the 2011 RMR strategy, date of receipt was sometimes employed to prioritise one response over another when multiple responses were received from the same HH, CE, or individual. For 2021 several options were always considered including first receipted, last receipted, and receipted closest to Census day. The best option was selected based on the circumstance.  However, in the overarching Census Collection design there is more focus on the significance of iForms than there was in 2011. In addition to encouraging individuals to complete iForms for the reasons previously outlined in the ‘iForms first’ principle above, public facing Census Support will also guide people asking how to revise information they have already provided also towards competing an iForm. These design decisions render ‘last receipted’ a more predominant option amongst alternatives than in 2011.

• To close this Section, there are four more aspects of the 2021 RMR design considered during the review that we think are worth mentioning in this more generic overview. These represent relatively important extensions to the 2011 RMR design for 2021 that we are confident will lead to improved performance.

• Retention of information. We mentioned at the beginning of this Section that the retention of as much information as possible was one of the main aims when integrating multiple or duplicate records belonging to the same HH, CE or individual. For 2021 this will generally be implemented in the same way as 2011 in that once we have determined which record to retain as the baseline response through the decision-making strategies discussed so far, missing data in that record will be backfilled using information from the other duplicate records.  There was a considerable amount of discussion about optimising this process, particularly around trying to maintain consistency between responses within person and between people within a HH.  However, on review, the Working Group decided that trying to implement an editing strategy within RMR was out of scope and the resolution of inconsistencies would best be left to the following statistical Editing and Imputation methodology.

There was however, one area where we were able to improve upon the 2011 RMR design.  One of the inevitable consequences of resolving duplicates is that the original HH structure can be disrupted. For example, we could start with a 7-person HH, but persons 2 & 3 end up being duplicates and resolved into one.  This means that when restructuring HHs at the end of RMR person 4 becomes person 3 and so on.  The problem here is that with a paper questionnaire the relationship matrix is abbreviated. In 2011 we collected all the relationship between persons 1 to 6 but once the questionnaire reached person 7 it would only ask for the relationship to person 1. Consequently, by making person 6 person 5 after RMR all the relationships that would have been collected had there not been a duplicate are now missing.  This would have been left for Edit & Imputation to resolve.

The problem in 2021 is likely to be more significant as abbreviation of the relationship matrix starts at person 5 rather than person 6 in the paper questionnaire potentially increasing the amount of imputation required into the relationship matrix for this response mode.  However, the electronic questionnaire in 2021 has been designed to collect the entire relationship matrix regardless of the number of people in the HH.  While relatively complex, the 2021 RMR strategy has been coded to retain that information and adjust the relationship matrix accordingly when tidying up the HH structure, replacing what would have been imputed data with observed data.

• Making use of the ‘Tuning Phase’. In contrast to 2011, the 2021 Census response data will be streamed live into the processing pipelines daily from the day that the Census goes live during Collection. This represents an early opportunity to start exploring the data and adjusting method parameters and processes based on analyses of the actual 2021 Census data prior to committing to a final end-to-end run of the Census cleaning and adjustment methods once all the data has been collected. This was considered throughout the review of the RMR strategy and there were two areas in the decision-making logic space we felt that we could program in new strategies with adjustable parameters that not only took advantage of this ‘tuning phase’, but could potentially provide additional support to our aim of not inadvertently introducing bias into the Census data.

We have already mentioned that in 2011, RMR often used completion rate to determine which of a set of multiple questionnaires returned by the same individual would be considered the baseline for integration, and that typically, the most completed record is selected for this purpose before ongoing integration.  For 2021 we have extended this function in that rather than just being based on the basic sum of completed Census questions, it is now based on the weighted sum, with the weight being a parameter that can easily be adjusted through a configuration file read by the RMR program.  This allows evidence-based prioritisation of any Census variable or subset of variables following analyses of the 2021 Census data during the Census Collection.

We have also mentioned that in the resolution of residuals all RMR modules will converge on distributing multiples and duplicates randomly where there are no valid business rules or other strategies to suggest otherwise.  For 2021, we have programmed RMR to accept and insert a sequence of up to 10 user defined conditional propensity statements.  This provides the opportunity to fine tune the probability distribution of residuals if required or deemed appropriate rather than automatically fall back on a completely random allocation.  It wasn’t possible to implement this functionality through a configuration file as we have for for the weighted sum of valid observed data, but the code has been written in such a way that adding them during live processing would not be that difficult.

• Extending the search area. Following the 2011 Census, it was noted that duplicate responses were not always constrained to a unique enumeration address.  This is not unusual with Census data or other data sources but as duplicates of any kind contribute to overcount in statistical estimates, aggregate adjustments for this type of error were made through the 2011 statistical Estimation and Adjustment methodology.  It was noted, however, that some of these duplicates occurred within small area geographies such as postcode or LSOA.  Consequently, a recommendation was carried forward to 2021 to explore the possibility of extending the remit of RMR to identify and resolve these duplicates earlier on in the Census processing pipeline.

To address this recommendation research was conducted with 3 primary aims. First, to identify the primary source of the problem with a view to try and understand whether it was likely to occur again in 2021.  Second, to explore the potential problem of establishing an appropriate rule-based resolution strategy that could effectively resolve all between HH duplicates.  And third, to evaluate whether implementing a proportional rule-based resolution strategy as in RMR had any real value compared to a statistical adjustment.  It is important to note here that ONS Methodology were already developing statistical Matching and Estimation methodology to address this type of overcount based on Census to Census matching.

The results of the research indicated that these localised duplicates could primarily be attributed to errors in the 2011 Address Register, typically, where two or more questionnaires had been sent to the same address.  Following consultation with the ONS Address Register Team we were assured that the risk of this occurring again in 2021 was far lower than it was in the previous Census.  In addition, mapping out the potential problem space for a rule-base resolution strategy applied across the full gamut of possible duplicate combinations that could occur at the person level between 1 or more HHs, with up to 30 individuals within a HH, demonstrated that this would be extremely complex and time-consuming exercise.  It was also concluded that due to this complexity, a full rule-based resolution strategy was likely to increase the risk of introducing bias into the Census data rather than reducing it.

All in all, the Working Group concluded that fundamentally, a statistical approach through Estimation and Adjustment to what was likely to be a relatively low impact problem in 2021 was a far better strategy than extending RMR.  However, it was also agreed that the idea was not completely redundant. Implementing the statistical Matching strategy to identify duplicates within the desired area is a relatively easy extension to the RMR program.  Consequently, it was agreed that RMR would be extended to identify and flag these duplicates as this information could make a significant contribution to subsequent statistical Matching and Estimation strategies. It was also agreed that in the relatively simple case of wholly duplicate HHs a simple rule-based approach of retaining only one of the HHs would be appropriate and carry little or no risk.  While we still have to finalise the detail, this is likely to be based on the principle of retaining as much information as possible from the duplicate HHs.

• Using Admin or Alternative data. Making use of administrative and/or alternative data sources has been a consistent theme throughout the Census programme and this was considered throughout the RMR review. Specifically, following the 2011 Census it was noted that the 2011 RMR strategy may have struggled to resolve multiple responses where it was not clear whether a ‘dummy form’, representing a unique address with no valid Census return was a regular property or CE.  There were obvious candidate administrative sources that might help this issue such as the VoA data.  However, in this case it became relatively clear that reliable information could be pulled into the RMR process from the 2021 Field Work Management Tool (FWMT) to achieve that same thing, but without having to consider potential problems with administrative data such as its accuracy and how it may lag in time relative to Census day.  While the FWMT data is not administrative it is alternative data that was not used in 2011 so we expect far better resolution of dummy forms in 2021.

It is fair to note that during the review, we did not identify anywhere else within the RMR process where other administrative data sources would be particularly useful or improve the quality of the outcome.  Much of this was due to improvements in areas such as the Census Address Register, the FWMT, and the Census Collection Strategy itself, which, by definition, have reduced the propensity for erroneous duplicates to occur in the Census data in the first place.  That said, we have left open the possibility of linking the RMR process to the Census Intelligence Database (CID), containing a suite of pre-linked administrative data sources for use elsewhere cross the Census programme.

Table 2. General comparison of the 2011 & 2021 RMR matching & resolution strategies

 RMR Modularisation: Detailed Methods & Rationale