Ethical considerations related to the inclusivity of data for research and statistics

Methods and quality

It is important that we understand the limitations of our research and are transparent and reflexive in understanding how these limitations can be mitigated. Inclusive research methods in qualitative and quantitative research emphasise the important contributions that all people can make to research and take measures to ensure that data collection methods are as inclusive as possible, with careful consideration given to previously under-represented groups or minorities.

Inclusivity in research involves the way in which all voices are brought into conversations, as well as ensuring that these voices are heard. Therefore, being inclusive in all stages of your research is important to ensure that no one is forgotten in the statistics that we produce. Considering how your approach and research methods can be adapted to ensure that all groups are represented is necessary when first developing a project.

Respondent Centred Design can be applied to you research project to ensure that the needs of respondents are identified, and methods are designed in a way that meets those needs, being inclusive and ethical in the project’s design. This understanding of respondents seeks to produce appropriate methods, reduce respondent burden and build trust to encourage reliable responses, increased data quality and greater representation. The Respondent Centred Design Framework by the Government Statistical Service contains further details and outlines best practice, and this will be referred to throughout.

There are also several ethical considerations related to whether research methods have the potential to limit, harm or undermine these groups, which align with appropriate consideration of public good, confidentiality and data security mechanisms, and legal compliance.

The boxes below overview some key questions to think about when considering the potential inclusivity of your methodological approach, focusing on different stages of the research process:

Conceptualisation

Aspects to consider

In initial stakeholder consultations, are you considering the views of individuals from a diverse range of backgrounds?
Have you carried out preliminary research to consider how other work and research have approached inclusivity?

How can we address these issues?

Identify the groups that are relevant to your work at the earliest opportunity, particularly those who may not have previously been reached or where traditional methods of engagement have not been effective.
Consult with stakeholders from, or that represent, protected or under-represented groups to inform the development of accessible and inclusive participant involvement. Considering representation, for example, of the stakeholders you are reaching out to, or advisory boards, is important to ensure that a range of views are considered.
Learning from existing research that is recognised as good practice may be helpful when considering how you will ensure inclusivity in your project’s approach. There are links to a range of existing resources and projects in the ‘find out more’ section of this guidance.

Research design

Aspects to consider

Have sampling methods and approaches been carefully considered, in relation to representation and inclusivity?
How can you support diverse groups to engage with the project? What type of support might different groups need, and how can you help make involvement in data collection as easy as possible for them?
What may be the physical, social, institutional and attitudinal barriers to inclusion in data collection for different groups that you want to include in your project?
Are protected characteristics considered and acknowledged in your research design? Will participants feel supported and included in the content, aims and scope of your work?

How can we address these issues?

Take steps to improve sample sizes and maximise the inclusivity of your data, such as using targeted oversampling of under-represented groups. Effective sampling methods allow for diverse participants and perspectives to be included, which can reduce bias and increase the validity of results.
Secondary research may help you identify which groups are under-represented in data and research and how best to access them. What is the best way of engaging with them to ensure that they can participate? Do you adequately understand their differing needs so that you can ensure you are not inadvertently excluding people with certain requirements?
Consider different recruitment strategies for those groups that have not been previously engaged with traditional recruitment methods. Could you use a more purposive sampling method to ensure that different communities are well represented in your project? For example, if you are trying to recruit pregnant women, could you recruit via a local or online pregnancy support group, using a group gatekeeper to assist?
It is important to consult with representatives of your audience of interest to ensure that the content of any research materials is sufficiently clear, understandable, and does not make assumptions or reinforce implicit biases.
Have conversations about the implications of physical, social, institutional and attitudinal barriers to participation. Is it possible to involve stakeholders in co-creating solutions to reduce any potential barriers?
Clearly communicate the benefits of including under-represented groups in your work and ensure that your efforts do not appear to be tokenistic. This should include communicating any expected benefits to the individual or group as well as the wider intended public benefits of the project. What are the benefits of inclusion for those you are trying to reach? Is this clear and easy to understand from their perspective? Is it relevant to their needs? And most importantly, is it plausible that the stated benefits will be realised at an individual level? Being honest and open about potential benefits will help to ensure that trust is not damaged by over-promising or unrealistic expectations.

Data collection, methods, products, and infrastructure

Aspects to consider

Are the overall data collection methods that have been / will be used, including the supporting infrastructure, accessible and inclusive? How could this impact on potential inclusion in data sources? This includes considerations of:

- Any physical access needs
- The potential for exclusion depending on which mode is used (e.g., digital exclusion – are all participants able to access online materials?)
- The potential for exclusion related to stereotypes, misinformation, or negative experiences and perceptions of the research or data collection process by individuals.
- Potentially differential rates of consent among certain groups.
Does the content of your research materials support different lingual, sensory and cognitive needs? Are your materials understandable and appropriately tailored to the diverse needs of your audience?
Do your questions provide sufficient granularity of characteristics to enable useful disaggregation and interpretation of findings?

How can we address these issues?

Think about how you can be flexible in your data collection approach to enable both project teams and participants to contribute most effectively to the work. Applying the Respondent Centred Design Framework keeps those involved in the project at its core by understanding and responding to their needs.
Identify whether multiple data collection approaches may be beneficial or necessary to enable greater representation of diverse groups. For example, could you consider giving participants the option to participate in different physical locations (e.g., at home, in a community centre, at work) or online and over the phone? It is also important that methods are designed optimally for each mode and can be tailored to respondent needs, which the GSS call an ‘optimode’ approach.
Ensure that you make your research materials more inclusive so that they are suitable and meet accessibility standards. For example, have you ensured that someone with a visual impairment can use a survey and is not excluded? This could involve utilising resources from or consulting with organisations like the Digital Accessibility Centre or using Web Content Accessibility Guidelines. The Government Design Principles outline how to design accessible systems that are recommended for use across government.
Think carefully about the wording that you use in any research materials. Wording should be as clear as possible and using reading age checkers (e.g., the Hemingway App) can help content meet the UK reading age and encourage the use of less complex language. It is important to not design content based on assumptions and applying Respondent Centred Design can ensure that design is insight led and meets the needs of respondents to encourage greater quality responses.
The potential for micro-aggressions related to the language and terminology that you use should be considered. This includes avoiding the use of broad categories that do not support meaningful disaggregation and interpretation of data.
Think about the order of your response choices in any survey questions that you use, as these can reinforce implicit biases.
Give participants the opportunity to self-identify their characteristics and provide alternative options for people to contribute answers that you may not have identified or considered in pre-defined questions. Using open ended questions, or multi-selection checkboxes, and allowing space for participants to provide their own answers may be beneficial in case respondents do not feel that the pre-determined answers match the way that they identify themselves. Use the Government Statistical Service harmonisation standard for questions focused on protected characteristics wherever possible.
If possible, materials should be pilot tested with different individuals and redeveloped to ensure that you understand how different people may perceive or respond to your tasks and questions. The Respondent Centred Design Framework recommends combining cognitive and usability testing to create ‘cogability’ testing which can ensure that content is useable and meets respondent needs.

Data analysis

Aspects to consider

Are analysis techniques sufficient for the sample size and coverage of the data (i.e., is there sufficient representation across different groups)? Is the data of sufficient quality for the research questions that you are hoping to address?
Can the data be disaggregated to a reasonable and useful level in relation to protected or other characteristics that may be relevant?
Does your dataset contain data points that may be considered a proxy for protected characteristics (i.e., data points that strongly correlate with protected characteristics)? If so, how are you managing and analysing this data to ensure sufficient data protection, data quality and transparency in how this data will be used?
How much uncertainty is present in the data? Is this understood and documented appropriately in relation to its influence on the accuracy and applicability of findings to different groups?

How can we address these issues?

Consider and document how your sample size will influence the representation of different groups in your analysis and subsequent results. Understand the limitations of your dataset. For example, if your sample size is too small, it may be difficult to make any valid or reliable conclusions about different groups within the population and it may increase the risk of disclosure. Take further steps to improve sample sizes and maximise the inclusivity of your data if necessary, such as pooling multiple years of a dataset and exploring other opportunities to supplement data using existing datasets.
Seek advice from methodological colleagues to consider appropriate disaggregation and analysis techniques based on your available sample. Your sample should support an intersectional approach to analysis, rather than disaggregation based on a single characteristic.
Consider lessons learnt following the project regarding missing data or under-represented groups. What was done to minimise this and was it effective? If not, what could be done differently next time?

Dissemination of results

Aspects to consider

Have you considered how to appropriately report results to avoid the unfair labelling of particular groups?
Have the risks of disclosure issues from published outputs been considered and appropriately mitigated? This may be particularly important when considering potential impacts on different groups, and the wider social or historical context in which the project operates.

How can we address these issues?

Consult with Civil Society Organisations and representatives of your audience of interest to ensure that the dissemination of your results is socially and culturally sensitive and avoids stigmatisation.
Seek advice on appropriate statistical disclosure control methods from relevant colleagues for any published outputs. Consider the wider context of your work and any additional risks that this may present related to disclosure, such as the potential for harm on particular groups.
Where possible, share your methods, data and analysis openly, clearly and in an accessible format and document uncertainties and caveats. This allows others to learn from your project’s approach, easily access data and use it appropriately.

The report of the Inclusive Data Taskforce provides several examples of good practice related to inclusive data in research and statistics.

Using Administrative Data

If you are using administrative data, there are also particular aspects that should be considered from an ethics perspective, even though you are not directly collecting data. Indeed, using pre-collected data presents additional challenges from an inclusivity perspective, with a need to ensure that data sources are sufficiently representative.

Coverage

Are the data representative of individuals from minority groups? Particular groups may be less likely to access certain public services, either by choice (e.g., lack of trust), or constraint (e.g., accessibility issues). Could they be under-represented in the data and does this create biases?

Accessibility

Have data suppliers ensured that data collection methods are accessible and inclusive? Refer to the considerations outlined above to assess this.
Have data subjects been able to self-identify their characteristics and been given the opportunity to fully disclose all information, or has this been collected by others on their behalf?

Data Quality

Are harmonised standards used by data suppliers to measure protected characteristics? Have sufficient standards been developed, are they outdated, and have they been applied consistently over time?
Uncertainties and bias that may exist in the data should be identified and mitigated. For example, is there a low response on items relating to inclusion? May certain groups be hesitant to disclose certain information to particular organisations or in certain contexts?
Do the data provide sufficient granularity of characteristics to allow for appropriate disaggregation and robust results? Disaggregation of certain administrative data may lead to smaller sample sizes, which could present issues for maintaining the confidentiality of data subjects, accurately interpreting findings, and the analysis of intersecting inequalities.

Analysis

Are there specific risks associated with the research methods used for large datasets, like administrative data? For example, methods like machine learning and data mining may result in unexpected research findings where relationships are based on error, bias or discrimination, which risks perpetuating social biases and stereotypes. The UK Statistics Authority’s Centre for Applied Data Ethics has produced specific ethics guidance on the use of machine learning for research and statistics.

Efforts made to fully understand the data source, collection methods and subsequent findings can minimise and mitigate the risks associated with the use of administrative data. These may include:

Identifying and accessing guidance that may have been produced alongside administrative datasets to understand how inclusivity has been addressed in the collection methods and how procedures have been followed in practice.
Contacting data providers to further understand the context of potential data quality issues.
Clearly identifying and documenting uncertainties and bias that may exist in the data and outlining approaches taken to minimise the impact of these to increase representation within the data.
Triangulating data with other sources to assess accuracy, validity and the representation of certain groups and communities within the dataset, addressing elements of uncertainty.
Linking administrative data to other sources to fill identified gaps so relevant characteristics are reflected appropriately, ensuring coverage and enhancing the understanding of relevant groups.
Carefully considering outputs and the dissemination of research findings so that social biases are not perpetuated.
Considering the additional use of other research methods (e.g., qualitative approaches) to gain a further understanding of the perspectives and experiences of individuals who are under-represented in existing datasets.

Using data from children and young people

Ensuring that children’s voices are heard when collecting data that relates to them is important in ensuring that their views and experiences are appropriately captured, rather than relying on others responding on their behalf. The inclusion of children who are traditionally missing from data and therefore under-represented in research is particularly important, for example, Gypsy, Roma and Traveller children, disabled children, looked after children, migrant children, young carers and children experiencing neglect, as highlighted by the Inclusive Data Taskforce.

However, engaging with children and young people directly can bring additional ethical considerations that should be addressed.

In particular, it is important to consider what harm might look like in the context of your project for those that you are engaging with in order to identify potential risks and ways to minimise them. Engaging with civil society or other organisations that have experience working with the population groups that you are interested in will be beneficial, alongside carrying out a risk assessment and ethical review to ensure that the welfare and safety of children is at the centre of all project activities. Some areas to consider are:

Could the research topic be considered sensitive?
Does the research have the potential to impact groups of children differently, considering different contexts and cultures? This impact may be positive or negative.
Have you established a protocol to follow if concerns arise about a child’s welfare when collecting the data? This could include contacting the police, relevant child protection services or the NSPCC Helpline. The confidentiality of data subjects must be balanced with the need to disclose welfare concerns. More information on this can be found in the ONS Safeguarding Policy.
Do you have a follow-up procedure to support children and young people after the research has been carried out, for example, ensuring that they have access to appropriate advice and services?
Have you considered how the data collection process may harm researchers, for example, the impact of discussing sensitive or difficult topics, and ensured that they have access to support? Even if data collection is not face-to-face, it may still have the potential to impact researcher wellbeing, for example, analysing existing data sources.

Enabling children, their caregivers and gatekeepers to fully understand the purpose of the research project and what their involvement will look like can help to manage expectations and allow informed decisions to be made about participation. This may include using different methods, like pictorial or audio-visual materials, to support decision making by communicating what participation will look like in an accessible way. This may be particularly relevant if those that you are engaging with have low literacy levels. Including families and communities in meaningful, culturally sensitive discussions around participation may help to build trust and be transparent, maximising the chance for children’s voices to be heard. It may be helpful to seek advice from colleagues and legal teams when considering the participation of children and young people in the context of your project. Any data that you collect or hold on children and young people may be classed a highly sensitive and have specific data protection considerations. It is important to ensure that any information given to data subjects on the collection and use of their personal data is communicated in an appropriate way that considers their individual needs, to ensure their full understanding.

Using appropriate data collection methods can help to make the research process more inclusive and ensure that the data accurately reflects the views of children and young people. When designing research methods, the Respondent Centred Design Framework can be applied so that the needs and abilities of child participants are understood and met. For example, where additional needs have been identified, further support or modifications should be offered so children can participate fully in the research, reducing burden, encouraging responses and increasing data quality.

Using creative research methods with children and young people can help them to articulate their thoughts and feelings in ways other than verbal or written communication, for example, using art, movement and technology. In this example, data is collected from children’s involvement in a creative workshop which includes activities such as drawing, drama and film making.

Finally, there is a risk of power imbalances between researchers and child participants which may result in bias and less accurate results. Reducing this power imbalance can help children to feel comfortable in the research process and support the provision of more accurate data. There are several ways to mitigate this risk of power imbalance in your work:

Consulting children to gain an understanding of how they think they can be best supported to participate in the research
Consulting charities, advocacy groups and other organisations that work with children when planning the research
Designing child-led research and co-research methods to give children and young people more agency and control and to ensure that research addresses their interests (this article provides an example where children are involved as co-researchers)

When using existing data for research and statistical projects relating to children, like official statistics and administrative data, it is important to assess whether this gives a voice to children themselves. The Office for Statistics Regulation have identified three areas (their ‘3 Vs’) which can help to consider the representation of children in data sources:

Visibility– Whether children and young people are included in data collections and analyses relevant to them to enable informed decision making.
Vulnerability– Whether the experiences of children and young people who are vulnerable to poorer outcomes are collected and analysed separately.
Voice– Whether statistics reflect the views of children and young people and can be used by them.

For further information on understanding research and statistical projects involving children and young people, see the ‘find out more’ section within this guidance.

« Previous