The potential for bias
There are several ways in which bias can be reflected within machine learning which could influence how algorithms are created, used, and interpreted by analysts. Again, whilst there may be some overlapping relevance to operational or other uses of machine learning, this guidance focuses on the implications of different ethical challenges for research and statistics. In many circumstances, bias in machine learning is a result of cognitive human error – problems are introduced by those who design or train the systems. For example, the training data may be incomplete, or unrepresentative of the correct population. Bias though does not always exist as a result of human error, for example, it may be introduced into data sets or models as a consequence of previous societal norms.
The potential biases associated with machine learning are not unique to machine learning methods (for example, sample bias is a typical ethical (and quality) consideration within traditional statistical methods), and mitigations used to limit bias in traditional statistical methods can certainly help when thinking about machine learning. That being said, issues of bias in machine learning may be presented in more complex ways (particularly when using large datasets) and can often be more prominent for machine learning application. Detecting bias within machine learning studies can be very difficult, and understanding and limiting the effects of bias once it is identified can be a complex process.
There are a number of different biases which can be found within machine learning projects. Four of the most common are outlined below.
Sample bias
This occurs when the data used to train a model is not representative of the correct population.
Algorithmic bias
This is the bias that occurs when the algorithm being used is faulty (for example there may be technical issues with the algorithm stopping it from working), or not being used in the way it was intended (and therefore not appropriate for the new application).
Prejudicial bias
This refers to a situation in which the model reflects existing prejudices within the training data. A model will apply the same stereotyping that exists in society, if it is fed prejudicial data, and this will influence its results. Similar to prejudicial bias is observer bias, which relates to the conscious and unconscious prejudices or judgements which a researcher may bring to their data. This may affect both the representativeness of the training data and therefore the results from the system, but also the interpretation of results or recommendations.
Exclusion bias
If a data point or set of features (such as location, geography etc.) is left out of the training data, exclusion bias may occur. This normally happens when a researcher takes a data point out of the training data thinking it is not consequential, but this can lead to inaccuracy in the system.
A common solution to make sure training data doesn’t perpetuate bias from sensitive attributes (such as gender, ethnicity, socio-economic status etc.) is to remove these features from the training data. However, this may not eradicate the bias completely, and can in some instances, worsen the problem, as correlated attributes still existent in the data may still reflect the bias you are trying to erase. For example, a name might be correlated to gender, or a postcode may be correlated to socio-economic status.
If you do decide to discard a sensitive attribute (or indeed, any part of your data), you should document this thoroughly, investigating and analysing the feature’s relevance to the data before deletion, and the relationship between this feature and your label(s). It may help to ask a colleague for a second opinion.
The Data Science Campus present methods for cleaning faulty data to improve reliability and accuracy in their research on “Estimating vehicle and pedestrian activity from town and city traffic cameras”, which provides an interesting example of machine learning being used for statistical purposes.
- Take time to reflect on possible biases that may find their way into the data you are using. Any potential for bias, no matter how big or small, and the impacts that may arise from this should be clearly communicated with all stakeholders and documented throughout the project. Reflexivity on the part of the researcher is key.
- It may be useful to discuss your project with colleagues who are not involved in the project as part of an independent review.
- Ensure that training datasets are as representative of the correct population as possible. This may help counteract sample and prejudicial bias.
- You cannot expect your model to learn what a cat looks like, if you only feed it images of dogs. It is important that all the cases that we might expect a model to be exposed to are examined and ensure that a well-balanced data set is used.
- Where possible, the algorithms and data sets should be tested and validated to mitigate any possible biases, and systems should be systematically monitored to ensure that biases do not occur as the systems continue to learn.
- Machine learning models will sometimes perform differently in the real world than they did when being trained, and so it is important that models are continuously modelled and audited so that any issues can be identified quickly. Documenting the decisions that are made at all stages can be beneficial in determining where, and how, bias could have been introduced.