Accountability
Though there are many complex moral and legal issues surrounding accountability, every person who is involved in the creation of a machine learning system has a degree of responsibility for considering the system’s ramifications, good or bad (and researchers will need to be aware of their organisation’s own different approaches and standards). It is not the responsibility of the machine or system to be considerate of these ideals. This is the responsibility of those who establish these systems, and they should be accepting of scrutiny and appropriate accountability.
Accountability is the willingness to accept responsibility for a process, and therefore be answerable to any query or issue that may arise as a result. Organisations should agree on appropriate mechanisms to ensure accountability in-line with accepted standards. The Office for Statistical Regulation have produced guidance on good practice in designing, developing, and using models, which may be useful in helping to think about these mechanisms.
Ensuring the appropriate use of machine learning, and machine learning models
When designing a research project it is imperative that the benefits and challenges of different methods are weighed up, and that the researchers are able to justify the method(s) that they have chosen to use. This should be clearly documented.
In the case of machine learning, it is also the collective responsibility of those who develop and train models, and those who deploy pre-existing models to ensure that the use case of a model is clear, and that models are not used beyond their intended use.
Training a machine learning model can take anywhere between hours and months, depending on the problem you are trying to solve, and the data being used. Using a pre-existing model with the same intended use as a basis for your project may save time and money. In addition, training machine learning models also has an environmental impact, which could be lessened by using pre-existing models. However, in order to ensure transparency and accountability, it is important to consider how the model being used was developed, and for what purpose. Utilising models beyond their intended use without care and caution may impact upon the accuracy and validity of your results. Using pre-trained models may be particularly problematic if the model lacks transparency, as it may be harder to identify the processes used to train the model, and existing biases.
Whilst developers cannot stop others from using their models, to remain accountable, anyone making and training models should try to be explicit in communicating the intended use of a model, so that models are not created that are then used by others in the wrong way. If you are considering using a pre-trained model, it is important to understand what the model was built to be used for, and ensure before it is implemented, that it is suitable for your own project.
Potential Mitigations
- If you play any role no matter how big or how small in the design or development of a machine learning project, it is your responsibility to understand what your role is, and the policies relevant to your role. You should be accepting of scrutiny – this could improve your research, help you with your decision-making process, and ensure a level of public trust.
- Ensure accountability by design, by having governance processes to ensure human oversight at the appropriate organisational level throughout design and implementation.
- It is vital that a model is audited at regular intervals, and that this is well documented. Sufficient time should be built into the project plan to allow for this. This will enable assurance that the machine learning technique fulfils the intended purpose without any unwanted consequences. Regular audit allows scope drift and risk to be identified, mitigated against, and documented as required by the relevant stakeholders.
- A continuous chain of human responsibility should be established through the full lifecycle of the machine learning model. All human involvement and oversight should be transparent, documenting all roles and actions taken to ensure human responsibility can be identified.