Consumer views on trust and ethics
People are increasingly thinking about trust in terms of transparency and data privacy and indeed people say that they want control over their data.
A survey of 4,000 people in the USA by Rock Health in the Autumn of 2018 asked people who they would be willing to share health data with:
- My doctor - 72% willing to share health data
- My health insurer - 49%
- My pharmacy - 47%
- Research institution - 35%
- Pharmaceutical company - 20%
- Government organization - 12%
- Tech company - 11%
It is unsurprising that the list is topped by doctors given the strong professional framework that underpins their trusted position with patients and the public; something that is true to a lesser extent for pharmacists.
It is somewhat surprising that health insurers are second, but we can speculate that the people surveyed may feel that the health insurer is on their side because the health insurer does well when the customer is healthiest.
The fact that tech companies score badly is not surprising, but that they score worse than government is testament to the impact of recent scandals. Rock Health dug deeper into this category to see which companies were more likely to be trusted by consumers:
- Google - 60%
- Amazon - 55%
- Mircrosoft - 51%
- Apple - 49%
- Samsung - 46%
- Facebook - 40%
- IBM - 34%
Rock Health themselves express surprise at this result, having expected Apple to score highly given how much they promote privacy as a core value. Indeed, the top and bottom positions of the list seem to make no sense at all. Perhaps the best that might be said of this is that consumers do not see much differentiation between tech companies. This suggests that we, or any company, will have to work hard to truly demonstrate that we can be trusted with health data.
Ethical frameworks we have drawn on:
The IEEE Global Initiative on Ethics of Autonomous and Intelligent Systems
IEEE is an extensive effort aimed at developing standards specific to different uses of AI, such as robotics, mixed reality software (VR, AR, etc), and autonomous weapons systems. The entire effort is guided by a set of general principles:
- Human Rights - ensuring that AI does not infringe on internationally recognized human rights
- Prioritizing wellbeing - they define wellbeing as human satisfaction with life and the conditions of life, along with an appropriate balance between positive and negative affect.
- Accountability - designers and operators of AI are aware of what an AI is doing, why and be able to take responsibility for its actions
- Transparency - AI systems should be able to explain why they took an action, both to experts and lay individuals
- Awareness of potential misuse of technology - with a focus on education of developers, operators and users of AI
IEEE is also doing a lot of work to understand what values should be embedded in AI so that it can do good and be ethical. In doing so they inevitably struggle to come up with a set of universal values, instead highlighting that norms must be identified for a particular community. This embrace of moral relativism is to be constrained by universal human rights.
The European Commission’s High-Level Expert Group on AI
This group takes a rights-based approach, drawing on the UDHR and the EU Charter on Human Rights, with a particular focus on the following: respect for human dignity; freedom of the individual; respect for democracy, justice and the rule of law; equality and non-discrimination; and citizens’ rights. The group then sets out five principles for AI:
- Beneficence - "do good" - improve individual and collective wellbeing. This is not defined in great detail
- Non-maleficence - "do no harm" - reference is made to various human rights but a full definition of harm is not given
- Preserve human agency - humans must remain in control of their own actions and decisions and not be undermined by the AI
- Be fair - ensure that the development, operation and use of AI remains free from bias
- Operate transparently - be able to explain the operations of AI to people with varying degrees of knowledge. This principle also relates to transparency with respect to business models
Confusingly, having set out five principles, the EC group then lists ten separate requirements of Trustworthy AI: accountability; data governance; design for all; human oversight of AI; non-discrimination; respect for human autonomy; respect for privacy; robustness; safety; and transparency. It is not clear how these relate to the principles.
The Association of Computing Machinery
The ACM set out its Statement on Algorithmic Transparency and Accountability in January 2017. It sets out seven principles:
- Awareness - owners, designers, builders, user and other stakeholders of analytic systems should be aware of the possible biases involved in their design, implementation and use and the potential harm that biases can cause to individuals and society
- Access and redress - regulators should encourage the adoption of mechanisms that enable questioning and redress for individuals and groups that are adversely affected by algorithmically informed decisions
- Accountability - institutions should be held responsible for decisions made by algorithms that they use, even if it is not feasible to explain in detail how the algorithms produce their results
- Explanation - systems and institutions that use algorithmic decision-making are encouraged to produce explanations regarding both the procedures followed by the algorithm and the specific decisions that are made
- Data provenance - a description of the way in which the training data was collected should be maintained by the builders of the algorithms, accompanied by an exploration of the potential biases induced by the human or algorithmic data-gathering process. Public scrutiny of the data provides the maximum opportunity for corrections. However, concerns over privacy, protecting trade secrets, or revelation of analytics that might allow malicious actors to game the system can justify restricting access to qualified and authorized individuals
- Auditability - models, algorithms, data and decisions should be recorded so that they can be audited in cases where harm is suspected
- Validation and testing - institutions should use rigorous methods to validate their models and document those methods and results. In particular, they should routinely perform tests to assess and determine whether the model generates discriminatory harm. Institutions are encouraged to make the results of such tests public
The UK House of Lords Select Committee on AI has also proposed five principles:
- AI should be developed for the common good and benefit of humanity
- AI should operate on principles of intelligibility and fairness
- AI should not be used to diminish the data rights or privacy of individuals, families, or communities
- All citizens should have the right to be educated to enable them to flourish mentally, emotionally and economically alongside AI
- The autonomous power to hurt, destroy, or deceive human beings should never be vested in AI