- Machine learning algorithms identify patterns in data and make predictions for new, unseen data based on those patterns.
- Using a large amount of data doesn’t necessarily make an algorithm more precise, and not all correlations imply causation. Unwanted correlations can lead to biased algorithms that perform poorly on new data.
- Unwanted discriminations may happen without explicitly providing sensitive personal data, as other attributes can implicitly reveal this information.
- Human bias is a well-studied form of bias that can find its way into machine learning algorithms through biased training data.
- Selection bias, or bias in the process of collecting data, is another source of bias that can cause ML algorithms to learn and enforce bias. Because algorithms can be deployed at scale, even minimal systematic errors can lead to reinforced discrimination.
Kinds of Bias in machines
Bias in machine learning algorithms can lead to discrimination and poor performance on new data. Machine learning algorithms are designed to identify patterns in data, and their major strength is the ability to find and discriminate classes in training data, and use those insights to make predictions for new, unseen data.
However, in the era of “big data,” the assumption is that the more data is used, the more precise the algorithm and its predictions become. The problem with this assumption is that not all correlations imply causation, and no matter how large the dataset is, it still only remains a snapshot of reality.
Conclusion
Machine learning algorithms are strongly dependent on the data they use to create the predictive model. These training data may be biased, particularly in the form of human biases or selection biases.
Because the algorithms are prone to any such effects, and due to the potential of getting deployed at scale, even minimal systematic errors in the algorithms can lead to reinforced discrimination.
The people in charge should be aware of the importance of eliminating unwanted correlations and designing algorithms that do not discriminate and produce accurate predictions.
Impacting the unprivileged groups
In a training data set on claims of car insurance, red cars may have caused more accidents than cars of another color. The ML algorithm detects this correlation, but there is no scientific proof of causation between the color of a car and the risk of accidents.
If the algorithm is not designed to notice and eliminate this kind of unwanted correlation, it may be biased and result in poor predictions on new data.
There is a second, even more severe problem when the predictions impact people and the algorithm is biased to favor privileged groups over unprivileged groups, resulting in discrimination. This kind of discrimination can happen without explicitly providing sensitive personal data, as other attributes can implicitly reveal this information serving as a proxy.
For example, a car model can hint at the owner’s gender, or the zip code may correlate with a resident’s ethnicity or religion.
Sources of Bias
The first source of bias is human bias, which can be introduced when data labels include human judgment that may have been labeled with prejudice. The labels serve as ground truth, so any bias contained gets reproduced at scale in the model.
The second source of bias is selection bias, which can occur when the data do not reflect the real distribution. If a machine learning algorithm is using biased data for training, it will learn and enforce the bias.
