Bias in Machine Learning and AI
Presenting new ethical challenges in business
AI and Machine Learning (ML) technology have become a major part of the armoury for many industries whether it be private companies, financial services, healthcare or governments.
These data driven tools are used to make ever more important decisions that can have far-reaching impact on individuals and societies both positive and negative.
As these solutions have evolved and become more widely used, new human rights issues have been brought to light as biases are uncovered within the decision-making systems we design. This also causes legal, ethical and brand reputation issues for the entity involved.
Beyond Analysis can help you assess your AI and Machine Learning solutions for bias and provide the solutions to remedy this.
What is bias and how does it creep into AI solutions?
Bias is a disproportionate weight in favor of or against an idea or thing, usually in a way that is closed-minded, prejudicial, or unfair.
In many cases, Artificial Intelligence (AI) or Machine Learning (ML) models be a force for good, working well to reduce humans’ subjective interpretation of data to help eliminate bias and treat everyone fairly. It can do this because AI and ML models learn to consider only the variables that improve their predictive accuracy, based on the training data used.
However when the wrong data is used to train models it can contain implicit racial, gender, or ideological biases. This is likely unintentional and usually occurs indirectly as a result of historical institutional bias that has gone unnoticed for years or simply through poor data governance practices.
So everything is fine the if training data is ok, but this is a big if. The reality is that many models end up being trained on data containing human decisions or on data that reflect second-order effects of societal or historical inequities. Although these causes are unintentional, the bias still occur, and the risks and results remain the same. There are a growing number of examples where this has happened.
Optimising models in today’s environment doesn’t just mean predictive performance, it is also essential to be in accordance with ethical & legal principles.
Why should we be doing something about it?
There are many obvious drivers to encourage individuals to uncover and rectify bias. Above all we have both a moral and legal obligation to treat others fairly without bias or discrimination.
On a professional or branding level, business need to consider bias for other important reasons also:
Customer Experience and Brand Trust: bias in AI systems can not only unwittingly erode trust between humans and machines that learn but also cause immense damage to brands unintentional or otherwise.
Regulatory Sanctions: regulators want to ensure that citizens are treated fairly by regulated commercial entities to ensure equality. For example, no individual or group is unfairly refused credit on the basis of their sex, race etc. The reach and the powers of regulators are increasing and their ability to impose sanctions and fines increasing.
Quality & Accuracy: the more we come to rely on models to make decisions, the greater the imperative to know how they work and confirm efficacy. Without a solid understanding of how a model works the potential to unwittingly increase business risk or erode margin increases.
Who should be concerned?
All sectors should be concerned about the risks of bias whether it be Health, Financial Services, Consumer Brands, Human Resources, or Judiciary, bias can become a challenging risk to overcome.
Models are being created everywhere but three areas we are paying close attention to include:
Financial Services models for mortgage rates and approvals, savings and loan rates, credit card fraud protection and virtual assistants providing automated advice
Insurance models across all product types be it car, home, life and health
Gaming and other closely regulated services where protecting consumers is a key responsibility.
The BA Solution
At Beyond Analysis we believe passionately in the power of data to do good. Ensuring our clients are fully aligned with how their models operate and perform and have the information to take the best ethical decisions is a critical part of our value to them.
Our Bias Solution supports businesses to create the required transparency of their models through independent validation. We look to address the two main challenges: ensuring the right stakeholders fully understand how the model works and fixing the underlying data.
The underlying data is often the source of the issue and so our solution interrogates all aspects of the data inputs for bias. We consider the following issues:
Training data contains human decisions and reflects second-order effects of societal or historical inequalities.
Poor sampling techniques
User generated data
Statistical correlations that are unacceptable or illegal.
Enabling non-technical employees to build a relative understanding of AI is important to allow them to understand how the unintentional bias occurs in their models.
Growing awareness and giving everyone accurate and measurable views on the relative importance and significance of features is a first step.
An additional layer of independently verified testing of the model function and outputs provides the validation required internally to mitigate internal/institutional bias.