tech challenges

Is AI Turning The World To Stereotypes?

AI discrimination in banking

Less than 70 years from the day when the very term Artificial Intelligence came into existence, it’s become an integral part of the most demanding and fast-paced industries.

The rise of AI in the financial industry proves how quickly it’s changing the business landscape even in such traditionally conservative sectors. Despite evidences, we don’t realize how much Artificial Intelligence is involved in our day-to-day life. Especially in the financial sector. However, where there are opportunities, we are facing challenges too. Let’s see a few examples.

How AI helped Financial Sector

Artificial Intelligence provides a faster, more accurate assessment of a potential borrower which leads to a better-informed, data-backed decision. AI helps lenders distinguish between high default risk applicants and those who are credit-worthy but lack an extensive credit history. Moreover, algorithms analyze the history of risk cases and identify early signs of potential future issues.

Today Digital banks and loan-issuing apps use machine learning algorithms to use alternative data to evaluate loan eligibility and provide personalized options. Financial institutions use AI as a powerful ally when it comes to risk management. In the banking sector, Artificial Intelligence has been very successful in battling financial fraud or money laundering. Machines recognize suspicious activity and help to cut the costs of investigating the alleged money-laundering schemes.

Moreover, AI powers the smart chatbots that provide clients with comprehensive self-help solutions while reducing the call-centers’ workload. Today, a number of apps offer personalized financial advice and help individuals achieve their financial goals. These intelligent systems track income, essential recurring expenses, and spending habits and come up with an optimized plan and financial tips.

AI has a Lot to Work When it Comes to Discrimination

However, according to experts, the next generation of AI comes with a familiar bias problem. Bias occurs in many algorithms and AI systems — from sexist and racist search results to facial recognition systems that perform worse on Black faces. Generative AI systems are no different.

In an analysis of more than 5,000 AI images, Bloomberg found that images associated with higher-paying job titles featured people with lighter skin tones, and that results for most professional roles were male-dominated. AI has a lot to work on when it comes to discrimination. And the problem of amplifying existing biases can be even more severe when it comes to banking and financial services.

AI Processes are Inherently “objective” and “neutral”

Rumman Chowdhury, Twitter’s former head of machine learning ethics, transparency and accountability, said that lending is a prime example of how an AI system’s bias against marginalized communities can rear its head.

Algorithmic bias refers to the systematic and replicable errors in computer systems that lead to unequally and discrimination based on legally protected characteristics, such as race and gender. When assessments consistently overestimate or underestimate a particular group’s scores, they produce “predictive bias”. Unfortunately, these discriminatory results are often overlooked or disregarded due to the misconception that AI processes are inherently “objective” and “neutral”.

Has AI a Lack of Fairness?

When AI systems are specifically used for loan approval decisions, experts have found that there is a risk of replicating existing biases present in historical data used to train the algorithms. In the US, we have concrete examples of how loans have been denied to people from marginalized communities or minorities.

Apple and Goldman Sachs, for example, were accused of giving women lower limits for the Apple Card. But these claims were dismissed by the New York State Department of Financial Services after the regulator found no evidence of discrimination based on sex.

According to AI experts, the “personalization” dimension of AI integration can also be problematic. When AI is applied to banking it’s harder to identify the “culprit” in biases when everything is convoluted in the calculation.

“Today In the United States, credit unions and banks that deny consumers credit cards, car loans or mortgages without a reasonable explanation can be subject to fines due under the Fair Credit Reporting Act. It's a problem that some government agencies are trying to address, but there is no easy fix”, said Moutusi Sau, an analyst at Gartner.

The problem, according to Kim Smouter, director of the group European Network Against Racism, is that it can be challenging to substantiate whether AI-based discrimination has actually taken place.

What Next?

Our current financial system suffers not only from centuries of bias, but also from systems that are themselves not nearly as predictive as often claimed.

The spread of hi-tech solutions such as ML and AI offers tremendous opportunity to rectify substantial problems in the current system. Existing anti-discrimination frameworks are ill-suited to this opportunity. Refusing to hold new technology to a higher standard than the status quo results in an unstated deference to the already-biased current system.

As we said, Artificial Intelligence is increasingly used in the financial sector to screen loans and select financial product sales recommendations. AI used for such operations is generally based on historical data of financial institutions to build AI models. Although AI is used in many fields, not limited to finance, there have been several reported cases where AI has caused bias problems.

What to Expect in The Future From AI in the Financial Industry

All kinds of digital assistants and apps will continue to perfect themselves thanks to cognitive computing. This will make managing personal finances exponentially easier, since the smart machines will be able to plan and execute short- and long-term tasks, from paying bills to preparing tax filings.

This said, as of late 2018, only a third of companies have taken steps to implement Artificial Intelligence into their company processes. Many still err on the side of caution, fearing the time and expense such an undertaking will require –, and there will be challenges to implementing AI in financial services.

“Without the existence of common standards in the financial services industry, it becomes hard to measure what is treated as bias”, said Moutusi Sau. As the use of AI in financial services increases, it will become even more important to examine bias in the data. Regulators, in particular, are not going to be satisfied with the output of any algorithm if they cannot understand what underlies it.

Companies must use explainable AI to avoid making unfair and biased decisions about consumers. Some use machine learning tools; others avoid personally identifying information. However, AI bias is still pervasive in the finance industry and the whole sector has a lot of range to improve.