Resetting the Conventional Wisdom: Using AI to Reduce Bias

Concerns have been steadily increasing about potential bias in nontransparent or “black box” algorithmic and artificial intelligence or machine learning (AI/ML) systems. Regulators throughout the world, such as Rohit Chopra, director of the Consumer Financial Protection Bureau (CFPB), have issued warnings around the potential misuse of artificial intelligence in lending, employment, medical and other forms of business decision-making. Many jurisdictions are also developing guidelines on AI technology use. ​New York City passed a first-of-its-kind law requiring all automated employment decision tools, including AI-driven ones, to be audited for bias. These concerns are legitimate – nobody wants high-stakes decision-making systems perpetuating bias at scale. However, the discussion overlooks the bigger picture that AI actually provides a tremendous opportunity to improve fairness of a huge swath of decision-making versus current methods.

Traditional decision-making procedures, which AI systems can replace, have a long history of being rife with bias. Unlike these traditional methods, new AI explainability and bias-mitigation technologies coupled with alternative data sources mean that AI systems can systematically measure and mitigate bias. We need to reset the conventional wisdom that AI is a threat to fairness with a new recognition that – properly designed – responsible AI represents the greatest opportunity to make the world less biased. 

The current discussion about bias in AI/ML systems misses this bigger picture for two primary reasons: (1) failure to assess AI systems in comparison to existing decision-making methods and (2) lack of awareness that new technologies have fundamentally changed the capability of developers to measure, manage and optimize the fairness of AI systems.

Missing the Status Quo Comparison: Existing Systems are Often Very Biased

The first shortcoming of the typical discourse on AI is that little attention is paid to the extent of bias in human-driven or non-AI-driven automated decision-making systems. There has been significant room for improvement in the bias of many sociotechnical systems, dating back decades before AI ever became relevant to the conversation. Here, we use the term “bias” to describe what in legal terms would be referred to as disparate impact, or group-level disparities in outcomes.

Financial lending is a prime example. Black and Hispanic communities have long experienced disparate lending decision outcomes versus white communities. This history is why the U.S. Congress passed the Fair Housing Act (FHA) in 1968, Equal Credit Opportunity Act (ECOA) in 1974 and The Community Reinvestment Act (CRA) in 1977. Over time, lending and credit decision-making has moved from human-driven to algorithm- and model-driven through the use of credit scoring and logistic regression models. Even though some improvement in reducing bias has occurred because of this, bias still persists. Studies have shown that credit score-driven models result in much lower lending approval for Black and Hispanic applicants than for their white counterparts.

Employment is another context that has long been rife with disparate impact concerns. Current media narratives often argue that AI platforms are excluding diverse candidates from opportunities, but human decision-makers and traditional methods can be some of the worst culprits when it comes to bias. Audit studies have repeatedly shown that hiring managers evaluate individuals with identical qualifications differently depending on their demographic identity. For example, a resume submitted with a white-sounding name gets 10 interviews, whereas an identical one with a Black or Hispanic name gets 8 and 7, respectively. Prehire assessments are even worse. Cognitive ability tests, used by 50% of employers, including the well-known Wonderlic test used by the NFL, pass only three African American and four Hispanic candidates for every 10 white candidates. Although Title VII of the Civil Rights Act attempted to reduce the use of problematic methods in some cases, legal loopholes have allowed many employers to continue using them today. Compare these extensive findings of bias to the results of a recent meta-analysis, which shows that AI improves the hiring of underrepresented minorities and women by significant amounts. 

Ignoring Technological Progress: AI-based Applications Can Be Transparent and Unbiased

The second main reason that current concerns miss the opportunity that AI represents is that they are based on the outdated assumption that AI systems (1) will necessarily persist historical biases and (2) are unavoidably nontransparent black boxes. These concerns have legitimate roots. AI/ML decision-making models can “learn” bias from the data on which the model is automatically and algorithmically built or “trained.” In addition, because ML systems are automatically created by algorithms instead of handcrafted by humans, the reasons for their predictions can be nontransparent, making it hard for ML system developers or operators to know whether the system’s decisions are fair.

Today, such concerns have become less relevant due to advancements in technology. In recent years, researchers have developed explainable methods that can translate the output and decisions of an ML model into a format that is interpretable for human users. Furthermore, technology now exists to not only measure the bias of ML models but also divine the root cause of any bias, allowing ML developers to optimize lack of bias in outcomes of models. Used properly, these technologies can remove the ML black box.

In addition, ML techniques excel at using new sources of data that can be used to develop decision-making tools to reduce the historical biases associated with traditional data sources. For credit decisions, data on education, behavior, and cell phone, rental and utility bill payment can be used in addition to past credit behavior data. In employment, more nuanced information about individual aptitudes can be collected with modern assessment methods, derived from fields such as cognitive science and behavioral economics, without relying on resumes, transcripts, referrals or self-reported inventories. Without AI/ML, it’s hard to make use of such varied information, but building models out of large and diverse datasets is where machine learning technology shines.

The potential to turn alternative data into novel decision-making procedures is one of the main reasons why AI/ML models can improve on the fairness of traditional methods. For example, in financial lending contexts, studies have shown that using these alternative forms of data in AI/ML credit decisioning models can lead to both fairer and more accurate loan decisions. TruEra, an AI quality and explainability company, and Demyst Data, an alternative data provider, demonstrated this as a winner of the Veritas Global Challenge, sponsored by the Monetary Authority of Singapore. Experian, a U.S.-based credit agency, has launched a service called Experian Boost that allows consumers to add utility, phone and other payment information to their record so that they can receive an increase to their credit score, which the company has claimed increases credit availability to underserved communities.

In the employment context, pymetrics, a behavioral science platform, uses novel cognitive science signals to assess soft skills and help employers find candidates who perform better and stay longer while simultaneously improving diversity. Because the aptitudes that pymetrics measures are broadly distributed across people of varied genders, races, socioeconomic backgrounds, neurodiverse categories and education levels, data inputs do not serve as proxy variables for demographic identity. The result is more equitable candidate evaluations. Over an 18-month period in which approximately 400,000 candidates were evaluated, nearly three times as many Black and Hispanic candidates were identified as strong fits to roles, compared with the number typically observed with human recruiters or traditional hiring tests. Simultaneously, the platform ensured that individuals requesting accommodations for disabilities, such as attention-deficit/hyperactivity disorder and dyslexia, were not disadvantaged when compared to individuals completing the standard version of pymetrics.

Less Biased AI Represents a Business and Social Opportunity – and Regulators Can Help Drive It

The existence of transparent, bias-mitigating AI technologies creates a huge strategic opportunity for all companies who could use automated decisioning systems in high-stakes contexts, including banking, insurance, employment and healthcare. First, AI creates an opportunity to expand markets. Many segments of society are underserved because current decisioning tools deem them unworthy of credit or unqualified for employment opportunities. If companies are willing to adopt different evaluation procedures, many of these historically disenfranchised individuals could become profitable customers, qualified borrowers and high-potential job applicants. Second, these technologies create a new opportunity to realign a company’s purpose and brand to fairness and inclusion ideals.

Having the technology to address fairness concerns is part of the solution, but more can be done to create an environment that facilitates the adoption of these technologies. Policymakers can and should create regulations and laws to require transparency and bias reporting in all automated decision systems, including AI/ML systems. Some are already doing this. Regulators in U.S. financial services (CFPB), employment (Equal Employment Opportunity Commission), trade (Federal Trade Commission), healthcare (Food and Drug Administration) and more, and others internationally, have already started developing guidelines to govern automated decisioning systems and AI. Legislators have also started to propose laws, such as the European Commission’s Artificial Intelligence Act. Governments can also create incentives for the use of bias-mitigating and transparent technologies by providing tax credits or purchasing these technologies themselves.

The discussion around bias in algorithmic- and AI-driven systems is important. We need to make sure that any new decision-making technology doesn’t lead to biased outcomes. However, to date, the discussion has been one-sided and focused on risk and downsides. With the development of new bias-mitigating and transparency technologies, we now have the opportunity to reset the conventional wisdom and shift the discussion toward how to maximize the potential advantages of algorithmic systems while minimizing the potential risks. The reality is that given recent technology advances, if you care about reducing bias in society, then not only should you be calling for transparency and bias mitigation of automated/AI systems but you should also be encouraging greater use of these systems in important decision-making in ways that improve outcomes versus existing suboptimal, status quo decision-making systems. Properly regulated, bias-mitigated, transparent AI systems represent the greatest opportunity ever to truly measure and reduce bias in our society. And that opportunity is something we should wholeheartedly embrace.