Assessing and mitigating unfairness in credit models with the Fairlearn toolkit

As AI plays an increasing role in the financial services industry, it is essential that financial services organizations anticipate and mitigate unintended consequences, including fairness-related harms, such as denying people services, initiating predatory lending, amplifying gender or racial biases, or violating laws such as the United States’ Equal Credit Opportunity Act (opens in new tab) (ECOA). To address these kinds of harms, fairness must be explicitly prioritized throughout the AI development and deployment lifecycle.

To help organizations prioritizing fairness in AI systems, Microsoft has released an open-source toolkit called Fairlearn (opens in new tab). This toolkit focuses on the assessment and mitigation of fairness-related harms that affect groups of people, such as those defined in terms of race, sex, age, or disability status.

Using a dataset of loan applications, we illustrate how a machine learning model trained with standard algorithms can lead to unfairness in a loan adjudication scenario, and how Fairlearn can be used to assess and mitigate this unfairness. The model, which is obtained by thresholding the predictions of probability of default (PD), leads to an uneven distribution of adverse events for the “male” group compared to the “female” group even though this model does not use sex as one of its inputs. Fairlearn’s mitigation algorithms reduce this disparity from 8 percentage points to 1 percentage point without any (statistically significant) impact on the to the financial services organization.

We emphasize that fairness in AI is a sociotechnical challenge, so no software toolkit will “solve” fairness in all AI systems. However, software toolkits like Fairlearn can still play a valuable role in developing fairer AI systems—as long as they are precise and targeted, embedded within a holistic risk management framework, and supplemented with additional resources and processes.