Getting Model Explainability Right Is Everything In AI Lending

Zest AI has helped more lenders put AI-driven credit underwriting models into production than anyone on the planet. We’ve learned -- sometimes the hard way -- what it takes to get up and running with compliant AI credit models that approve more good borrowers faster with no added risk and full compliance. We think (and we’re not alone on this) that every lender will eventually switch from traditional credit scoring to AI-driven lending. The gains are just too big to ignore.

But lenders need to get the AI right. So, we want to share what we’ve learned with an industry that’s just beginning to get its arms around the seismic shift.

Our latest Zest Guide, The Five Building Blocks For Compliant AI Lending, just came out this week and focuses on the best practices in getting ML through compliance. It covers all the key concerns:

• fair lending
• bias
• transparency
• safety and soundness
• data sources and monitoring

Sure, ML adds complexity. But it’s the kind of complexity that comes with initially mastering any new technology -- and we can help you manage the learning curve. We spent the last half-decade building the software automation tools that will help any lender spot more good borrowers overlooked by legacy credit scores.

Here’s one piece of advice from the new Zest Guide: When it comes to decide how you’re going to explain a credit model, be sure you pick a method that matches your decision-making engine. For example, if you outsource your underwriting to an industry score provider, you can (legally) rely on what they tell you. But if you use an old-school custom model, you can use old-school explainability methods like drop one or impute median. And if you use ML, you need to switch over to an explainability method that relies on game-theory math from Nobel Prize-winning mathematician Lloyd Shapley.

Regulators are generally okay with a method so long as it produces accurate reasons for approvals and denials.

As you probably know, lenders are legally required to explain why they made their lending decisions, whether the decisions are made automatically using an algorithm or using manual underwriting. For approvals, safety, and soundness, regulators want to ensure lenders aren’t making bad loans. And for denials, consumer protection regulators want to ensure that lenders comply with the Equal Credit Opportunity Act and state and federal fair lending laws. For algorithmic decisions, this means that lenders must explain and document how the algorithm works.

Download the new Zest Guide, The Five Building Blocks For Compliant AI Lending, here. 

When lenders rely on the algorithm behind a general industry score produced by a credit reporting agency (“CRA”), they are legally allowed to rely on reason codes provided to them by the CRA. That makes life easy in one sense because the lender has effectively outsourced its underwriting to a third party. But, in another sense, life is much more complicated because industry scores are becoming less accurate every day and because industry scores are widely recognized as racist and unfairly biased against black people. So, those algorithms and explanations will face increasing regulatory scrutiny as time goes by.

Custom algorithms, especially ML ones, can solve the accuracy and racism problems (more on that later), but require lenders to produce their own explanations. There are several ways to explain an algorithmic decision--whether it’s made by an ML model or a traditional one. Regulators are generally okay with a method so long as it produces accurate reasons for approvals and denials. That’s where things can get a little tricky with ML.

Some lenders try to explain ML models the same way they explain other algorithms. They tend to use one of two seemingly reasonable methods: “drop one” and its cousin “impute median.” With drop one, lenders test which model variables contribute most to the model score by removing one variable and measuring the change in the score. With impute median, lenders do the same thing but, instead of dropping a variable, they replace each variable, one at a time, with the median value of that variable in the dataset.

The problem with drop one and impute median is that they can only handle single-direction, single-variable relationships (an increase in variable X always increases the score). A machine learning model can have hundreds of credit variables with both positive or negative influences on the score. You need serious math to explain what’s going on inside the ML model.

So how does one accurately capture and explain the interaction of so many variables (ML models often have hundreds or thousands) to identify the most important factors influencing the model’s decision? The answer turns out to be quite simple: Those mathematicians started trying to quantify how each player on a sports team contributed to the game’s final score, taking into account the number of baskets, touchdowns, or goals the player scored and the player’s assists, passes, and blocks. Game theory pioneer Lloyd Shapley eventually won the Nobel Prize in Economics because of this work.

Shapley’s proofs turn out to be a rock-solid way to explain how ML models make decisions. In the case of an ML model, the “players” are the model variables, the “game” is the model, and the “score” is the model’s output (in credit, this usually represents the probability of defaulting on a loan). That’s why applying game theory math to ML models works so well.

Shapley’s method precisely quantifies the significance of model variables in generating a given score for a given applicant. It also considers variable interactions to know precisely how much each variable contributed to the credit decision. Shapley proved his method is the only reasonable way of assigning credit in games of this sort. We have used Shapley’s approach (and its formal extensions) to explain millions of ML lending decisions for many different kinds of models throughout our history as a company.

Without accurate explainability, lenders don’t have real insight into their lending decisions. That leaves them exposed to financial and compliance risks, often without even knowing it. In contrast, accurate explainability renders the opaque transparent and allows lenders to reap the many benefits of ML underwriting: higher approvals, lower risk, increased fairness and inclusivity.

Download the new Zest Guide, The Five Building Blocks For Compliant AI Lending, here.