Credit & Risk

Why Credit Underwriting Is No Longer A Set-It-And-Forget-It Job

Zest AI team

September 20, 2021

We already shared with you the insight about how more accurate credit underwriting requires looking at a borrower’s trend line, not focusing on point-in-time data. That’s especially crucial in the current moment as we continue to see a lot of uncertainty and market volatility. Credit scores went up after Covid-19 tanked the economy. More than a year later, credit scores continue to rise, even though the economy is not out of the woods yet. Are we going to see credit scores nosedive? What sort of reliability or stability do we have from those scores?

In times of change, using the same generic scores or waiting for one to two years to revisit your scorecard could put you in a world of hurt. Lending teams need to be able to respond more quickly when change accelerates to avoid the real cost of taking too long to update their scoring. Formulating the right response means having real transparency or interpretability in your current credit models.

The latest Zest Guide, "Six Million Credit Applications Later, What Have We Learned About AI-Driven Lending," is out now. Download it today.

What we’ve been saying for a number of years now -- and hopefully it’s catching on -- is that legacy credit models dependent on logistic regression (like FICO) are not as straightforward as they seem. Some vendors of logistic regression models lay claim to using machine learning in the upfront process of building model variables. But in doing so they’re making some pretty strong assumptions about the lack of collinearity between features in these models. What they’re saying is, “Well, I couldn't use machine learning. So I used machine learning to build a feature, and then I shoved the feature into just a logistic regression model. But it's still a logistic regression model, so I'm okay." Which is absurd. We’ve taken to calling these models “LRINO,” or logistic regression in name only.

Features that you create through joint distributions, maybe through interesting non-linear transformations of the data, create significant sensitivity that is not well managed or not well tracked by the industry at large yet a lot of logistic regressions models have gotten this rubber stamp of approval. Our position has always been that the same rigor should be applied to these logistic regression models that people apply to every other credit model that's powering your business. 

That brings up a second point about not “setting and forgetting,” and that’s the paramount need for robust model monitoring. You need to be able to manage the model once you've put it into production. Not every lender is prepared to do the rigorous monitoring required to make sure that the model and loan performance are as expected. Adequately sensitive monitors can indicate when features are drifting or the score distributions are changing, and when it’s time for model or policy adjustments due to changes in market conditions or shifts in borrower characteristics.

For example, even before the pandemic, one of our long-time customers, Akbank, one of the largest banks in Turkey, was using Zest model monitoring and caught indications of a looming currency-driven recession. The bank was able to make adjustments proactively to its credit policy prior to the economy hitting a rough spot. Fast forward a couple of years to early 2020 when a personal loans customer in the U.S. saw a sudden and dramatic spike in credit-line drawdowns. With the naked eye, it could have been a sign of bust-out fraud, but more sensitive multivariate ML monitors showed it was merely customers with solid credit taking out funds just as the pandemic was hitting. The model’s approvals had been fine. The model was behaving as it was intended during the period of development.

Many lenders we work with recognize there’s a new approach needed to model management, one that’s more proactive and agile and driven by better monitoring. We’re not talking about free-wheeling AI that’s changing all the time. Far from it. These are rigorously documented and locked-down models once they’re trained and validated. But we are talking about doing model refits within a matter of weeks, not a year when conditions are in flux. And it’s not feasible to do these kinds of monitoring and performance checks with logistic regression models or scorecards or even manually. A software-driven approach is a key to managing the additional volume and complexity that these modeling techniques bring and helping risk teams know what they need to keep their eyes on. You don't need to dread re-evaluating those scorecards or models. With software automation throughout the model lifecycle, you’ve now got another tool in the toolbox to find additional value in your lending business.

The latest Zest Guide, "Six Million Credit Applications Later, What Have We Learned About AI-Driven Lending," is out now. Download it today.

Thank you for subscribing!
Something went wrong while submitting the form.