Four tips for establishing an automated model risk management plan

There are significant benefits for organizations transitioning to AI-automated underwriting technology when it come to using automation in the model risk management (MRM) process. Automation improves operational efficiency and allows your organization to keep up with changing market conditions. An automated MRM process also facilitates knowledge transfer to new employees and provides your regulators with all the information they need to verify your model.

So, how do you make sure that you have a robust plan in place for automated MRM?

With new improvements and uses of machine learning coming every day, automated model risk management processes are a must for financial institutions working to stay ahead of the curve. Following these best practices will help ensure that your AI credit decisioning model is accurate, powerful, and safe — allowing you to maximize operational efficiencies and improve your institutions ability to better identify risk.

   
1. Document the data your model uses – It all begins with data. It’s important to track, validate, and measure the data used to generate the model. A strong MRM process should include the reasons for using different features, how missing data is being imputed, and the code that can be used to replicate the exact dataset for audits. During model development, it’s also important to catalog which algorithms were used and why, along with how these models performed based on a diverse set of performance metrics.
   
2. Ensure you understand how your model makes decisions and their business impact – Your model must be explainable before going into production. That requires understanding global feature importance, individual decision drivers, and knowing how the input variables interact with one another. Knowing a model’s blind spots, potential for leaky variables, and its stability and scalability in production are also all essential before going live. The economic impact of the model must be quantifiable too. How does the model impact revenue? How does it affect the distribution of scores? Is the approval or default rate changing? The answers to these questions must be quantified, cataloged, and stress-tested. Without this level of explainability for evaluating the model and its business impact, the model you put into production will look nothing like the model you trained in development. Which could translate into a major risk, should the regulators ever come knocking at your door.
   
3. Proactively satisfy regulatory requirements and expectations – Ensuring that protected classes are not being discriminated against in the model is an ethical imperative and a legal requirement. Yet, all too often, banks take a wait-and-see approach with their model’s credit underwriting decisions, waiting to get the results and then evaluating whether there was disparate impact. However, disparate impact analysis should be performed in real time, not after potentially discriminatory credit decisions have been made. Any variable that is driving scores in the model must be explainable to a potential borrower.
   
4. Monitor your model – A robust MRM process will limit the number of surprises when a model is put into production, but it cannot completely eliminate the unknown. Ongoing model monitoring ensures that the model is performing as expected. There must be a process in place to make sure any degradation in the model can be immediately investigated and remedied. Monitoring how different variable distributions are changing over time can alert the business when it’s time to refit the model. Strong model monitoring can help you get ahead of the next macroeconomic shock and thrive in a tough environment while competitors lag behind and struggle to survive.

‍
By applying these four tips to your AI-automated lending practice, you can make better, smarter credit decisions and achieve an overall better underwriting experience at your organization.