Choosing a Machine Learning Algorithm Under Constraints: An Example From The Financial World

Data science is increasingly being utilised in the financial industry. Applications include but not limited to, optimization of cash held in ATMs, location analytics of ATMs and bank branches as well as risk analytics in credit allocation. Various machine learning methods are used to solve the above like challenging problems. Inherent to any machine learning problem, of course, is the central issue of model selection. In this post, we will take a look at what different factors might affect the selection of machine learning algorithm in real life problem specific to the financial world.

Model Complexity

One of the aspects that distinguishes machine learningblog1
15 models is the complexity of the model. The more complex
the model is, it will be more likely to clasp on subtler, more
indirect and non-linear relationships in the data.

So, why not use the more complex models more often, then? Well, complex models are not free of drawbacks, of course. Most importantly, they run the risk of "overfitting" the data in hand. That is, they may learn the randomness in the data as well as the underlying pattern — when it comes to making predictions for the future, their predictions can go awry.

Therefore, solving any machine learning problem involves a balance between many concerns. Most of the time, what one can ultimately judge a machine learning algorithm is by how well its predictions perform on a new data set it has not grasped before.

So let's compare two algorithms in an example case: risk level prediction for credit card customers.

Risk Level Prediction Example



To match the right credit card limit with the right customer is one of the major problems in the banking sector.The goal of credit card issuers is to classify potential credit-card accounts into two categories: those who will default in the near future, and those who will not.

In this example, we will compare two machine learning models within the same data set: a logistic regression and an artificial neural network. A logistic regression is a widely used algorithm that is simple yet robust. On the other hand, artificial neural networks are very powerful and very complex algorithms - thus are sometimes prone to the overfitting problem. Which one will prevail in this case?

Comparing the Performance of the Two Algorithms

We trained both algorithms on a set of real life data, and assessed their performances. When we analyse the results, we see that the artificial neural network outperforms logistic regression. Given that 10,936 people who actually defaulted, the logistic regression was able to detect 1964 of them, whereas the artificial neural network could detect 2142. Moreover, while the logistic regression falsely predicted 1274 people to default, the neural network lowered this number to 905. So in this case, artificial neural networks seem to have the upper hand. With overfltting having been kept in check, artificial neural networks' capability of capturing nonlinear relations and interactions between different features seem to have given it an edge over the logistic regression in this case.

The Verdict

Given their power, one would expect artificial neural networks to be used widely enough in risk level prediction, and in banking sector more generally. This is not exactly the case, however. Sometimes, the performance itself is not the only metric. In certain cases, what may be more preferable in an algorithm is that its results are interpretable. Although more powerful, ANNs are better described as "black box" algorithms. For example, when an ANN is used, it's not necessarily very clear with which features contributed the most to predict whether a person will default or not: is it her/his income level, is it her/his spending patterns, is it the demographics, or something else? On the other hand, this information is readily available in a logistic regression model. In cases where performance is the only concern, including those in the financial sector, a "black box" algorithm might be perfectly fine. But in other cases, it's important for the stakeholders that the resulting decision process is transparent as well as reasonably accurate.

Developing the right solution to a real life problem is rarely just an applied mathematics problem. It requires awareness of business demands, rules and regulations, and stakeholders' concerns as well as considerable expertise. In solving a machine problem, being able to combine and balance these is crucial; those who can do this can create the most value.




Data Science Lab