Overfitted algorithms lead to an overconfident model when the learner is more confident in its prediction than what the data depicts. For the model, the whole source of truth is the training data. When the model rote learns instead of understanding the underlying characteristic it results in overconfidence. Another reason is that the deep learning models have way too many parameters and use ReLU as activation function which yields high-confidence.“Today AI-ML models are used throughout the DevOps cycles. And an overconfident model can ruin the entire lifecycle of the project. The challenges that any company may face include failure of the supply chain, wrong price, and cost estimations which are the pillars for any company. In Banking it may lead to higher fraudulent transactions as well as lesser leads. And it can cost lives in the healthcare sector,” said Anjna Bhati – Head, Data Analytics and AI at BluePi Consulting.
So what to do?
Industry leaders suggest adopting and using Bayesian methods to quantify uncertainty. Few of the reasons for overconfident models are overfitting, smaller datasets which can easily be dealt with using regularisation and data augmentation but that doesn’t solve the problem as we need to quantify uncertainty because in real data there will always be garbage input which model hasn’t seen.
Bhati advises having a posterior over the weights approximated with a Laplace approximation in the last layers of neural networks.
“Also we should pass some garbage images and assess the model confidence on those images during training and test as well. Businesses can save themselves from overconfident models with continuous assessment of their data pipelines and the model,” she added.
Prasanna Sattigeri, – Research Staff Member, IBM Research AI, MIT-IBM Watson AI Lab also suggested using better models such as for ensembles and others that follow Bayesian principles. These tend to suffer less from overconfidence.
“Improve existing models through recalibration techniques. These are post-processing methods that can be used to reduce overconfidence,” he said.
For predictive models and ML models, a lot depends on the training data. We can use training data to check if the predictions are right or wrong. Prashanth Kaddi, Partner, Deloitte India explains how:
“Let’s say we have data of 36 months from Jan 2018 to Jan 2021. We will only use 30 months of data starting from Jan 2018 to July 2020 to make predictions. And then match these predictions with what really happened from August 2020 to Jan 21. This way your data will tell you if it’s predicting right or wrong. You will know if the model is overconfident or overfitted,” Kaddi said.
A few other measures like testing the model for 3 months before putting it in place will help rule out any bias, overfitting, and overconfidence. If there seems to be a constant bias in the predictions it hints that there may be an issue.
