New journal article published in Transactions on Machine Learning Research

Denis Huseljic, Marek Herde, Yannick Nagel, Lukas Rauch, Paulius Strimaitis and Bernhard Sick have published an article titled The Interplay of Uncertainty Modeling and Deep Active Learning: An Empirical Analysis in Image Classification in the journal Transactions on Machine Learning Research. This is the topic of the article:

Deep active learning (AL) seeks to reduce the annotation costs required for training deep neural networks (DNNs). Often, deep AL strategies focus on instances where the predictive uncertainty of a DNN is high. Furthermore, Bayesian concepts to model uncertainty are frequently adopted. Despite considerable research, a detailed analysis of the role of uncertainty in deep AL is still missing, especially regarding aleatoric and epistemic uncertainty, both related to predictive uncertainty. This article provides an in-depth empirical study analyzing the interplay of uncertainty and deep AL in image classification. Our study investigates four hypotheses that provide an intuitive understanding of the effects of accurately estimating aleatoric and epistemic uncertainty on existing uncertainty-based AL strategies but also, in the opposite direction, the impact of uncertainty-based AL on the quality of uncertainty estimates that are needed in many applications. By analyzing these hypotheses on synthetic and real-world data, we find that accurate aleatoric estimates can even impair instance selection, while accurate epistemic estimates have negligible effects. Moreover, we provide a publicly available toolbox for deep AL with various models and strategies to facilitate further research and practical applications. Code is available at github.com/dhuseljic/dal-toolbox.