The Institute for Statistics and Mathematics is pleased to invite you to the next research seminar, taking place on campus:

Aliaksandr Hubin (Department of Mathematics, University of Oslo)
Boosting Performance of Latent Binary Neural Networks With a Local Reparametrization Trick and Normalizing Flows
Friday, November 25, 2022, 10:30 am, Building TC, Room TC.3.01

Abstract:
An artificial neural network (ANN) is a powerful machine learning method that is used in many modern applications such as facial recognition, machine translation and cancer diagnostics. A common issue with ANNs is that they usually have millions or billions of trainable parameters, and therefore tend to overfit to the training data. This is especially problematic in applications where it is important to have reliable uncertainty estimates. Bayesian neural networks (BNN) can improve on this, since they incorporate parameter uncertainty. In addition, latent binary Bayesian neural networks (LBBNN) also take into account model uncertainty, enabling inference in both model and parameter space. In this paper, we will consider two extensions to the LBBNN method: Firstly, by using the local reparametrization trick (LRT) to sample the hidden units directly, we get a more computationally efficient algorithm. Secondly, by using normalizing flows on the variational posterior distribution of the LBBNN parameters, the network learns a more flexible variational posterior distribution than the mean field Gaussian. Experimental results show that this improves significantly on predictive power compared to the LBBNN method, while also obtaining a more sparse network. Additionally, we perform a simulation study where the normalizing flow method performs best at variable selection.

Joint work with Lars Skaaret-Lund and Geir Storvik.

We aim to stream all on-campus talks via Zoom. A direct link to the stream will be posted on our website.

For further information and the seminar schedule, please see:
www.wu.ac.at/en/statmath/research/resseminar