On weight and variance uncertainty in neural networks for regression tasks
arXiv:2501.04272v1 Announce Type: new
Abstract: We consider the problem of weight uncertainty proposed by [Blundell et al. (2015). Weight uncertainty in neural network. In International conference on machine learning, 1613-1622, PMLR.] in neural networks {(NNs)} specialized for regression tasks. {We further} investigate the effect of variance uncertainty in {their model}. We show that including the variance uncertainty can improve the prediction performance of the Bayesian {NN}. Variance uncertainty enhances the generalization of the model {by} considering the posterior distribution over the variance parameter. { We examine the generalization ability of the proposed model using a function approximation} example and {further illustrate it with} the riboflavin genetic data set. {We explore fully connected dense networks and dropout NNs with} Gaussian and spike-and-slab priors, respectively, for the network weights.
Abstract: We consider the problem of weight uncertainty proposed by [Blundell et al. (2015). Weight uncertainty in neural network. In International conference on machine learning, 1613-1622, PMLR.] in neural networks {(NNs)} specialized for regression tasks. {We further} investigate the effect of variance uncertainty in {their model}. We show that including the variance uncertainty can improve the prediction performance of the Bayesian {NN}. Variance uncertainty enhances the generalization of the model {by} considering the posterior distribution over the variance parameter. { We examine the generalization ability of the proposed model using a function approximation} example and {further illustrate it with} the riboflavin genetic data set. {We explore fully connected dense networks and dropout NNs with} Gaussian and spike-and-slab priors, respectively, for the network weights.
Moein Monemi, Morteza Amini, S. Mahmoud Taheri, Mohammad Arashi
Go to original source