Spotlight, Neural Information Processing Systems, 2024
We introduce rule extrapolation, a novel metric of OOD compositional generalisation on formal languages, and evaluate it on several AR Language model architectures. We also propose a normative model to explain OOD behaviour via simplicity bias.
Download here
Spotlight, International Conference on Machine Learning (ICML), 2024
We illustrate how statistical generalisation does not explain the emergent abilities of today’s AR probabilistic models, and propose relevant research directions in OOD-generalisation and inductive biases.
Download here
1st Mathematics of Modern Machine Learning Workshop, NeurIPS, 2023
We improve empirical PAC-Bayes bounds by estimating their optimal value using HMC samples from the Gibbs (generalized) posterior.
Download here
14th International OPT Workshop on Optimization for Machine Learning, NeurIPS, 2022
We connect the Sharpness-Aware Minimization (SAM) algorithm to mean-field variational inference and derive novel optimization algorithms building on the said connection.
Download here