Kalman Linear Attention: Parallel Bayesian Filtering For Efficient Language Modelling and State Tracking
ICML 2026
We introduce Kalman Linear Attention, which formulates sequence modelling as a Bayesian filtering problem.
I am a Senior Lecturer (Associate Professor) at the School of Engineering, University of Edinburgh. I co-lead the Bayesian and Neural Systems research group.
My research interests in machine learning include:
- Understanding what works in practice
- Automated machine learning
- Data and compute efficiency
- Memory and adaptation in agents
I have an MEng in Engineering Science and a DPhil (PhD), both from the University of Oxford. My DPhil was on visual recognition in art, supervised by Andrew Zisserman in the VGG group.
I am an ELLIS Scholar and have received an EPSRC New Investigator Award.
In addition to research, I am Programme Manager for Electronics and Electrical Engineering at Edinburgh, and I have developed a machine learning course.
Team
Current
- Shiwen Qin (PhD student, with Shay Cohen)
- Cameron Barker (PhD student, with Henry Gouk)
- Thomas Woehrle (PhD student)
- Graeme Kempf (PhD student)
Former
- Dr Miguel Espinosa (PhD)
- Dr Linus Ericsson (Postdoc)
- Dr Chenhongyi Yang (PhD)
- Dr Jack Turner (PhD, with Michael O'Boyle)
Carefully selected publications
Transferrable Surrogates in Expressive Neural Architecture Search Spaces
AutoML 2025Code
We accelerate NAS in expressive search spaces by using transferrable performance prediction methods.
einspace: Searching for Neural Architectures from Fundamental Operations
NeurIPS 2024Code
A new expressive search space for neural architecture search based on a probabilistic context-free grammar.
PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition
BMVC 2024Code
A simple non-hierarchical state space model designed for general visual recognition.