Sheheryar Zaidi

I am a Research Scientist at Google DeepMind. Previously, I was a PhD student at the Department of Statistics of the University of Oxford supervised by Yee Whye Teh and Arnaud Doucet, generously funded by Aker Scholarship and the Google PhD Fellowship in Machine Learning.

I am broadly interested in deep learning, with the aims of

• Fundamentally understanding neural networks and their training techniques
• Applying this understanding to build high performance deep learning systems for solving problems, particularly scientific ones

Specifically, I have worked on equivariance in deep learning, uncertainty estimation & robustness, learning dynamics, neural architecture search and applications of deep learning to scientific domains.

Before starting my PhD, I completed a bachelor’s in math at the University of Oxford – receiving the Junior Mathematical Prize for graduating top three. I obtained a master’s in math (Part III) with distinction at the University of Cambridge. I mostly specialized in pure math, particularly algebra, topology and representation theory.

I completed a research internship at DeepMind with Razvan Pascanu in 2022. Before that, I interned at Man AHL (summer 2019) and Morgan Stanley (summer 2017) in quantitative trading/research.

You might find my first name difficult to pronounce – the short version is Sheh (pronounced ‘Sha’). 🦔

---

Publications

Pre-training via Denoising for Molecular Property Prediction  
Sheheryar Zaidi*, Michael Schaarschmidt*, James Martens, Hyunjik Kim, Yee Whye Teh, Alvaro Sanchez-Gonzalez, Peter Battaglia, Razvan Pascanu, Jonathan Godwin
*Equal contribution.
Spotlight presentation at ICLR, 2023
arXiv   GitHub   Poster

When Does Re-initialization Work?  
Sheheryar Zaidi*, Tudor Berariu*, Hyunjik Kim, Jörg Bornschein, Claudia Clopath, Yee Whye Teh, Razvan Pascanu
*Equal contribution.
Spotlight presentation at I Can’t Believe It’s Not Better Workshop at NeurIPS, 2022
Published in PMLR volume 187
arXiv   Poster

LieTransformer: Equivariant Self-Attention for Lie Groups
Michael Hutchinson*, Charline Le Lan*, Sheheryar Zaidi*, Emilien Dupont, Yee Whye Teh, Hyunjik Kim
*Equal contribution.
ICML, 2021
arXiv   GitHub

Provably Strict Generalisation Benefit for Equivariant Models
Bryn Elesedy, Sheheryar Zaidi
ICML, 2021
arXiv

Neural Ensemble Search for Uncertainty Estimation and Dataset Shift
Sheheryar Zaidi*, Arber Zela*, Thomas Elsken, Chris Holmes, Frank Hutter, Yee Whye Teh
*Equal contribution.
NeurIPS, 2021
Oral presentation at ICML 2020 Worshop on Uncertainty & Robustness in Deep Learning
arXiv   GitHub

Effectiveness and resource requirements of test, trace and isolate strategies for COVID in the UK
Bobby He*, Sheheryar Zaidi*, Bryn Elesedy*, Michael Hutchinson*, Andrei Paleyes*, Guy Harling, Anne M Johnson, Yee Whye Teh
*Equal contribution.
Royal Society Open Science, 2021
DOI

Efficient Bayesian Inference of Instantaneous Reproduction Numbers at Fine Spatial Scales, with an Application to Mapping and Nowcasting the Covid-19 Epidemic in British Local Authorities
Yee Whye Teh, Avishkar Bhoopchand, Peter Diggle, Bryn Elesedy, Bobby He, Michael Hutchinson, Ulrich Paquet, Jonathan Read, Nenad Tomasev, Sheheryar Zaidi
Royal Statistical Society’s Covid-19 Task Force: Special Topic Meeting on R/local R/transmission, 2021
PDF   Website

Target–Aware Bayesian Inference: How to Beat Optimal Conventional Estimators
Tom Rainforth*, Adam Goliński*, Frank Wood, Sheheryar Zaidi
*Equal contribution.
JMLR, 2020
URL