Soufiane Hayou

Soufiane Hayou is an assistant professor in the Department of Applied Mathematics and Statistics at Johns Hopkins University and a member of the university’s Data Science and AI Institute (DSAI). Prior to joining Johns Hopkins, he held faculty and research appointments at leading institutions including the National University of Singapore, where he was a Peng Tsu Ann Assistant Professor of Mathematics, and the Simons Institute for the Theory of Computing at UC Berkeley, where he contributed to the Collaboration on the Theoretical Foundations of Deep Learning under the mentorship of Bin Yu and Peter Bartlett.

Hayou’s research sits at the intersection of theory and application in modern machine learning. His work leverages mathematical tools to understand the behavior of large-scale neural networks and inform principled strategies for improving their training, fine-tuning, and deployment. A central focus of his current research is the efficiency and scalability of large language models, aiming to develop Pareto-optimal methods across all stages of the modeling pipeline—from pre-training to post-training and inference.

His contributions to the field include the invention of LoRA+, an advanced fine-tuning technique that extends LoRA and is now integrated into widely used libraries such as HuggingFace’s PEFT and LlamaFactory. He also introduced Stable ResNet, a novel architecture that improves stability in deep networks, and co-developed Depth-μP, a method for scaling neural networks by depth. His theoretical work has advanced understanding of commutative width and depth scaling in residual networks, providing key insights into how deep learning models behave as they grow in complexity. His research is regularly published in leading venues such as ICML, NeurIPS, ICLR, and JMLR.

Hayou has received multiple honors for his work, including the 2024 Gradient AI Fellowship and recognition as a Rising Star in AI by both CPAL (2025) and KAUST (2023). Earlier in his academic career, he was awarded the Natixis Best Master Thesis in Quantitative Finance.

In addition to his research contributions, Hayou is active in graduate and undergraduate education. At the National University of Singapore, he taught courses such as Principles of Machine Learning and Advanced Topics in Machine Learning, reaching over 100 students per semester. He has also served as a teaching assistant and class tutor at the University of Oxford, where he supported instruction in Bayesian inference and simulation methods.

His industry experience includes consulting roles with AI startups and internships at major financial institutions such as J.P. Morgan, Bloomberg, and G-Research, where he developed machine learning algorithms for financial applications.

Hayou is a sought-after speaker and has delivered invited talks at institutions and companies including Nvidia Research, Microsoft, Uber, EPFL, the University of Toronto, and the Alan Turing Institute. He actively contributes to the research community by reviewing for major conferences (NeurIPS, ICML, ICLR) and journals (JMLR, TMLR, SIMODS, PNAS, BERNOULLI), and has been recognized as a Top Reviewer across multiple venues.

He earned his PhD in Statistics and Machine Learning from the University of Oxford under the supervision of Arnaud Doucet and Judith Rousseau. He also holds a master’s degree in Applied Mathematics and an engineering diploma from École Polytechnique, as well as a master’s in Probability and Financial Mathematics from Pierre et Marie Curie University (DEA El Karoui, Paris VI).