My research is motivated by the need to accelerate foundation model (FM) adoption toward solving humanity’s most challenging problems. Doing so is a long-term effort requiring substantial community involvement. The goal of my Ph.D. research is to take steps towards realizing this high-impact vision, categorized roughly into three sub-topics:
- The science of scaling laws,
- Automation for improving FMs beyond naive scaling, and
- Determining how FMs interact with data.
While furthering these directions for language, I have had the unique opportunity to pretrain LLMs at industrial scales. On the other hand, to accelerate adoption of FMs beyond language, I have also worked with a wide array of problems from different scientific domains, which includes solving PDEs, protein folding, climate modelling, and beyond.