Sr Director Applied Research

Basic Qualifications:

1. Ph.D. plus at least 6 years of experience in Applied Research or M.S. plus at least 8 years of experience in Applied Research
2. At least 5 years of people leadership experience

Preferred Qualifications:

1. PhD in Computer Science, Machine Learning, Computer Engineering, Applied Mathematics, Electrical Engineering or related fields
2. LLM PhD focus on NLP or Masters with 10 years of industrial NLP research experience
3. Core contributor to team that has trained a large language model from scratch (10B + parameters, 500B+ tokens)
4. Numerous publications at ACL, NAACL and EMNLP, Neurips, ICML or ICLR on topics related to the pre-training of large language models (e.g. technical reports of pre-trained LLMs, SSL techniques, model pre-training optimization)
5. Has worked on an LLM (open source or commercial) that is currently available for use
6. Demonstrated ability to guide the technical direction of a large-scale model training team
7. Experience working with 500+ node clusters of GPUs
8. Has worked on LLM scaled to 70B parameters and 1T+ tokens
9. Experience with common training optimization frameworks (deep speed, nemo)

Behavioral Models:

1. PhD focus on topics in geometric deep learning (Graph Neural Networks, Sequential Models, Multivariate Time Series)
2. Member of technical leadership for model deployment for a very large user behavior model
3. Multiple papers on topics relevant to training models on graph and sequential data structures at KDD, ICML, NeurIPs, ICLR
4. Worked on scaling graph models to greater than 50m nodes

Optimization (Training & Inference):

1. PhD focused on topics related to optimizing training of very large language models
2. 5+ years of experience and/or publications on one of the following topics: Model Sparsification, Quantization, Training Parallelism/Partitioning Design, Gradient Checkpointing, Model Compression

Finetuning:

1. PhD focused on topics related to guiding LLMs with further tasks (Supervised Finetuning, Instruction-Tuning, Dialogue-Finetuning, Parameter Tuning)
2. Demonstrated knowledge of principles of transfer learning, model adaptation and model guidance
3. Experience deploying a fine-tuned large language model

Data Preparation:

1. Numerous Publications studying tokenization, data quality, dataset curation, or labeling
2. Leading contributions to one or more large open source corpus (1 Trillion + tokens)
3. Core contributor to open source libraries for data quality, dataset curation, or labeling