Principal Generative AI Systems Engineer - GPU Performance Architecture

Advanced Micro Devices, Inc.
USD $193,600.00/Yr.-USD $290,400.00/Yr.
United States, Texas, Austin
7171 Southwest Parkway (Show on map)
Jan 28, 2025
WHAT YOU DO AT AMD CHANGES EVERYTHING We care deeply about transforming lives with AMD technology to enrich our industry, our communities, and the world. Our mission is to build great products that accelerate next-generation computing experiences - the building blocks for the data center, artificial intelligence, PCs, gaming and embedded. Underpinning our mission is the AMD culture. We push the limits of innovation to solve the world's most important challenges. We strive for execution excellence while being direct, humble, collaborative, and inclusive of diverse perspectives. AMD together we advance_ THE ROLE: As a Principal Engineer, you will spearhead the next generation of AI infrastructure by defining GPU architecture specifications that enable massive model training at scale. Your expertise will drive 2-3x performance gains in both training and inference pipelines through innovative system design and optimization. You will champion the adoption of cutting-edge techniques across the engineering organization, from efficient attention mechanisms to advanced parallelization strategies. By establishing comprehensive best practices for distributed ML systems, you will create a framework that enables seamless scaling from single-GPU to thousand-GPU deployments. THE PERSON: You have a deep understanding of GPU microarchitecture, memory hierarchies, and their impact on large-scale ML workloads You are passionate about software engineering and possess leadership skills to drive sophisticated issues to resolution. You are able to communicate effectively and work optimally with different teams across AMD. KEY RESPONSIBILITIES: Lead performance modeling and optimization for multi-trillion parameter LLM training/inference including Dense, Mixture of Experts (MoE) with multiple modalities (text, vision, speech) Model/optimize novel parallelization strategies across tensor, pipeline, context, expert and data parallel dimensions Architect memory-efficient training systems utilizing techniques like structured pruning, quantization (MX formats), continuous batching/chunked prefill, speculative decoding Incorporate and extend SOTA models such as GPT-4, Reasoning models (Deepseek-R1), and multi-modal architectures Collaborate with internal and external stakeholders/ML researchers to disseminate results and iterate at rapid pace. REQUIRED EXPERIENCE: Extensive and Senior experience optimizing large-scale ML systems and GPU architectures Deep expertise in CUDA programming, GPU memory hierarchies, and hardware-specific optimizations Proven track record architecting distributed training systems handling large scale systems Expert knowledge of transformer architectures, attention mechanisms, and model parallelism techniques PREFERRED EXPERIENCE: PyTorch, CUDA, TensorRT, OpenAI Triton Distributed systems: Ray, Megatron-LM Performance analysis tools: NSight Compute, nvprof, PyTorch Profiler KV cache optimization, Flash Attention, Mixture of Experts High-speed networking: InfiniBand, RDMA, NVLink ACADEMIC CREDENTIALS: Bachelors, MS/PhD in Computer Science/Engineering or equivalent industry experience LOCATION: Austin, Tx or Santa Clara, Ca strongly preferred; Remote is a possibility for the right candidate #LI-MR1 At AMD, your base pay is one part of your total rewards package. Your base pay will depend on where your skills, qualifications, experience, and location fit into the hiring range for the position. You may be eligible for incentives based upon your role such as either an annual bonus or sales incentive. Many AMD employees have the opportunity to own shares of AMD stock, as well as a discount when purchasing AMD stock if voluntarily participating in AMD's Employee Stock Purchase Plan. You'll also be eligible for competitive benefits described in more detail here. AMD does not accept unsolicited resumes from headhunters, recruitment agencies, or fee-based recruitment services. AMD and its subsidiaries are equal opportunity, inclusive employers and will consider all applicants without regard to age, ancestry, color, marital status, medical condition, mental or physical disability, national origin, race, religion, political and/or third-party affiliation, sex, pregnancy, sexual orientation, gender identity, military or veteran status, or any other characteristic protected by law. We encourage applications from all qualified candidates and will accommodate applicants' needs under the respective laws throughout all stages of the recruitment and selection process.