Skip to content
Six Figure Jobs
Modular logo

Modular

AI Framework Engineer

  • Remote · United States
  • Posted 2d ago
  • Full-time
  • 6+ yrs experience

Salary

$120K - $180K

Listed in the posting

Pays more than 11% of 242 ai engineer listings in USD.

This job
$120k–$180k
Typical
$204k
Most pay
$169k–$240k
Apply on company site

Benefits

  • Generous PTO
  • 401(k) match
  • Retirement plan
  • Unlimited PTO
  • Health insurance
  • Home office stipend

Skills

AI
API
API Design
Cloud Computing
C++
Developer Experience
Jax
Observability
Python
PyTorch

Nice to have

Full job description

Description

  • At Modular, we are building a next generation AI platform to power modern applications and facilitate access to cutting-edge hardware
  • The MAX Framework is our developer-facing layer: it defines the APIs developers use to express models, integrate custom kernels, orchestrate execution, iterate on quality and ship systems into production
  • As an Senior AI Framework Engineer for MAX, you will own and evolve core APIs and developer experience for inference and training of AI models
  • You will work at the intersection of API design, systems engineering, and modern AI frameworks
  • Your output will be the specifications, abstractions, and reference implementations that make MAX feel coherent, powerful, and intuitive to use — while preserving performance, portability, maintainability and TCO
  • Influence the design of the API surface for MAX: namespaces, core abstractions, extension points, programming model, compatibility guarantees and developer experience
  • Design inference APIs that support real-world serving needs: model loading, distributed inference, quantization, tokenization/pipelines, configuration surfaces, batching/streaming, and deployment-oriented ergonomics
  • Design training APIs that scale from single device to distributed execution, with clear primitives for device placement, parallelism, checkpointing, and observability
  • Create a coherent programming model across Python and Mojo-adjacent surfaces: align naming, types, and conventions; avoid leaky abstractions; define the “pit of success”
  • Drive RFCs and technical specs: write and socialize proposals; gather feedback from internal model engineers and external users; iterate towards consensus
  • Partner cross-functionally with compiler/runtime, kernels, cloud/serving, and documentation/DevRel teams to ensure APIs map cleanly to underlying capabilities
  • Build reference implementations and exemplar code: golden-path examples, architecture templates, and best-practice patterns that teams can copy
  • Set quality bars for APIs: versioning policy, deprecation strategy, test strategy, and documentation requirements

Benefits

  • A variety of fantastic health benefits (health, dental, vision insurance; life insurance etc) are available
  • A 401k plan with up to 5% match
  • Free tax advice on Carta
  • Generous work-from-home stipend of $1500 to help you improve your home office
  • Unlimited paid time off and flexible work hours
  • PhD in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 4+ years of Software Engineering or related work experience
  • Bachelor’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 6+ years of Software Engineering or related work experience. OR
  • 3+ years of work experience with Programming Language such as Python, Mojo, Rust, C++, etc
  • Master’s degree in Engineering, Information Systems, Computer Engineering, Electrical Engineering, Computer Science, or related field and 5+ years of Software Engineering or related work experience. OR
  • Experience with inference and training systems (model execution graphs, compilation, runtime scheduling, distributed execution, checkpointing, performance tuning)
  • Strong understanding of modern AI frameworks and their design tradeoffs (e.g., PyTorch, JAX, TensorFlow, vLLM, XLA/MLIR-adjacent ecosystems)
  • Strong written communication: you can write specs that engineers can implement without ambiguity
  • Significant experience designing and evolving developer-facing APIs (SDKs, frameworks, or platforms)
  • High engineering standards, pragmatism, and a bias towards incremental development without compromising long term design
  • Fluency in one or more systems / performance languages (C++, Rust, Go) and one or more user-facing languages (Python; familiarity with Mojo is a plus)
  • Excellent taste for API ergonomics: naming, composability, types, error handling, configurability, and clarity
  • For candidates who fall outside of the listed requirements, we nevertheless encourage you to apply, as we may have openings at lower or higher levels than the one advertised
  • Onboarding for new hires is conducted in-person in our Los Altos, CA office

Apply on company site

Opens Modular’s own application page in a new tab.

Similar six-figure roles

Browse all Ai Engineer jobs →
NEW
IT Recruitment Solutions logo
IT Recruitment Solutions

AI Engineer

$350k - $500k
Salary shown
New York City Metropolitan Area, USA

As a Senior AI Engineer, you'll build the conversational intelligence behind a product that understands user preferences...

Posted 1d ago
NEW
ServiceNow logo
ServiceNow

Principal AI Platform Engineer

$221k - $387k
Salary shown
USARemotePrincipal

As the most senior technical member of the group, you set the solution architecture and deployment patterns the FDE team...

Agentic AIAIAutomationCpqCRM+3 more
Posted 2d ago

Get jobs like this one by email

AI Engineer jobs · USA

Only new matching jobs, nothing else. Confirm by email; unsubscribe in one click.

Explore more

Salary

$120K - $180K

Apply