Skip to content
Six Figure Jobs
ServiceNow logo

ServiceNow

Senior Staff Software Engineer (AI/ML Inference Engineering)

  • Santa Clara, California, United States
  • Posted 3h ago
  • Full-time
  • Staff / Principal
  • 8+ yrs experience

Salary

$201K - $352K

Listed in the posting

Pays more than 91% of 1,284 software engineer listings in USD.

This job
$201k–$352k
Typical
$200k
Most pay
$168k–$234k
Apply on company site

Benefits

  • Generous PTO
  • Retirement plan

Skills

AI
API
API Gateway
CI/CD
Containerization
Design Patterns
Distributed Systems
Envoy
GitHub
GitHub Copilot

Nice to have

Full job description

Description

  • We are seeking a Senior Staff Software Engineer - AI/ML Inference Engineering to lead the architecture, optimization, and operational excellence of our AI inference platform
  • You will own the end-to-end inference lifecycle—from onboarding cutting-edge models to deploying optimized inference engines in production across multiple backends on Kubernetes infrastructure
  • You’ll also architect the intelligent gateway layer using Envoy and ext_proc to enforce access control, provide observability, and enable advanced request routing
  • This role combines deep technical expertise in GPU compute, LLM APIs, inference frameworks, containerization, orchestration, and API gateway architecture with a bias toward pragmatic engineering and measurable performance impact

Benefits

  • Generous family leave
  • Matched donations
  • Annual learning stipends
  • Flexible PTO
  • Competitive retirement plan
  • Paid volunteer time
  • Knowledge of unit testing, profiling, and code tuning
  • Experience building new products that use challenging algorithms
  • Knowledge of core AI/ML techniques and algorithms
  • Expertise in Go or Python, OOP, Design Patterns, time and space-efficient algorithms
  • Expertise in coding efficient, object-oriented, modularized and quality software
  • Exposure to LLM APIs or ML-serving systems, or a clear ability to ramp up quickly
  • Model-serving and runtime experience, such as vLLM, TensorRT-LLM, Triton Inference Server
  • Artificial intelligence strategy is a critical skill at Experienced proficiency for IC5 — it applies to how you build and to what you build
  • Confident use of AI coding assistants such as GitHub Copilot or Cursor, paired with the critical eye to review AI-generated code as rigorously as any other — catching logic errors, security anti-patterns, and missed edge cases rather than treating model output as production-ready
  • AI-assisted debugging and trace analysis, and a clear understanding of the data privacy rules governing what goes into a prompt
  • A master’s degree may offset up to one year of the experience minimum. Equivalent practical experience is considered where technical depth is clearly demonstrable
  • Bachelor’s degree in computer science, software engineering, or a closely related technical field required
  • 8+ years of Backend software engineering experience
  • 4+ years of designing and operating distributed systems
  • Debug and resolve production incidents independently, participate in on-call, run root-cause analysis, and operate against defined service level objectives
  • Own observability for what you build — meaningful metrics, useful logs, real tracing, and alerts that fire on customer impact rather than on noise
  • Deploy, scale, and operate services in production Kubernetes environments, including Helm-based deployments and CI/CD pipelines that make releases repeatable and safe to roll back

Apply on company site

Opens ServiceNow’s own application page in a new tab.

Veeam logo
Veeam

Senior Software Engineer

$235k - $436k
Salary shown
USARemoteSenior

We are looking for a Software Developer to contribute to our Veeam Kasten data management platform , purpose-built for K...

Posted 10d ago

Get jobs like this one by email

Software Engineer jobs · USA

Only new matching jobs, nothing else. Confirm by email; unsubscribe in one click.

Explore more

Salary

$201K - $352K

Apply