Benefits
- Generous PTO
- Retirement plan
Skills
Nice to have
Full job description
Description
- Design & Architect Data Infrastructure — Design and oversee deployment of data architecture and pipelines to capture, manage, and store structured and unstructured data from internal and external sources; establish processes and data flows using cloud, local databases, and applicable storage forms
- Build & Automate Data Transformation — Develop technical tools using ML and data-engineering techniques to cleanse, organize, and transform data; implement automated processes to maintain data structure integrity and quality standards on an ongoing basis
- Define Agentic Evaluation Metrics & Ground Truth — Partner with product and AI teams to define evaluation metrics for agentic workflows (task completeness, tool use, workflow success); establish ground truth labeling standards and validation criteria; design evaluation datasets that reflect real-world agent execution
- Build Agentic Evaluation Pipelines — Design and implement automated evaluation infrastructure that measures AI agent performance using Now LLM Service models and execution logs; create dashboards, reporting, versioning, and reproducibility of evaluation datasets and results
- Establish Standards & Continuous Improvement — Create design standards and quality assurance processes for data systems; define quality gates and validation frameworks; analyze workflow performance and recommend optimizations to accommodate evolving CRM AI requirements
- Lead Cross-Functional Collaboration — Collaborate with product, engineering, and data science teams; mentor junior engineers on data engineering and evaluation design; generalize evaluation patterns and metrics across Agentic AI products
Benefits
- Generous family leave
- Matched donations
- Annual learning stipends
- Flexible PTO
- Competitive retirement plan
- Paid volunteer time
- Familiarity with AI/ML evaluation frameworks, LLM-based systems, or agentic workflows; ability to instrument and evaluate complex AI systems; understanding of model performance tracking
- Experience on ServiceNow platform is good to have
- Ability to design scalable, secure data architectures; experience with cloud platforms (GCP, AWS), APIs, event-driven systems, and data governance patterns
- 9+ years designing and building production data pipelines, data warehousing, ETL systems; strong expertise in SQL, Python, Javascript or equivalent, and distributed data processing
- 3+ years working on ML systems, model validation infrastructure, automated testing, or AI-powered applications; hands-on experience with data quality, feature engineering, and model monitoring
- Comfortable driving technical conversations across multiple teams; takes end-to-end ownership; works autonomously; mentors peers on data engineering standards and best practices
- Proven track record deploying, monitoring, and maintaining large-scale data systems in production; experience troubleshooting data quality issues, latency, and system reliability
Opens ServiceNow’s own application page in a new tab.