Principal Software Engineer, Applied AI
Job Description
About the Role
We are looking for a Principal Software Engineer to join our Applied AI team. This is a rare opportunity to work at the cutting edge of Generative AI and Agentic systems – applied directly to high-impact business problems at enterprise scale.
Highspot's AI challenges are uniquely interesting: our platform ingests millions of heterogeneous enterprise documents (slides, PDFs, spreadsheets, CRM records, analytics data), and our AI agents must reason across all of it to surface actionable insights for sales teams. You'll tackle problems spanning context engineering, multi-step agentic orchestration, retrieval and ranking over diverse content types, and evaluation of non-deterministic systems – all within the constraints of enterprise-grade reliability, multi-tenant security, and cost efficiency.
As the Principal Engineer on the Applied AI team, you will own the technical direction for Generative AI and Agentic solutions across Highspot. You'll set the architecture, ship production systems, and raise the AI engineering capability of the broader organization. You will work closely with machine learning experts, data scientists, product managers, and engineering leadership to drive high-impact business outcomes.
What You'll Do
-
Design and build agentic AI systems that orchestrate multi-step workflows across heterogeneous enterprise content – documents, CRM data, analytics, and more – to deliver contextual, actionable answers for sales teams.
-
Own the AI platform architecture – define the patterns, abstractions, and guardrails that enable multiple engineering teams to ship AI-powered features safely and independently.
-
Build evaluation and observability infrastructure for non-deterministic systems – including automated regression testing, LLM-as-judge pipelines, and production quality monitoring – so the team can ship AI features with confidence.
-
Drive model strategy and cost efficiency – evaluate and route across model providers, optimize context windows, and make principled latency/quality/cost tradeoffs at enterprise scale.
-
Raise the AI engineering bar across the organization – not just within your team, but across engineering. Coach engineers on applied AI best practices, build shared tooling, and cultivate a culture where AI fluency is widespread.
-
Drive operational excellence – ensure enterprise-grade reliability, security, and performance for all AI-powered features in production.
-
Communicate complex technical concepts clearly to both technical and non-technical audiences, and translate business needs into actionable engineering investments.
Your Background
-
Demonstrated depth in shipping production agentic AI systems – you've built multi-step, tool-using agents that operate on real data at scale.
-
Deep expertise in context engineering: retrieval pipelines, ranking, prompt orchestration, and the tradeoffs involved in assembling context for LLMs over large, diverse document corpora.
-
Hands-on experience building evaluation frameworks for non-deterministic AI systems – you understand why this is hard and have opinions on what works (synthetic evals, human-in-the-loop, LLM-as-judge, production monitoring).
-
Track record of designing systems that balance quality, latency, and cost – including experience with model selection, routing, and optimization across providers.
-
Strong instinct for platform thinking: you know how to build abstractions that accelerate other teams without over-constraining them.
-
8+ years of professional software engineering experience, with significant time spent on distributed, data-intensive production systems.
-
Strong programming skills in Python, Java, TypeScript, or equivalent. You're comfortable across the stack.
-
Experience developing and operating cloud services at enterprise scale (AWS, Azure, or GCP).
-
A track record of full product lifecycle ownership – from technical design through iterative shipping to production operation.
-
Experience mentoring engineers and building collaborative, high-performing teams.
-
Strong cross-functional collaboration skills — you work effectively with ML researchers, data scientists, product managers, and business stakeholders with different backgrounds and priorities.
What will set you apart:
Foundations: