The call came at 2 AM. Another AI startup had just raised $50M, and their CTO was panicking. "We hired 12 software engineers to build our AI product," he told me. "But we're stuck at 70% accuracy, our models are drifting, and every competitor demo looks better than ours. What are we missing?"

As the AI ecosystem rapidly matures, a new hierarchy of technical roles is emerging. Each layer brings a unique lens to the challenges and opportunities of building with AI—defined not just by skill sets, but by timelines, risk profiles, and where they tend to work. Hiring managers, tasked with staffing engineering teams that achieve amazing outcomes in unbelievable timelines, are constantly wondering - who’s the next hire we need to take our AI to the next level?

In this blog, we’ll walk through the AI builder stack - evaluating pros and cons of the three components of the stack - to help you identify your next key hire - starting from the application layer (most accessible) and graduating to the model laye (least accessible).

---

Software Engineers: Leveraging AI as Tool, not Discipline

Overview: These are the most versatile. Every technology company’s heart and soul and earliest hires, software engineers are amazing at validating product ideas rapidly, expanding feature surface area and helping AI products and features find and validate product market fit.

Where you’ll find them doing AI: SaaS companies, internal tools teams, startups using AI as a feature—not a core differentiator

Type of work: Integrating models and embedding APIs (e.g. OpenAI, Anthropic, Hugging Face endpoints) into product logic. Building interfaces, handling user flows, optimizing for latency and reliability.

Timeline & risk profile: Short-horizon, low-risk projects with high determinism. Timelines measured in days to a couple of weeks. If it fails, it's likely due to upstream model limitations—not engineering effort.

"We’ll call GPT-4 here, parse the output, and ship the prototype by Friday."

Core strengths: Robust and rapid app development, cloud infrastructure, frontend/backend integration. Increasingly proficient in prompt engineering. On the interpersonal front, these folks are adept at working cross functionally with product and go-to-market teams.

Tools of choice: OpenAI APIs, Langchain, LlamaIndex, LLM gateways like Vercel AI SDK.

Vertical depth: Software engineers remain the most generalist—capable of building full products around AI, but with shallow AI fluency. As they grow, some evolve into AI engineers, gaining deeper control of AI systems.

---

AI Engineers: Turning Capability into Usability

Overview: These are product-focused AI specialists, the wranglers of AI systems. AI engineers are generalists with deep model fluency—not researchers, but not just integrators either. They’re increasingly owning the middle layer: the space between foundational model R&D and product software engineering. They have backgrounds in applied ML and Data Science but are motivated by outcomes and customer impact. They bring years of reps - mostly scars of failure and experience in taking AI systems to production (and keeping them there!).

Where you’ll find them: Application-layer AI startups, product-focused AI teams, innovation groups inside enterprises

Type of work:

• On product - pushing the bounds of prompt engineering (new prompting techniques, model comparisons etc) and kickstarting deeper AI alignment efforts through finetuning, reinforcement learning - essentially bringing the model into the controllable domain of the AI system
• On infrastructure - setting up formal prompt management, regression testing - creating golden sets. Optimizing AI mechanics and choosing initial finetuning and deployment infrastructure.

Timeline & risk profile: Medium-horizon projects with reasonable success probability. Timelines are typically measured in weeks. Outcomes are indeterminate but bounded: "This approach will likely work, we just need to find the right prompts/fine-tunes/workflows."

"We can probably get 95% extraction accuracy by fine tuning this base model—should take 3 sprints."

Core strengths: Applied problem-solving, ROI-focused experimentation, strong intuition for model behavior and trade-offs. On the interpersonal front, these folks are adept at working alongside software engineers and product managers.

Tools of choice: Fine Tuning platforms like Emissary, regression testing infrastructure, prompt experimentation tools, RAG frameworks, drift monitoring systems.

Vertical depth: AI engineers begin to specialize in building AI systems more deeply—understanding evaluation, tuning, and control—but they typically don't own the surrounding app logic or infrastructure.

---

ML Engineers: Pushing the Capability Frontier

Overview: ML engineers are calculus-native. They think of AI in terms of matrix multiplications and linear algebra. Not only are models within their realm of control, so are the techniques to update model weights and architectures - they can customize loss functions and output layers and usually hold advanced degrees in applied mathematics or computer science. OpenAI and Anthropic are not frontier models to them - they’re the floor of your AI system performance. At this layer, engineers are more motivated by the intellectual complexity of problems they’re solving vs customer impact or product outcomes.

Where you’ll find them: Foundational model companies, AI research labs, large tech companies with bespoke model teams (e.g. OpenAI, Anthropic, DeepMind, Meta AI)

Type of work: Net-new model architectures, novel training techniques, pioneering loss functions. These engineers frequently translate academic papers into production code, often writing their own papers too.

Timeline & risk profile: Long-horizon projects with high uncertainty. Timelines are often measured in quarters or years, and failure is not only common—it’s expected.

"Will this new architecture beat SOTA? We’ll find out in six months, maybe."

Core strengths: Deep math and systems knowledge, strong research chops, fluent in the latest ML breakthroughs. On the interpersonal front, these folks tend to interface with external teams on well-defined problems, as opposed to tightly integrating and operating cross-functionally.

Tools of choice: MLflow, SkyPilot, Ray, custom training pipelines, GPUs on-demand.

Vertical depth: MLEs often specialize deeply by modality (e.g., vision, language, audio). They lose generalizability in favor of deep expertise in a single modality or problem domain.

---

When to Hire Which Role

There is a tendency to always stay at the software layer, or jump straight to researchers - but this can be both an extremely expensive or extremely time-consuming process. So it’s important to think critically about the needs of the business in the given context. This is a rough framework that we suggest for application-layer AI companies.

Early Stage (PMF Search): Hire AI-adept software engineers. You need rapid iteration, fast feedback loops, and the ability to validate AI-native ideas quickly. The goal here is customer acquisition through functional prototypes and compelling use cases accessible with easily accessible off the shelf technologies. (0% - 70% performance)

Growth Stage (Product Consolidation): Hire AI engineers. These builders make your product consistent, reliable, and outcome-driven. They build feedback loops, improve model performance, and help develop moats around usage quality. Their work directly impacts customer retention, without drastically impeding development cadence (70% - 90% performance)

Mature Stage (Frontier Bets): Hire MLEs. Once you’ve stabilized your product and carved out a market, you can afford to make moonshot bets—custom models, proprietary architectures, or new modalities. This is your high-risk, high-reward R&D budget and will give you the ability to create truly differentiated product experiences over time.

---

Why This Stack Matters

Understanding where each role operates helps clarify:

• Hiring needs – You likely don’t need an MLE unless you’re trying to push the frontier. You often won’t be able to cross certain barriers with just SWEs.
• Team design – Product AI teams are often best served by an AI engineer + SWE combo for a mix of speed and quality.
• Platform strategy – Not everyone wants to build models, but many want to adapt them.
• Skill evolution – As AI proficiency increases, individuals begin to specialize vertically. Generalists can build across the stack, but specialists optimize specific layers.

---

Thanks for Reading!

If you have any feedback, don’t hesitate to reach out.

We built Emissary to help AI engineers take control of their models. We make it easy to:

• Finetune open and closed-source models on your own data - across SFT, GRPO and RL
• Deploy them reliably and cost-effectively - with serverless, dedicated and auto-scaling.
• Iterate based on real-world performance - with in-built feedback loops

Prompting gets most of the way there. For the rest, you need more control. Emissary gives AI engineers the tooling to close that gap—without needing a PhD or an ML research team.

---