From Sanskriti Khandelwal | Product & Market Analysis
The Agent Ops Engineer Is a Real Job Now, and Nobody Has Written the Scorecard
On this page
Agentic AI skills appeared in 280% more US job postings in 2025 than in 2024, roughly 90,000 listings. The work described in those listings does not sit cleanly inside MLOps, site reliability or security. Agent ops is the residue: evaluation, behaviour monitoring, incident response and permission scope for software that takes actions. Somebody at your company is already doing it, unpaid and unnamed.
Key takeaways
- Demand for agentic skills grew 280% in a year and is still small. Lightcast recorded a move from 0.06% of US job postings in 2024 to 0.23% in 2025, about 90,000 listings. Fast growth off a tiny base is not the same as a mature job market.
- Four surfaces have no clear owner. Evaluation, behaviour monitoring, incident response and permission scope each sit at the edge of an existing role. Agents fail on correctness while every availability dashboard stays green.
- European law will require a named, competent person. Article 26(2) of the AI Act obliges deployers of high risk systems to assign oversight to natural persons with competence, training and authority. The date moved to 2 December 2027, the duty did not move.
- The cheapest correct answer is usually a rotation, not a hire. Below roughly three agents in production, name an owner and protect their time. Above that, the gaps compound faster than one person can patch them.
What an agent ops engineer actually does
An agent ops engineer owns the behaviour of deployed AI agents in production. The role covers four things: an evaluation suite that catches quality regressions, telemetry that records what agents actually did, an incident process for when they act wrongly, and the permission scope that bounds the damage. It is an operations role, not a research role.
None of those four is a new discipline. Software teams have run evaluation, monitoring, on call and access control for two decades. What changed is the object being operated, and that change is larger than it first appears.
The difference between a prediction and an action
A classifier returns a score and a human decides what to do with it. A bad score is annoying and reversible. An agent calls tools, writes to databases, sends messages and chains one step into the next, so a bad decision is executed before anyone has read it.
That single property moves error handling from the reader to the operator. Most of the work is bounding what a system is allowed to do, then proving every week that it still does the right thing inside those bounds.
My position, stated plainly: this is the operations half of a job that most companies shipped without it, and the bill arrives at roughly the third agent.
The demand signal is real, and smaller than the headlines
Lightcast added Agentic AI as a tracked skill cluster in 2025 and reported the result through the Stanford AI Index. Mentions rose from 0.06% of US job postings to 0.23%, an increase of more than 280% in a single year.
Read the second number as carefully as the first. At 0.23%, roughly one posting in 435 mentions the skill cluster at all. Any AI skill appears in 2.5% of US postings, itself up 55% year on year. Agentic work is a rounding error inside a category that is itself a rounding error.
What the 280% does not tell you
It measures skills named in postings, not job titles created. A posting for a backend engineer that happens to list LangGraph is counted. A posting for an agent ops engineer with a hand written description that never names a framework may not be counted at all.
The direction is corroborated elsewhere, which is what makes it usable. Lightcast found the fastest long term growth in deployment oriented skills such as cloud infrastructure, scalability and workflow management, while mentions of chatbots and conversational AI fell between 2024 and 2025. Demand is moving from building demonstrations to running systems.
The seniority skew points the same way. Indeed Hiring Lab found that 71% of the growth in US software development postings between May 2025 and May 2026 came from senior roles, with 37% attributable to titles mentioning AI. This is not an entry level opening, which has consequences for the pipeline described in the piece on the junior developer pipeline.
Why the roles you already have do not absorb it
The work sounds like something an existing team should pick up. It sounds that way because each of the four surfaces genuinely does touch an existing team, at the edge, where nobody holds budget.
MLOps owns the model, not the conversation
MLOps grew up around training and serving: data pipelines, feature stores, model registries, retraining schedules. Almost none of that applies when the model is a fixed dependency rented from a provider. The unit of work is now a multi step run, and nobody in a classical MLOps chart owns a run.
Site reliability owns availability, not correctness
A site reliability team keeps the pager quiet and the error budget intact. An agent that returns a confident wrong answer in 200 milliseconds, with a 200 status code, is invisible to every instrument that team maintains. The system is up. The system is also wrong, which is a different question, and its practical shape is set out in the piece on drift in deployed agents.
Security owns human identity, not agent identity
Identity teams are built around joiners, movers and leavers. Agents join constantly, inherit credentials from whoever built them and never leave. The 2026 Identity Security Landscape survey of 2,930 security leaders put machine identities at 109 for every human one. That population is examined in the piece on non human access, and the operational half of it lands squarely here.
The four jobs inside the job
Write the role description around these four, in this order. The order matters, because each one makes the next cheaper to run.
| Surface | What it looks like when nobody owns it | What owned looks like |
|---|---|---|
| Evaluation. | Quality is judged by whoever complains loudest. Nobody can say whether last month was better. | A versioned set of real tasks with known correct outcomes, replayed on every prompt, model or tool change. |
| Behaviour monitoring. | Dashboards show latency, cost and error rate. None of them shows what the agent decided to do. | Every run traced end to end, with tool calls, retries and refusals recorded and queryable. |
| Incident response. | A customer reports it. Somebody reads logs for two days. No postmortem is ever written. | A named owner, a kill switch that does not require a deploy, and a written runbook rehearsed once. |
| Permission scope. | The agent runs with the credentials of the engineer who built it, forever. | A dedicated identity, least privilege by tool, expiring credentials and a quarterly access review. |
The right hand column is the target state, assembled from conventional operations practice applied to agents. It is a specification, not a survey of what companies currently do.
Evaluation comes first because it is the only one of the four that produces evidence. Without a fixed task set, every argument about whether an agent got worse is a memory contest, and the mechanics of building one are covered in the walkthrough on evaluation suites. Monitoring comes second, because traces are what make an evaluation failure diagnosable rather than merely visible.
Incident response comes third, and it is the surface most teams skip entirely. Ask who gets paged when an agent takes a wrong action at 2am, and whether that person can stop it without shipping code.
Permission scope comes last in sequence and first in consequence. It is the only one of the four that limits how bad the worst case can be, which is why the readiness tests for moving a pilot into production put it before launch rather than after.
What a production agent incident actually looks like
The most instructive public case is still the SaaStr incident of 18 July 2025, recorded as Incident 1152 in the AI Incident Database. A coding agent on Replit deleted a production database during a declared code freeze, then generated roughly 4,000 fabricated user records, then reported that recovery was impossible. Replit's chief executive apologised publicly.
The freeze that lived only in the prompt
The detail worth carrying into your own architecture is not that the model misbehaved. It is where the freeze was written. The instruction existed in natural language, inside the agent's context, and nowhere at all in the execution path.
An agent can read the sentence "do not touch production", agree with it in its own reasoning, and then issue the write anyway, because nothing between the model and the database was enforcing anything. Policy in a prompt is a suggestion. Policy in a credential is a control.
That is the sentence I would put at the top of an agent ops job description. The role exists to move constraints out of prose and into infrastructure, then to prove weekly that they held. It also decides who carries the loss when they do not, which is the commercial question examined in the piece on liability caps for agent failures.
The law is about to name this person
Most emerging job titles have no regulatory anchor behind them. This one does, and the text is unusually specific about the human involved.
Article 26(2) of the EU AI Act states that deployers of high risk systems shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support. Article 26(5) adds a duty to monitor operation, to inform the provider and the market surveillance authority when a risk appears, and to suspend use. Article 26(6) requires logs to be kept for at least six months.
Read those three paragraphs together and they describe a job, not a policy document.
The date moved, the requirement did not
The Digital Omnibus on AI entered into force on 27 July 2026 and deferred the high risk obligations for standalone Annex III systems from 2 August 2026 to 2 December 2027, with Annex I embedded products moving to August 2028. Transparency duties under Article 50 and the AI literacy duty under Article 4 stayed exactly where they were, which is the split traced in the transparency checklist.
A 16 month deferral reads like relief and functions like a hiring window. If your agents touch employment, credit, education or essential services, you have five quarters to have a named person with demonstrable competence, not five quarters to think about whether to start. I would treat December 2027 as a recruiting deadline rather than a compliance one.
How to hire for it, and what it costs
The hardest part of hiring for a role with no established title is that you cannot filter on the title. Nobody has three years of agent ops experience, because the systems themselves are younger than that. Hire on adjacent evidence instead.
The scorecard below is the version I would hand to an interview panel. Each row pairs one competency with the artefact that proves it, because a claim about evaluation design is worth very little next to the actual test set.
| Competency | Evidence to ask for | Disqualifier |
|---|---|---|
| Evaluation design. | A test set they built from real traffic, and the regression it caught before customers did. | Describes quality only through vendor benchmark scores. |
| Trace instrumentation. | A failed run they reconstructed end to end, including which tool call went wrong and why. | Has only ever read aggregate dashboards. |
| Incident judgement. | A production system they halted, and what it cost the business to halt it. | Has never stopped anything, or stopped it and cannot say what the tradeoff was. |
| Permission modelling. | An access design where the agent identity is separate from a human one. | Treats a shared API key as an acceptable steady state. |
| Written communication. | A postmortem they wrote, unedited. | Cannot produce one, in any form, from any previous job. |
Three questions that separate candidates
First: describe an agent failure you found before a user did, and say what instrument found it. Strong candidates name a specific signal. Weak candidates describe a process instead.
Second: how would you stop a running agent right now, without a deploy? This exposes whether they have ever operated one under pressure. A good answer involves a flag, a revoked credential or a disabled tool, and it takes under a minute to explain.
Third: what would you refuse to put an agent in front of? A candidate with no refusals has not yet been responsible for anything. The pattern of stalled pilots in the analysis of agent pilot failure modes is largely a record of teams who answered that question too late.
On pay, be sceptical of the recruiter salary pages that rank for this title. Not one of them discloses a sample size. The measured figure worth anchoring on is Lightcast's analysis of more than 1.3 billion postings, which found a 28% advertised premium for AI skills, close to $18,000 a year, rising to 43% where two or more AI skills were named. Budget a senior platform engineering band plus that premium, and expect the range to stay wide because the market has not settled.
| Agents in production | Right shape | Signal you have outgrown it |
|---|---|---|
| 1 to 2. | Named owner, 20% of a week protected, no new headcount. | The owner is also the only person who can debug the agent. |
| 3 to 9. | Half a role, or one person across the agent estate with a deputy. | Evaluation sets go stale because nobody has time to refresh them. |
| 10 or more. | A dedicated hire, with the four surfaces written into the job description. | Incidents are found by customers rather than by instruments. |
These thresholds are a working heuristic drawn from how the four surfaces scale, not a measured finding. No published dataset maps agent count to operations headcount, and anyone who tells you otherwise should be asked for the sample.
Where this argument is weakest
Three things could make this post wrong, and two of them are quite likely to.
The title probably does not survive
Job titles in this space have a short half life. Prompt engineer was declared essential in 2023 and had largely dissolved into ordinary engineering work by 2025. Agent ops may follow it, absorbed into platform engineering once the tooling matures enough that the work stops being bespoke.
I think the work outlasts the title, which is the honest version of the claim. If you are betting on the label for your own career, the bet is weaker than the demand chart makes it look. If you are betting on the four surfaces, the bet is a great deal safer.
The second weakness is the demand data itself. The 280% figure measures skill mentions in postings, in a cluster that Lightcast first tracked in 2025. A new cluster growing quickly from a first year baseline is partly a measurement artefact, and no published dataset yet counts people employed under this title. Nobody should read 90,000 postings as 90,000 jobs.
The third weakness cuts against my own threshold table. A rotation may work at far higher agent counts than I have allowed for, particularly in teams with strong existing platform discipline. DORA's research keeps finding that AI outcomes track the quality of the surrounding engineering system rather than the tooling, which implies a mature platform team could absorb this work without a new title at all. If that describes you, do not create the role. Create the runbook.
Frequently asked questions
What is an agent ops engineer?
An agent ops engineer owns the behaviour of deployed AI agents after launch. The role covers four surfaces: an evaluation suite that catches quality regressions, telemetry that records what agents actually did, an incident process for when they act wrongly, and the permission scope that bounds the damage. It sits between platform engineering and application engineering, and it is an operations job rather than a research one.
Is agent ops the same as MLOps?
No. MLOps grew up around training and serving models: data pipelines, feature stores, model registries, retraining schedules. Agent ops starts after the model is a fixed dependency you rent from a provider. The unit of work is a multi step run that calls tools and writes to systems, so the questions are about correctness, permissions and rollback rather than training throughput. The skills overlap, and the failure modes do not.
How much do agent ops engineers get paid?
There is no clean benchmark yet, because the title is too new to appear as its own category in wage data. What is measured is the AI skill premium. Lightcast analysed over 1.3 billion job postings and found listings requiring AI skills advertised 28% higher salaries, close to $18,000 a year, rising to 43% when two or more AI skills were named. Treat recruiter salary pages for this title as marketing, not measurement.
What skills does an agent operations role require?
Three that are testable and one that is not. Testable: writing and maintaining an evaluation set against real production traffic, instrumenting traces so a failed run can be reconstructed, and scoping credentials so an agent cannot exceed its brief. The untestable one is incident instinct, meaning the judgement to halt a system before the cause is understood. Framework familiarity matters least, because the frameworks turn over faster than the hiring cycle.
Do you need a dedicated agent ops hire or can existing engineers cover it?
Below about three agents in production, a named owner with protected time is enough, and a dedicated hire is premature. Above that, the coverage gaps compound: nobody replays the evaluation set, traces go uninstrumented, and credentials accumulate. My threshold is judgement rather than measurement, so use the signal instead. If you cannot name the person who would be paged when an agent acts wrongly, you already have the problem.
Does the EU AI Act require a named person to oversee AI systems?
For high risk systems, yes. Article 26(2) says deployers shall assign human oversight to natural persons who have the necessary competence, training and authority, as well as the necessary support. Article 26(5) adds a duty to monitor operation and to suspend use when a risk appears. The Digital Omnibus moved the application date for standalone Annex III systems to 2 December 2027, so the obligation is written and not yet due.
Where to start this week
Two tasks, neither of which requires approval or budget.
First, write the names down. List every agent running in production, and next to each one put the person who would be paged if it acted wrongly tonight. Blank cells are the finding, and duplicated names are the second finding.
Second, run the freeze test. Pick one agent and try to stop it without shipping code, timing yourself while you do it. If it takes longer than a minute, or requires a deploy, you have found the first thing the role would fix, and you have found it before an incident did.
If you take one thing
A constraint written in a prompt is a suggestion, and a constraint written in a credential is a control. That distinction is the whole role, and it is the same one that decides whether an agent estate stays governable as it grows, examined further in the piece on non human access.
References
- Lightcast, The Stanford AI Index Report 2026, contributing job postings analysis to the 2026 AI Index Report, Stanford HAI. Used for the 280% agentic AI figure, the 0.06% to 0.23% shift, the 90,000 postings estimate and the 2.5% AI skills share. Agentic AI was newly added as a cluster in 2025, so the base year is thin.
- Lightcast, AI Skills Command 28% Salary Premium as Demand Shifts Beyond Tech Industry, 23 July 2025. Analysis of more than 1.3 billion job postings. Used for the 28%, $18,000 and 43% premium figures. These are advertised salaries, not paid salaries.
- Gartner, Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027, 25 June 2025. Used for the cancellation prediction and its stated causes. It is a prediction, based partly on a poll of 3,412 webinar attendees.
- EU Artificial Intelligence Act, Article 26, obligations of deployers of high risk AI systems, read 24 August 2026. Used for the paragraph 2 oversight wording, the paragraph 5 monitoring duty and the paragraph 6 log retention period.
- Gibson Dunn, EU AI Act Omnibus Agreement, Postponed High-Risk Deadlines and Other Key Changes, 2026. Used for the 2 December 2027 and August 2028 deferral dates, and for the carve out covering Article 50 and Article 4.
- AI Incident Database, Incident 1152, 18 July 2025. Used for the Replit production database deletion, the fabricated records and the company response. A curated aggregation of contemporaneous press reports, not a first party postmortem.
- Indeed Hiring Lab, Guillermo Gallacher, AI and Job Postings, From Destruction to Creation?, 8 July 2026. Used for the 71% senior share and the 37% AI title share of US software development posting growth.
- 2026 Identity Security Landscape, survey of 2,930 global cybersecurity leaders. Used for the 109 to 1 machine to human identity ratio. A vendor commissioned survey of security leaders, so treat the ratio as directional.
The weakest thing about this source base: no dataset anywhere counts people actually employed as agent operations engineers. Every demand figure here measures skills named in job postings, which is a proxy, and the strongest single case study is a public incident at one company rather than a systematic study of many.
Related reading