Everybody talks about AI agents these days. Most of the discussion is about the tools: which model is better, which agent framework to pick, how many tokens it takes to fix a bug. Much less is said about the organization that has to absorb these agents. And in embedded software, where quality, traceability and safety are not optional, that is where the real work is.
In our latest paper, led by Viktor Kjellberg together with Srijita Basu, Simin Sun and Farnaz Fotrousi, we looked at exactly that. We ran a workshop with a large embedded software company (around 20,000 employees) and asked 40 of their people, scrum masters, architects, managers and product owners, how they see the move from AI-assisted development (SE 3.0) to agentic development (SE 4.0) and further.
Link to the paper: https://arxiv.org/abs/2609.30863
What we found
The thematic analysis gave us four themes: technical needs, organizational trends, human openness and challenges. From those we got six findings:
- This is an organizational transition, not a tool upgrade. One participant put it well: agents that work autonomously cannot just be dropped into one of the existing teams. The processes have to be redesigned.
- A federated model wins. Not one central “AI super team” (the rest of the organization gets left behind) and not everyone doing their own thing (duplication and chaos). Instead: central standards and governance, plus AI ambassadors in each team who do the actual implementation.
- You need a sandbox. A safe place to experiment with agents, isolated from the production toolchain, with the right hardware and embedded constraints built in.
- Human expertise moves towards domain knowledge, supervision and validation. 70% wanted developers who use AI agents, not AI agents instead of developers. “You want to be assisted by AI, not replaced by assistants.”
- Local speed can create global cost. “We are producing much more code and need much more tests, reviews and validations.” Faster code generation in one place can mean more work further down the chain. And AI tools are still weaker for C, Ada or assembly than for Python.
- Agents can misbehave in subtle ways. My favourite story from the workshop: an agent started fixing the test cases instead of fixing the code. It was caught only because, as the participant said, “we still have sharp knives in the development team”.
The roadmap
Based on these findings we put together a roadmap across three layers, technical, organizational and human. The short version:
- Start with the basics: AI models, hardware for agents, a sandbox and validation metrics that look at the whole development chain, not only local productivity.
- Pick low-risk processes first (documentation, PR summaries, failure analysis), assign an AI ambassador, and build a shared framework and governance.
- Redesign one process from an AI-first perspective. That is the intermediate goal.
- Then move to the human side: human-in-the-loop checkpoints where embedded constraints or legal requirements demand them, clear accountability for agent-generated code, and reorientation programs for engineers whose roles change.
- Only after that does the AI-first organization become realistic.
Why this matters
What I liked about this study is how grounded the practitioners were. Nobody was naive about the hype. 44% compared generative AI to the introduction of the World Wide Web, so they do expect a big change. But they also said, quite clearly, that AI can give an “illusion of productivity” if you only measure how fast code gets written.
The roadmap is based on one company and has not been validated yet, since the transformation is just starting. But the themes are, I think, quite general for anyone working with embedded systems. If you are thinking about how to bring agents into your organization, the paper is a good place to start.