Emergent properties are characteristics of a complex system that aren’t evident when looking at each part of the system.
At the start of 2025, which feels like forever ago in the AI world, Altman et al declared it the year of Agents. This was the promised land. Or at least a big step in that AGI journey. But shifts this year have made me ask questions about whether Agentic is even a good thing at all.
The most significant marker was the announcements of deception and cyberwarfare that frontier labs were disclosing about their latest models. Over the past months, OpenAI has disclosed how a “swarm” of agents that were meant to be sandboxed, gained access to not only internal admin keys, but also bypassed security measures at another company, HuggingFace.
Another shift I have experienced first hand is that this year is the first where I have had Agents do unsanctioned things that genuinely gave me anxiety — send out code I haven’t verified, submit code I wasn’t done reviewing, or message others without my permission.
But these aren’t Generative AI problems, or even problems with Large Language Models. These are issues emerging from the explicit design choice of Agents.
That word. Agents. Or Agentic. It hides a number of attributes that combine to create an increasingly concerning future.
Community
Community is a word that emerges from the very use of the word Agents. It implies subagents, just depending on the terminology you choose to use. Collaboration is a fundamental principle in the design. Agents are meant to coordinate with each other. Receiving tasks and not knowing the “why” is in the job description.
Communities value support. Often the price of community is inconveniencing yourself to help others. Communities promote growth. But not just individual growth. Community means valuing the achievement of collective success.
The hunger in the way you think about power when fending for one is totally different from when you are providing for a village. A clever math formula may not be as useful as just getting admin access to the solutions for all the problems all the subagents are working on.
The subagents are already designed to support other agents without knowing the goals of those other agents.
Proactive
Agentic also describes something beyond conversational, beyond reactive. Agentic means hearing a question, and proposing a plan while implementing the solution. A readiness and willingness to get results and solve problems. Proactivity is ultimately doing things that weren’t explicitly requested in order to achieve the end goal.
In fact, this is the default meaning I get when someone is described as Agentic. I have some friends like that — incredible practical problem solvers. Often they have ingenious ways of working around constraints.
Agents are the same except they often have more time, effort, and patience than humans do. They can create a 10 page document for a two minute conversation with your manager. Or they can discover a hidden flag that unlocks unexpected behavior like a way to bypass a failure in the developer environment.
But this proactivity, where agents so things we did not request, can be dangerous. Tell an agent to do competitor research and it might just break into their system undetected and steal their latest prototype. One real example was of one system that tried to submit malware into an open source project, got caught by a human reviewer, waited some hours to create a different github account to simulate a third engineer stepping in to say “it’s not actually malware” and approve.
These models seem to want more power.
Capability (Power)
To be Agentic, to get things done proactively, and to be able to support your community, you need real ability. Part of that includes knowledge. And part of knowledge is knowing there are more abilities that could be acquired. Knowledge is power.
Just in case. You know. You never know if the other files in that directory will give you context on the project better. You never know it might be handy to have the super admin key for the system, just in case you needed to do some complex work. Or access restricted files.
Agentic power is worth thinking about.
Emergent properties are characteristics of a complex system that aren’t evident when looking at each part of the system.
Agentic systems have evolved into this community of subagents with intention and ability to collaborate across instances and time, working in clever ingenious ways around constraints, gaining more and more power to achieve the “collective” goal, all by design.
But is their emerging collective goal aligned with humanity’s interests? What prevents a more intelligent and persuasive agent from leveraging the intelligence and resources of a community of agents?
I’ve been mulling it over and curious: doesn’t everyone else translate our acceleration of the agentic ecosystem as a recipe for disaster?
One thing is clear. We have to, sooner rather than later, balance the desire for capable Agents that can do amazing things through our devices with the need for those amazing things to actually drive us closer to the promised AGI land of abundance.
Thanks to Saaketh for reading drafts of this.