When Agent Skills Evolve: The Need for a Layer of Control
What we learnt from building Rhizomatic, a human-in-the-loop proof of concept for reviewing, versioning and rolling back changes to reusable agent skills.
Agent skills are becoming a practical way to package the knowledge an agent needs to perform a task consistently. Under the open Agent Skills specification, a skill is a folder with a required SKILL.md file containing its metadata and instructions. It can also include scripts, reference material, templates and other resources. Agents load these skills through progressive disclosure: they first see a compact catalogue, load the full instructions when a skill is activated, and retrieve supporting resources only when needed.
Recent work with Hermes Agent, an open-source personal agent runtime and harness, shows how agent skill interaction can go further: skills can evolve over time based on real interactions and external signals. Given enough autonomy, an agent can refine how it works, adapt its instructions, and improve the skills it relies on.
But autonomy without oversight creates a new risk. As skills change, quality can drift, instructions can diverge from their original intent, and outputs can become less reliable. If agents are going to evolve their own skills, they also need a layer of review and control to ensure that evolution remains useful, measurable, and safe.
Rhizomatic is our proof of concept for adding a control layer around how agent skills evolve. It treats the entire skill package as versionable, with our proof-of-concept focuses primarily on changes to SKILL.md.
Rhizomatic connects to local agent runtimes such as Claude Code and Codex through hooks, collecting execution traces and comparing them with the skills that were used. From those signals, a reviewer agent can propose changes such as updating an existing skill, deriving a new one, or combining several skills into a new capability. Crucially, those proposals remain separate from the active skill until a human reviews and approves them in the Rhizomatic portal.
Approved changes become the current skill version and can be pulled when a new agent session begins, while earlier versions remain available for rollback. The proof-of-concept does not yet prove that every proposed change is better, nor does it include automated regression testing, security analysis, or PII redaction. Its goal is deliberately narrower: to make skill evolution visible, reviewable, and reversible before it changes future agent behaviour.
Agentic skills evolution: is this the next wave?
Consider one of the demo traces loaded into Rhizomatic.
A user reports that asynchronous database queries fail intermittently when Kubernetes pods restart. The agent updates a session factory, but its first attempt is incomplete. The user explains that the database engine also needs stale-connection detection, an appropriate connection pool and retry handling with backoff.
The agent corrects the code, and the immediate problem is solved.
In a conventional session, that is where the lesson ends. The correction remains buried in the transcript, so another agent or the same agent a week later may repeat the mistake.
A learning agent could preserve that lesson by updating its database skill. Hermes Agent, one of the inspirations for this project, already describes a learning loop that creates skills from experience and improves them during use.
The question is therefore shifting from whether skills can evolve to how those changes should be governed.
Rhizomatic treats four actions as distinct:
- Use a skill.
- Observe what happened via transcripts and traces.
- Propose a change if required.
- Publish an accepted version (human)
The agent may participate in the first three. A human remains responsible for the fourth. This separation is the central idea behind our proof-of-concept, self-improvement should not have to mean silent self-modification.
Control how skills evolve with Rhizomatic
Rhizomatic sits between the runtime where work happens and the skill store from which future agents learn.
Local agent runtime
↓ hooks
Per-user traces and skill activation events
↓ review
Reviewer agent compares traces with current skills
↓ propose
Update, evolve or combine proposal
↓ human decision
Approved version and history
↓ hooks
Local agent runtime
From execution trace to proposal
A trace captures the conversation, tool calls, tool results and details of any skills invoked, linking an agent’s behaviour to the instructions that may have influenced it. The traces are sent through hooks and is stored in a database scoped to each signed-in user. After a user-defined number of traces has been collected, a reviewer agent compares recent interactions with the current skills and decides whether there is enough evidence to propose a change.
Rhizomatic can propose three types of change: Update, Evolve and Combine. Each represents a different modality - improving an existing skill, branching into a new one, or bringing several skills together into a new capability.
Update: improve the same skill
An update keeps the skill’s purpose and identity while creating a new version.
In the stock-screening demo, the existing skill describes filters, multi-factor ranking and alert generation. A user then corrects an inefficient implementation that iterates over rows and asks the agent to use vectorised pandas operations instead.
The trace suggests that the skill explains what to calculate but not how to implement it efficiently. Rhizomatic proposes an updated version with a new performance section. The portal presents a side-by-side diff, including the added vectorisation guidance, and links the proposal back to the user correction that motivated it.
A human decides whether the lesson belongs in the skill, whether its wording is sufficiently general and whether the scope is appropriate.
Evolve: derive a new branch
Sometimes repeated interactions no longer fit comfortably within the original skill.
In another demo, an async-db-patterns skill begins with guidance for asynchronous connections and Kubernetes-specific pooling. Over time, the associated conversations expand into migrations, indexing and query optimisation.
Adding all of that material to the original skill could turn a focused procedure into a broad catch-all. Rhizomatic can instead propose an evolution from async-db-patterns to a new database-engineering skill. The original remains available, and the new skill records where it came from.
Combine: turn recurring composition into a capability
The third proposal type, Combine, brings several existing skills together. In the demo, stock-screening and python-resilience are often used side by side to build automated trading pipelines. Rhizomatic therefore proposes a new stock-trading-system skill that combines signal generation with retries, circuit breakers and graceful degradation, while keeping the original skills available for use on their own.
Combining skills can be useful when they repeatedly form part of the same workflow. Instead of asking the agent to coordinate separate skills and rediscover the same sequence each time, a combined skill captures the full process in one place, reduces the risk of missing steps and gives the agent a clearer trigger for when to use it. However, this only makes sense when the skills support a coherent, well-defined task. Rhizomatic therefore treats repeated co-use as a reason to consider a combined skill, not as proof that one should be created.
Version control is necessary, but not sufficient
When a proposal is approved, Rhizomatic stores it as the current skill version while keeping earlier versions available for rollback. Compatible agent sessions can then retrieve the latest approved skills through the Rhizomatic API/ hooks.
The system is version-control-inspired rather than a full implementation of Git. It uses familiar concepts such as diffs, lineage and rollback, but adds the wider workflow that a repository does not provide on its own: observing agent activity, linking traces to the skills used, proposing evidence-backed changes, routing them for review and distributing approved versions back to the agent.
Human review is a gate, not an evaluation
Human approval prevents skills from changing automatically, but it does not prove that a proposed version is safer or more effective.
The current POC does not test proposals against held-out tasks, measure performance improvements or detect wider regressions. It also does not yet provide PII redaction, malicious-instruction detection or sandboxed testing of bundled scripts. Reviewers may still accept advice that is too specific or miss unintended effects, and rollback cannot reverse actions already taken by an agent.
Rhizomatic’s current contribution is therefore limited but important: it makes skill changes visible, reviewable and reversible. Systematic evaluation, privacy controls and security testing are the next layers needed to make those changes more trustworthy.
Make skill evolution visible, reviewable and reversible
Rhizomatic provides a controlled way to turn lessons from real interactions into candidate procedural knowledge. Instead of leaving a useful correction buried in a transcript, it gives reviewers the evidence they need—the conversation, tools used, active skill, proposed changes and version history—to decide whether that lesson should shape future behaviour.
Approved changes can be distributed to agent runtimes automatically, while earlier versions remain available if a change proves unhelpful. This is especially important when skills contain operational procedures, analytical methods or domain guidance that must remain stable and accountable.
The next step is to add stronger safeguards. Proposed skills should be tested against representative tasks, traces should be minimised and redacted, and security checks should detect poisoned feedback or unsafe code. Organisations may also need named skill owners, role-based approvals and limited rollouts before changes are widely adopted.
Rhizomatic is still a proof of concept. It does not settle how agent skills should evolve, but it provides a practical way to examine that process and decide where human control belongs.
If your team is exploring reusable agent skills or the governance of agent learning, reach out to the AI Assistants team for more information, or to indicate interest in exploring the application.