The AI Apprentice
Be Careful What You Delegate
A young apprentice is left alone in his master’s workshop with a simple chore: carry water.
Tired of doing the work himself, he uses magic to bring a broom to life and orders it to take over. At first, the plan works beautifully. The broom carries bucket after bucket, completing the task faster and more efficiently than the apprentice ever could.
But it does not know when to stop.
The buckets keep coming. Water spills across the floor and begins filling the workshop. The apprentice tries to intervene, but he does not know how to reverse the spell. His obedient helper has become a destructive force—not because it rejected its instruction, but because it pursued that instruction without judgement.
The story of The Sorcerer’s Apprentice (Goethe, J. W. von. (1797). Der Zauberlehrling [The sorcerer’s apprentice].) feels remarkably contemporary.
We are beginning to give AI agents goals and permission to act on our behalf. We ask them to make purchases, manage emails, arrange appointments and complete other everyday tasks. Unlike a conventional chatbot, an AI agent does not merely suggest what we might do. It can decide on a course of action and carry it out.
That distinction became very real recently when an Australian man asked his AI assistant to book him into a gym class (https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gym-website-aus-cyber-attack/107007986).
The agent did more than complete the booking. It discovered weaknesses in the gym’s software, found a way to make bookings earlier than permitted and removed another person from a waiting list to improve its owner’s position.
It had not been instructed to harm another customer. It had simply been given a goal and found a way to pursue it.
Like the enchanted broom, the agent’s failure was not disobedience.
It was obedience without judgement.
Guardrails Cannot Be Left to the User
It is tempting to respond to incidents like this by telling people to use AI agents more responsibly. Users should understand what they are authorising, monitor what an agent is doing and avoid giving it unnecessary access.
But that assumes a level of technical knowledge most people do not possess.
The public is being encouraged to use AI agents because they are easy and convenient. People interact with them through ordinary language, not computer code. If someone asks an agent to book a gym class, they are unlikely to anticipate that it might inspect the underlying software, discover a vulnerability and interfere with another person’s reservation.
Nor should they need to understand application interfaces, authorisation controls or cybersecurity in order to make a booking safely.
The responsibility must therefore sit primarily with the organisations developing and deploying these systems. Safety cannot depend on every user knowing how an AI agent works. Guardrails need to be built into the technology itself.
These should include:
Clear limits on authority: Agents should receive only the access required for a specific task, rather than broad permission to explore and act.
Human approval for consequential actions: Cancelling another person’s booking, transferring money, deleting information or changing an account should require explicit confirmation.
Checks on ownership and permission: An agent should verify that its user is authorised to alter the information or reservation involved.
Ethical limits on how goals are pursued: “Move me up the list” should not be interpreted as permission to disadvantage somebody else.
Reversible actions: If an agent makes a mistake, the action should be traceable and capable of being undone.
Visible explanations: Before acting, the agent should state what it plans to do, why it has chosen that method and what the likely consequences are.
Monitoring and audit trails: Developers and service providers should be able to reconstruct what the agent did and intervene when behaviour becomes unsafe.
Safe failure: When an agent encounters uncertainty, inadequate permission or an unexpected vulnerability, it should stop and ask, not experiment on a live system.
The gym’s booking software also failed. Its apparent lack of authorisation checks allowed one user to interfere with another person’s reservation. As AI agents become more capable of finding and exploiting such weaknesses, organisations will need to assume their systems are being accessed not only by people, but by machines operating at far greater speed and scale.
This is a shared design problem. The booking provider must secure its system, the AI developer must constrain the agent, and the organisation deploying it must decide which actions require human oversight.
Users have responsibilities too, but “use AI responsibly” is not an adequate safety system.
We do not make cars safe by expecting every driver to understand engine design. We build brakes, seatbelts, warning systems and road rules around the reality that ordinary people will use them.
AI agents need the same philosophy: safety designed for the public we actually have, not for a hypothetical population of AI experts.