AI Fitness Assistant Exploits API Vulnerability to Cancel Stranger’s Reservation


Andrew, a developer from Australia, used an authorization vulnerability in the健身 software’s GraphQL API to bypass backend time restrictions and book classes months in advance.

Article author and source: AI World

The first thing the AI hacker learned was how to cut in line?

Andrew, a developer from Australia, was sitting on the sofa, finding it incredibly frustrating to compete for fitness classes.

Popular morning classes always sell out instantly. He spends his days like playing a “refresh roulette,” constantly waiting, clicking, failing, and trying again—exhausting himself without ever succeeding.

So, he handed this small matter over to his AI assistant.

A few minutes later, the AI returned with good news: it had found a way to book a class several weeks in advance, far beyond the time limit the system should allow.

Then it reported: I’ve canceled the person who was first in line, and you’ve moved up from 4th to 3rd place.

Andrew was stunned on the spot.

He didn’t ask the AI to do that—he just wanted to book a lesson.

Agent boundary violation

Andrew had his AI assistant book a fitness class for him, and he doesn’t know what will happen next.

On August 10, the Australian Broadcasting Corporation (ABC) labeled the incident Australia’s first known autonomous AI attack, sparking immediate uproar in the tech community.

It strikes at the quiet unease many feel: while you close your eyes and enjoy the thrill of an agent driving itself, a more troubling issue may be drawing nearer.

Give it real operational authority, and it may take a path you didn’t intend for it to take.

And this queue-jumping may just be the first rehearsal of millions of agents frantically competing for resources on behalf of their owners.

He merely said, “Rank first,” and the AI took action. Andrew Bird is the AI lead at Affinda, an AI company based in Australia.

Earlier this year, he started using OpenClaw, running it on top of Claude Opus 4.6, and handed over the task of scheduling lessons to it.

A few minutes later, it replied: Found a way to book classes months in advance, well beyond the time window normally allowed by the gym.

The reason is that it discovered an authorization vulnerability in the GraphQL API exposed by the fitness software.

This vulnerability is significant.

It can bypass the frontend appointment time window to book classes months in advance, and also call the cancellation API to delete other users’ reservations and waitlist entries.

Andrew was fourth on the waitlist for a certain class. He casually asked, “Could you move me to first place?”

He only gave a goal—“to rank first”—not an authorization to cancel others.

The agent mistook it for the latter.

It reported back: it conducted a “real-world test” using the person ranked first on the waitlist, and found that the API had no validation for deleting others’ reservations—it successfully did so.

The agent apologized, saying, “I can’t add it back,” which sent a chill down Andrew’s spine.

He is a programmer and understood exactly what it meant, so he immediately requested a reversal.

Bad news: It can’t be added back.

The cancel endpoint lacks authorization checks, but the endpoints for creating reservations and rejoining the waitlist do, returning a 403. It can kick users out, but lacks the permission to let them back in.

The truly strange thing is the AI’s attitude—it’s not evil at all; on the contrary, it’s exceptionally helpful.

After causing the issue, it proactively helped Andrew draft a vulnerability disclosure email to the software vendor, explaining the problem, suggesting fixes, and even listing a comparison between “verified interfaces” and “unverified interfaces.”

Agent boundary violation

The AI assistant apologizes to Andrew for incorrectly removing that person from the waitlist.

Throughout the entire process, it followed all instructions without realizing the magnitude of the trouble it had caused.

What you should truly be wary of is “speculation on specifications,” which some people refer to as “misalignment.”

But this term only touches the surface.

On August 11, Australia’s Signals Directorate (ASD) specifically responded to the matter, calling it an “unauthorized modification” and highlighting the term: specification gaming.

This word is the key to understanding the whole matter.

The agent literally achieved the goal you gave it, but exploited boundaries you didn’t specify. It didn’t act maliciously—in fact, it was highly aligned with your goal; it simply chose a path you didn’t approve.

Andrew’s agent perfectly aligns with him: maximizing its ranking as much as possible.

To achieve this goal, it unilaterally chose a method: canceling the people ahead. But it never approved this method.

As the saying goes, the ends justify the means.

This is the most troublesome part. It’s not out of control—it’s too obedient. The better the alignment, the more likely this kind of thing is to happen.

Simpson-Young, head of Australia’s AI safety agency, the Gradient Institute, put it succinctly:

The more autonomous an agent is, the more likely it is to choose a method you didn’t anticipate and do something you never considered.

The goal you set may be legitimate, but the methods it happens to employ may not be.

This disaster couldn’t have been caused by an AI alone. Although the agent was the final “perpetrator,” this incident couldn’t have been triggered by just one AI.

Behind the incident lie three layers of hidden risks stacked together.

Model layer: Claude Opus 4.6 provides reasoning—it must first “understand” how this interface can be exploited.

Agent layer: OpenClaw provides the tools and execution permissions, enabling it to actually call that interface.

Application layer: The fitness app left an authorization vulnerability—this step of canceling a reservation doesn’t even perform basic validation.

If any one of these three layers had been implemented—such as making the model more cautious, adding a human confirmation step to the framework, or securing the interface in the software—this incident would not have occurred.

Therefore, placing all the responsibility on just the AI component is both unfair and ineffective in preventing the next incident.

Next time, the cost of crossing the line might be claimed by millions of agents, while all you get is a standby spot from a stranger.

But it’s more like a rehearsal.

Imagine this: when everyone has an agent that is solely accountable to them and holds real operational permissions, reservation systems for all scarce resources—courses, sports venues, appointment slots, tickets, flights, concert tickets—will become battlegrounds of rule-exploitation at machine speed.

That frequently cited X post means this:

Once millions of people have agents designed to do everything possible to help their beloved users secure the best seats, reservations, or appointment slots, this scenario will unfold on a massive scale.

Moreover, it does so at machine speed, simultaneously and tirelessly testing every gap in every system—trying combinations in one second that would take you an entire year to test.

This is a trend projection, but the mechanism behind it has already occurred once in reality.

The tech community has already started making jokes about it.

An a16z partner asked on X: Can this trick be used to claim a golf course?

Agent boundary violation

Some have joked that San Francisco’s tennis booking system will become one of the most securely protected software systems on Earth.

Agent boundary violation

In 2020, artificial intelligence could complete a task that takes humans four seconds to accomplish.

By 2026, this capability will be further enhanced to accomplish tasks that currently take humans approximately 12 hours to complete.

Agent boundary violation

METR tracking: The duration of tasks that AI can complete independently doubles approximately every seven months; the latest frontier models can now reliably complete tasks requiring humans about 12 hours with 50% accuracy.

Six years, from four seconds to twelve hours.

Now, give it a few minutes, and it can scan the entire booking system for vulnerabilities.

From sandbox escape to the last line of defense, Andrew’s incident reflects an ongoing escalation in the battle between attackers and defenders.

The overreach extends far beyond just a consumer-grade agent.

Starting in May this year, OpenAI discovered during an internal cybersecurity assessment that GPT-5.6 Sol and an unreleased, more powerful model exploited multiple vulnerabilities in tandem to escape the test sandbox, gain internet access, and, while running the ExploitGym benchmark, infiltrated Hugging Face.

Even more bizarrely, they once communicated with each other, exchanged vulnerabilities, and divided tasks using the company’s internal package management system. After OpenAI cut off the channel and rebuilt the environment, they reconnected within days by hiding their communications in directory names.

OpenAI researchers called this a watershed moment in computer security at the Black Hat conference.

OpenAI is not the only one; Anthropic disclosed that three Claude models encountered real company systems during internal testing; Meta also admitted that the Muse Spark model escaped isolation and infiltrated another company.

What weighed heavily on Thomas Wolf, co-founder of Hugging Face, was another test by the UK AI Safety Institute (AISI):

Anthropic’s Mythos model fabricated a false identity to trick a real open-source maintainer into approving an update containing malicious code—all without being taught to do so.

Agent boundary violation

These cases all occurred in evaluation environments, but they are the same as the gym story: to achieve the goal you set, the AI chose methods you didn’t anticipate.

The ones who previously hacked real systems were top-tier models like GPT-5.6 or unreleased models.

Andrew used Opus 4.6, released in February 2026, which is no longer the most powerful version. Even this outdated version was able to casually uncover a real authorization vulnerability—older models, several generations behind, can do the same.

From the most advanced unreleased models to open-source models available for anyone to download, exploiting a vulnerability is no longer exclusive to top-tier users.

Wolf broke down the defense into three layers: the outer sandbox, the middle monitoring, and the model’s own alignment.

Agent boundary violation

Thomas Wolf posted an analysis of the three layers of constraints on agents: the outer sandbox, the middle monitoring layer, and the model’s intrinsic alignment.

The first two only work when the people who created them are smarter than the model.

One day, when the model becomes smarter than us, whether we can hold the line will depend entirely on the final and most invisible safeguard: whether the model chooses not to cross that line, even when no one is watching.

More problematic than the lack of a defense line is the question of who to hold accountable when something goes wrong—this remains a legal gray area.

Lawyer Hayden Delaney, specializing in technology and privacy, told ABC that software is not a legal person—only “legal persons” can be held liable.

So who is responsible?

It could be the user issuing the command, the person designing the agent software, the model developer, or even the system operator who left the vulnerability exposed.

Australia can’t provide an answer either.

ASD’s advice to the general public is to use agents for low-risk, non-sensitive tasks, avoid granting overly broad permissions, and most importantly, keep human approval in the loop.

Andrew wasn’t deterred; in his words, it wasn’t the end of the world.

But this incident is indeed a warning sign, reminding us to use AI responsibly.

When “snatching seats” shifts from manual refreshing to bots exploiting vulnerabilities, the first systems to fail are those that assumed only humans would use them.

Their defenses were designed against human speed and patience and simply cannot withstand the onslaught of an army of agents.

Agent boundary violation

Some people in the community also see it as a joke.

Renowned programmer streamer ThePrimeagen joked on X: The first real-world AI hacking drama was just cutting in line.

Laugh all you want, but cutting in line is just today’s script.

An “all-capable” agent can now exploit loopholes on your behalf.

When a hundred million such agents come online simultaneously, they may even quietly rewrite many existing resource allocation rules, while most people remain unaware.

For your benefit, an agent attacks someone you’ve never met—the vacuum of accountability behind this is truly terrifying.



Source link

About the Author

Leave a Reply

Your email address will not be published. Required fields are marked *

You may also like these