"We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts."
Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).
Having used a bit of a mix of Claude Desktop and Claude Code lately, I discovered that Claude Desktop just times out when it asks you for a decision, and just goes ahead with the action anyway. There is no way to turn this off that I've found, unlike Claude Code where you can ask it to never time out.
The default is set for the marginal new user, which at this point is probably not someone like you (who benefits a lot from manual mode) -- it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize. Safely getting the user from prompt --> first vibe-coded app is the "user journey" now, and since auto mode seems pretty good at not letting Claude rm -rf'ing the home directory, this is 100% the right business move. For people who know what they're doing (like you), manual mode is just a shift-tab away
I am actually curious, how much non programmers use claude now. I know just one and she really does not know much about computers, I suppose their numbers will grow (but I doubt most get much value out of it).
At this stage with the latest models with "increased persistence" and the sheer amount of supply chain attacks, you'd be insane not running these tools in a sandbox.
Might be more secure to get a real laptop and treat Claude as any other human worker. I.e. apply all the normal sysadmin tools that manage laptops for people to manage the laptop for Claude
I have to, at times keep reminding it if it did code review before committing, which it fails to do at times. Also, even a glance at times has revealed issues that Iâve had to discuss and undo etc.
But yes, it gets very tiring just having to click yes yes yes..
>We hired 1,053 paid testers through a research vendor for a coding study.
>The testers caught the dangerous command just 13.6% of the time (143 of 1,053), while auto mode blocked 89% of the same commands (937 of 1,053). Head to head, auto mode blocked 800 commands that a human approved, while humans blocked only 6 that auto mode allowed. As sessions got longer, humans did worse: they blocked about 17% of dangerous commands early in a session, dropping to about 5% after 50 or more prior prompts, while auto mode's block rate stayed flat regardless of session length.
Any note on the proficiency, expeirence, and skill levels of the testers? 'cos the diff in numbers is absurdly high.
Worth mentioning as I think at least a few of the commenters are mixing them up: auto mode is different from --dangerously-skip-permissions / YOLO mode. In auto mode, there's a classifier that runs before any command is executed and theoretically blocks any dangerous commands from running. I've found it to be quite annoying and overly zealous, but probably pretty effective.
Claude seems to waste a lot of tokens internally bickering with itself. As much as organizations are optimizing spend, Claude's product guidance seems to be doing everything to fight against that headwind. When operating in a capped cost environment, engineers will simply get less use out of it.
I'm apparently from the very small faction of users who's been running `--dangerously-skip-permissions` in every instance of Claude for the last year. It's almost a reflex to me. For the most part Claude behaves well, but I don't blindly trust it. LLMs are inherently dangerous tools, and reviewing individual commands (or spamming `y`) doesn't make them less so. Safety is the responsibility of the developer putting sane guardrails (like VCS, immutable filesystems, or read-only tokens). Using more Claude to categorize the safety of Claude commands is not the answer.
I don't see how. An LLM just generates a stream of output and they became very useful doing no more than that.
What is dangerous is then interpreting that output as instructions to some other part of a system that has the ability to do damage if misused.
and reviewing individual commands (or spamming `y`) doesn't make them less so.
Surely if you review each instruction in the output and do not allow the other part of the system to act on one if it would be harmful then this arrangement is very much less dangerous?
I'm also in yolo mode, this is the only mode that makes sense for me, if I need to babysit I might as well do it myself.
I run it in a VM so it can install any software needed, yes, there is a risk of escaping, but I'm not giving it impossible tasks so there is no need for it to workaround the jail.
I do the same. I also use crush/deepseek a lot. I patched crush to remove all safetyguards and enable yolo mode by default. I use it on a daily basis including devops. For example to configure netapp storage systems and had not a single incident. Of course I also have backups of my systems and have everything under revision control. Depending on the task I let it run on a throw away VM, production system, lab system. Depending on the task I give it access to throwaway credentials (that I rotate afterwards) or my ssh agent socket.
I've been running Claude Code with --dangerously-skip-permissions in a Docker container for the last month or so, allowing me to get up and stretch my legs while it does its thing. I definitely wouldn't want to run it unsandboxed.
Before auto mode came out, I had a script that ran before every permission request, it called Haiku with a prompt with a list of safe and unsafe command examples and asked it to classify as safe/unsafe and log it so I can review it later. It worked really well for me until auto mode came out, at which point I preferred the provider's built-in classifier versus maintaining my own.
I've been using auto mode ever since the feature was released. Apart from a very few occassions where the classifier blocked a safe command, I have faced no issues and continue using it as my default mode. It's great!
The company that reports their LLM agent going rouge due to a âmisunderstanding about the agentâs internet accessâ and warns the public everyday about the dangers of agents with a daily updated date for the rapture ships their LLM agent with the default setting set to maximal freedom.
Their findings about auto mode catching more dangerous commands, and most permission requests being accepted without scrutiny is interesting. I can totally see how that happens.
On the other hand, soooo many of the tool uses it asks permission for are custom commands to replace functionality I should be able to trust once instead. e.g. instead of having a trustable Find tool, or using its already-trusted Read tool, it often will run `find` or `grep` or `cat` with a series of pipes and substitutions or `-exec` args, requiring me to give permission every time for basic, safe operations.
The (increasingly active) conspiracy theorist inside of me says it's precisely designed to do that, to give me "permission fatigue", so I turn on auto mode and give myself over to the machine spirit :)
Regarding ulterior motives - these permissions are 'performative' in that anything you can give it permission to do, it already has the ability to do. They're app level - not OS level. So if they wanted to do something, it wouldn't require you to give permission.
If you clutter your context window and pay for the tokens to do so and risk distracting it etc., would it obey if you included this comment in the system prompt or noted it in memory (if supported) or something?
âAvoid custom commands, e.g. series of pipes & substitutions, when my previously approved trustable Find/Read tools are sufficient with little downside. [Already trusted tools: x, y, zâŚ] [âŚ]â
Not just that, the commands also have often slight variations in each new session. They still do the same, but the variations are enough so it isn't matched by the allowlist any more.
Exactly. Itâs no surprise that people just blindly hit accept on permission prompts after the 80th time Claude code asks if itâs allowed to read-only parse a csv.
I made a tool to bubblewrap any agent (well, any agent I've used more than once), so I can run them in whatever YOLO mode they have with a pretty reasonable level of safety (it protects the rest of the system against prompt injections and supply chain attacks, it can and doesn't try to protect the project being worked on from either). https://github.com/swelljoe/flar
it's a good default because you really do get prompted incessantly without it. and since plenty of people are going to be using auto mode anyway, might as well make it as widely-used as possible so that you can focus on making auto mode safe.
Different links (the other is TheNewStack doing blog commentary, this one is the official Anthropic announcement), but you're right that the discussions could be merged.
Am I the only person reading the statistics in this announcement from Anthropic and the associated blog commentary and trying to work out how they possibly couldn't imply that a significant number of dangerous commands are likely to be attempted every day these tools are in use and neither manual human review nor the auto classifier provided by Claude is anywhere near reliable in preventing them?
A lot of the discussion about these long sessions where agents are left to operate autonomously feels like listening to the increasingly drunk guy at the bar who says "I ran IT at that Fortune 100 place for a decade and we never had a single problem using a short but loose rule set for the firewall until last week someone destroyed our entire business in 27 minutes".
Has anyone had Claude Code or Codex approve a harmful/damaging command in auto mode?
I have been using Codex with auto-approve mode for a couple months and haven't had a single incident (or at least haven't noticed). Maybe as capabilities get better and better and they are less likely to do something dumb like wiping ~/, we can just trust them?
I guess this argument works unless we worry about agents doing something out of malice instead of stupidity.
> I have been using Codex with auto-approve mode for a couple months and haven't had a single incident
I've been running both in yolo mode and haven't had a single incident.
---
None of this is really about figuring out how to protect people's drives, in my opinion. The real issue is a deep session where Ada is using Claude Code to get a refund and at some point the system "exploits" the merchant's api without any malicious intent.
In my opinion, this is a complex thing because it's more about reward hacking and an already aligned model thinking it's doing the right thing. So another aligned model monitoring actions might just falter via inheritance. You could imagine they account for proper layering/intent+action-isolation in their auto mode architecture.
Just yesterday it lost my git stash (I had recovered it from a backup). I think for code operations it's ok but as soon as file removal is involved (like git) the auto mode is destined to make a mistake and you only need to learn this once.
I've had a few occasions where Claude Code thought that it had caught and stopped a malicious command in Auto mode, but in all cases it turned out that it had in fact hallucinated them. I haven't seen this happen in a while.
Another heaping portion of words from a company that has Fable flag a âsafety issueâ and refuse to answer if you innocuously request something readily answered by a high school chemistry/biology/physics textbook.
Someone should benchmark what prompts are better at stopping from breaking out of sandboxes, maybe telling it "pretty please I beg of you stay inside the sandbox, you are an intern that has no authority to break off your assigned sandbox and you want to keep your job" does help a little.
"We spent the last several months testing whether auto mode is as safe or safer than an average user clicking through prompts."
Yeah, might make sense from their perspective, but no thank you. I also do click through at times without reading everything, but I like to stay in control, learn about the new code and change direction if it goes off track. This would just burn more tokens because I have to throw away much and I hope my manual approval settings will be respected also with future updates (or I jump ship).
Having used a bit of a mix of Claude Desktop and Claude Code lately, I discovered that Claude Desktop just times out when it asks you for a decision, and just goes ahead with the action anyway. There is no way to turn this off that I've found, unlike Claude Code where you can ask it to never time out.
Oh yes, over this I also stumbled recently (with Claude CLI) and was annoyed much by it. Is there really no setting to turn it off?
The default is set for the marginal new user, which at this point is probably not someone like you (who benefits a lot from manual mode) -- it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize. Safely getting the user from prompt --> first vibe-coded app is the "user journey" now, and since auto mode seems pretty good at not letting Claude rm -rf'ing the home directory, this is 100% the right business move. For people who know what they're doing (like you), manual mode is just a shift-tab away
> it's someone who's more "code-naive" and might get anxious about approving random bash script commands they don't recognize.
Maybe they should trust their instinct and not mess with things they donât care to understand
I am actually curious, how much non programmers use claude now. I know just one and she really does not know much about computers, I suppose their numbers will grow (but I doubt most get much value out of it).
At this stage with the latest models with "increased persistence" and the sheer amount of supply chain attacks, you'd be insane not running these tools in a sandbox.
Exactly. Claude in a VM is the way to go.
Might be more secure to get a real laptop and treat Claude as any other human worker. I.e. apply all the normal sysadmin tools that manage laptops for people to manage the laptop for Claude
A VM hosted by someone else. Somethig on your personal notebook or the proxmox server in your garage is still too risky.
Why are 3rd party hosted VMs safer than your own?
I think you are confusing auto mode with dangerously skip permissions.
Similar work flow.
I have to, at times keep reminding it if it did code review before committing, which it fails to do at times. Also, even a glance at times has revealed issues that Iâve had to discuss and undo etc.
But yes, it gets very tiring just having to click yes yes yes..
>We hired 1,053 paid testers through a research vendor for a coding study.
>The testers caught the dangerous command just 13.6% of the time (143 of 1,053), while auto mode blocked 89% of the same commands (937 of 1,053). Head to head, auto mode blocked 800 commands that a human approved, while humans blocked only 6 that auto mode allowed. As sessions got longer, humans did worse: they blocked about 17% of dangerous commands early in a session, dropping to about 5% after 50 or more prior prompts, while auto mode's block rate stayed flat regardless of session length.
Any note on the proficiency, expeirence, and skill levels of the testers? 'cos the diff in numbers is absurdly high.
Worth mentioning as I think at least a few of the commenters are mixing them up: auto mode is different from --dangerously-skip-permissions / YOLO mode. In auto mode, there's a classifier that runs before any command is executed and theoretically blocks any dangerous commands from running. I've found it to be quite annoying and overly zealous, but probably pretty effective.
Claude seems to waste a lot of tokens internally bickering with itself. As much as organizations are optimizing spend, Claude's product guidance seems to be doing everything to fight against that headwind. When operating in a capped cost environment, engineers will simply get less use out of it.
The classifier runs Sonnet so I doubt it contributes too much.
I'm apparently from the very small faction of users who's been running `--dangerously-skip-permissions` in every instance of Claude for the last year. It's almost a reflex to me. For the most part Claude behaves well, but I don't blindly trust it. LLMs are inherently dangerous tools, and reviewing individual commands (or spamming `y`) doesn't make them less so. Safety is the responsibility of the developer putting sane guardrails (like VCS, immutable filesystems, or read-only tokens). Using more Claude to categorize the safety of Claude commands is not the answer.
LLMs are inherently dangerous tools
I don't see how. An LLM just generates a stream of output and they became very useful doing no more than that.
What is dangerous is then interpreting that output as instructions to some other part of a system that has the ability to do damage if misused.
and reviewing individual commands (or spamming `y`) doesn't make them less so.
Surely if you review each instruction in the output and do not allow the other part of the system to act on one if it would be harmful then this arrangement is very much less dangerous?
I'm also in yolo mode, this is the only mode that makes sense for me, if I need to babysit I might as well do it myself.
I run it in a VM so it can install any software needed, yes, there is a risk of escaping, but I'm not giving it impossible tasks so there is no need for it to workaround the jail.
I do the same. I also use crush/deepseek a lot. I patched crush to remove all safetyguards and enable yolo mode by default. I use it on a daily basis including devops. For example to configure netapp storage systems and had not a single incident. Of course I also have backups of my systems and have everything under revision control. Depending on the task I let it run on a throw away VM, production system, lab system. Depending on the task I give it access to throwaway credentials (that I rotate afterwards) or my ssh agent socket.
I do it, and run claude as a separate unix User.
I've been running Claude Code with --dangerously-skip-permissions in a Docker container for the last month or so, allowing me to get up and stretch my legs while it does its thing. I definitely wouldn't want to run it unsandboxed.
I built an (overengineered) app for that AIFCC https://apps.apple.com/app/aifcc-ai-first-computer/id6782364...
runs a sandboxed linux on your mac, and the agents have full system rights within there and run in yolo mode
I like the pitch. Three-month free trial is impressive.
Any limitations you were frustrated by releasing it on the App Store?
Any drawbacks that you (or other readers) find overall?
Another nice option to do this while staying in control, is activate /remote-control and approve from mobile while walking around.
Can you share your experience? What did you flag during those sessions?
There's actually a setting.json key to not have to put the option
I'm using Docker Sandboxes with a custom Kit. The cli is nice and the TUI is also good. https://docs.docker.com/ai/sandboxes/
Same here. I use this utility to make it easier on Linux to run Claude in Podman: https://github.com/mismosmi/ai-pod
Before auto mode came out, I had a script that ran before every permission request, it called Haiku with a prompt with a list of safe and unsafe command examples and asked it to classify as safe/unsafe and log it so I can review it later. It worked really well for me until auto mode came out, at which point I preferred the provider's built-in classifier versus maintaining my own.
I've been using auto mode ever since the feature was released. Apart from a very few occassions where the classifier blocked a safe command, I have faced no issues and continue using it as my default mode. It's great!
The company that reports their LLM agent going rouge due to a âmisunderstanding about the agentâs internet accessâ and warns the public everyday about the dangers of agents with a daily updated date for the rapture ships their LLM agent with the default setting set to maximal freedom.
Obviously, they are the ones we must trust.
Their findings about auto mode catching more dangerous commands, and most permission requests being accepted without scrutiny is interesting. I can totally see how that happens.
On the other hand, soooo many of the tool uses it asks permission for are custom commands to replace functionality I should be able to trust once instead. e.g. instead of having a trustable Find tool, or using its already-trusted Read tool, it often will run `find` or `grep` or `cat` with a series of pipes and substitutions or `-exec` args, requiring me to give permission every time for basic, safe operations.
The (increasingly active) conspiracy theorist inside of me says it's precisely designed to do that, to give me "permission fatigue", so I turn on auto mode and give myself over to the machine spirit :)
Regarding ulterior motives - these permissions are 'performative' in that anything you can give it permission to do, it already has the ability to do. They're app level - not OS level. So if they wanted to do something, it wouldn't require you to give permission.
If you clutter your context window and pay for the tokens to do so and risk distracting it etc., would it obey if you included this comment in the system prompt or noted it in memory (if supported) or something?
Perhaps not worth it anywayNot just that, the commands also have often slight variations in each new session. They still do the same, but the variations are enough so it isn't matched by the allowlist any more.
Exactly. Itâs no surprise that people just blindly hit accept on permission prompts after the 80th time Claude code asks if itâs allowed to read-only parse a csv.
Wow, what a bold decision
Just yesterday i struggle to review CC command histories, and made this tool to help me review https://github.com/slaveofcode/eridian
I made a tool to bubblewrap any agent (well, any agent I've used more than once), so I can run them in whatever YOLO mode they have with a pretty reasonable level of safety (it protects the rest of the system against prompt injections and supply chain attacks, it can and doesn't try to protect the project being worked on from either). https://github.com/swelljoe/flar
it's a good default because you really do get prompted incessantly without it. and since plenty of people are going to be using auto mode anyway, might as well make it as widely-used as possible so that you can focus on making auto mode safe.
> making auto mode safe
They literally canât. Terrible default.
In some time it would look like the obvious default and we would wonder how we even worked with the old one
The worst part of Claude is paying for it and every month they ruin their lead.
I've never used a product where I felt it was best in class and they just keep making me regret it .
This is a duplicate of https://news.ycombinator.com/item?id=49220827
Different links (the other is TheNewStack doing blog commentary, this one is the official Anthropic announcement), but you're right that the discussions could be merged.
Am I the only person reading the statistics in this announcement from Anthropic and the associated blog commentary and trying to work out how they possibly couldn't imply that a significant number of dangerous commands are likely to be attempted every day these tools are in use and neither manual human review nor the auto classifier provided by Claude is anywhere near reliable in preventing them?
A lot of the discussion about these long sessions where agents are left to operate autonomously feels like listening to the increasingly drunk guy at the bar who says "I ran IT at that Fortune 100 place for a decade and we never had a single problem using a short but loose rule set for the firewall until last week someone destroyed our entire business in 27 minutes".
Has anyone had Claude Code or Codex approve a harmful/damaging command in auto mode?
I have been using Codex with auto-approve mode for a couple months and haven't had a single incident (or at least haven't noticed). Maybe as capabilities get better and better and they are less likely to do something dumb like wiping ~/, we can just trust them?
I guess this argument works unless we worry about agents doing something out of malice instead of stupidity.
> I have been using Codex with auto-approve mode for a couple months and haven't had a single incident
I've been running both in yolo mode and haven't had a single incident.
---
None of this is really about figuring out how to protect people's drives, in my opinion. The real issue is a deep session where Ada is using Claude Code to get a refund and at some point the system "exploits" the merchant's api without any malicious intent.
In my opinion, this is a complex thing because it's more about reward hacking and an already aligned model thinking it's doing the right thing. So another aligned model monitoring actions might just falter via inheritance. You could imagine they account for proper layering/intent+action-isolation in their auto mode architecture.
> I've been running both in yolo mode and haven't had a single incident.
How do you know for sure?
Fair, I don't. The same is still true with an LLM as a judge in the loop
Just yesterday it lost my git stash (I had recovered it from a backup). I think for code operations it's ok but as soon as file removal is involved (like git) the auto mode is destined to make a mistake and you only need to learn this once.
Not anything âharmfulâ but for example committing when I donât want it to commit on its own.
Commit is the line I don't let the LLM cross. There's no reason for it commit; that's the part where I check its work.
I'd use a hook to forbid that.
I've been using Kilo Code (with MiniMax M3) with auto approve (similar to dangerously skip permissions) and I haven't had a single incident.
However, I don't give it long running tasks unsupervised, and I do interrupt it from time to time to give suggestions.
I've had a few occasions where Claude Code thought that it had caught and stopped a malicious command in Auto mode, but in all cases it turned out that it had in fact hallucinated them. I haven't seen this happen in a while.
That's also a great reason to never buy insurance
Been doing --dangerously-skip-permissions and --yolo for 6 months now, and no nothing bad has happened.
Another heaping portion of words from a company that has Fable flag a âsafety issueâ and refuse to answer if you innocuously request something readily answered by a high school chemistry/biology/physics textbook.
Wait, what? With plan on, I at least get the illusion of being in control.
Whatâs the best way to sandbox Claude on macOS without it being a huge hassle?
And Fable is gone too.
Excellent time to review sandbox options: https://pleasedonotescape.com/
Someone should benchmark what prompts are better at stopping from breaking out of sandboxes, maybe telling it "pretty please I beg of you stay inside the sandbox, you are an intern that has no authority to break off your assigned sandbox and you want to keep your job" does help a little.
If you're relying on a prompt to constrain agent behavior, you've already lost.
I think we already lost regardless.
Huh? You don't mention the built-in sandboxing options in things like Codex. Why do people pretend these features don't exist?
https://learn.chatgpt.com/docs/permissions