About this session
Personal agents are most useful when they can actually do things: run commands, send messages, browse the web, and even delete files. While model alignment and safeguarding techniques are improving rapidly, such capabilities mean agents still carry risk — even a well-intentioned model might perform a harmful action in pursuit of a goal. This raises security questions regarding both the models themselves and what happens around them. This talk walks through framing one can use to tackle this problem, drawn from Anthropic engineers building and deploying agentic products like Claude Code and Claude Cowork.
In general, every action an agent wants to take can resolve one of three ways: block it, allow it, or triage for permission. Each of those leads to unique problems when taken in excess, so the interesting work is in deciding which applies when, and designing one's execution environment to limit the blast radius of mistakes. We will cover how those decisions change depending on who the user is — from developers who can read a bash command to knowledge workers who can't — and what prompt injection means for agents that read untrusted content. You should leave with practical tips to apply to your own agentic designs before giving them real access, whatever framework or model you build on.
Learning objectives
- Understand several ways to frame agent security
- Recognise failure modes of excessively allowing, triaging, or even denying agent actions
- Holistically consider agent risk — knowing when some risk is acceptable, how to mitigate it by controlling blast radius, and how this calculation is evolving over time
