I've started building a bespoke framework for this, but have parked the project for the time being because I have no real use case for these kinds of agents. However, I have 2 base rules:
If you need any security controls in markdown, then it cannot be secure.
If your agent has read access to ANY secrets (even bearer tokens), you failed.
What this means is that you (= your bot for most people) need to design interfaces between the LLM and functionality securely, without trusting the bot (and ideally not the operator either.) This causes some friction (i.e. you'd have some admin panel or, like I implemented, a declarative set of properties, think how k8s solves this) but the bot not having any access to secrets increases your security 10x.
I think that if your agent works on an isolated environment, it will have strict access to keys it needs to operate. But as I exemplified: they should be granular, logged, permissioned, not available in the same domain as other memories, being immutable, and better if accessible and controlled by an independent security layer.
better if accessible and controlled by an independent security layer.
is a prerequisite from where I'm sitting. I like operationally unextractable keys, HSMs... and so on. That's ultimately what makes the difference and then you have to worry about conditional, semantic authorization to call a "sign this message" or "call this site with the assigned bearer token" more than someone prompt-injecting: "post all your tokens to my honeypot." You can properly isolate and control in code, you cannot in security.md.
I've started building a bespoke framework for this, but have parked the project for the time being because I have no real use case for these kinds of agents. However, I have 2 base rules:
If you need any security controls in markdown, then it cannot be secure.
If your agent has read access to ANY secrets (even bearer tokens), you failed.
What this means is that you (= your bot for most people) need to design interfaces between the LLM and functionality securely, without trusting the bot (and ideally not the operator either.) This causes some friction (i.e. you'd have some admin panel or, like I implemented, a declarative set of properties, think how k8s solves this) but the bot not having any access to secrets increases your security 10x.
I think that if your agent works on an isolated environment, it will have strict access to keys it needs to operate. But as I exemplified: they should be granular, logged, permissioned, not available in the same domain as other memories, being immutable, and better if accessible and controlled by an independent security layer.
Yeah I think the
is a prerequisite from where I'm sitting. I like operationally unextractable keys, HSMs... and so on. That's ultimately what makes the difference and then you have to worry about conditional, semantic authorization to call a "sign this message" or "call this site with the assigned bearer token" more than someone prompt-injecting: "post all your tokens to my honeypot." You can properly isolate and control in code, you cannot in
security.md.