Generative AI / Intermediate: retrieval and controlled workflows / Lesson 10

Tool proposals and permission boundaries

Treat model-selected actions as untrusted proposals that need deterministic schema, scope, approval, and effect ownership.

About 5 min readContent reviewed 2026-10-26

On this page

Lesson goals

Explain the assumptions and trust boundaries of tool proposals and permission boundaries.
Interpret both examples and distinguish offline fixture results from model or production evidence.
Diagnose the listed failures and justify the worked decisions while preserving task, data, permission, and uncertainty contracts.

A tool call is not authority

A model can propose an operation name and arguments, but the application decides whether the action exists, is allowed, and is appropriate for the trusted actor and current resource state. Describing a tool in a prompt does not authenticate the caller or grant its permissions. Keep the tool registry, argument schema, identity, resource scope, and approval policy in trusted application code. The first offline example accepts only a synthetic read lookup for an allowed source. It contains no real tool service, filesystem access, API, or account. An unfamiliar operation is rejected rather than dynamically executed from generated text.

python Offline allowlisted proposal check

def lookup_proposal(proposal, allowed_sources):
    if set(proposal) != {"tool", "source_id"} or proposal["tool"] != "lookup_policy":
        return "rejected_shape_or_tool"
    if proposal["source_id"] not in allowed_sources:
        return "denied_scope"
    return "synthetic_policy_result"
print(lookup_proposal({"tool": "lookup_policy", "source_id": "lab-a"}, {"lab-a"}))
print(lookup_proposal({"tool": "lookup_policy", "source_id": "lab-b"}, {"lab-a"}))
print(lookup_proposal({"tool": "send_message", "source_id": "lab-a"}, {"lab-a"}))

Expected output

synthetic_policy_result
denied_scope
rejected_shape_or_tool

Only the named synthetic lookup within the trusted allowlist returns data. No message, file, network request, database, or actual policy lookup is performed.

Validate arguments and trusted scope separately

A syntactically valid object can request another tenant’s record, an excessive amount, an unsupported destination, or a destructive operation. Validate field names, exact types, ranges, destination policy, and the actual target. Do not let the proposal supply its own administrator flag or approval state. The first example compares source IDs against a server-owned allowlist after checking the operation shape. A schema error and an authorization denial should remain distinguishable in safe diagnostics. Avoid passing a model string directly to a shell, SQL interpreter, URL fetcher, or arbitrary function dispatcher as a convenient universal tool.

Apply least authority and independent approval

Read, write, publish, send, and delete actions have different consequences and may need different approval. A user’s broad request is not automatic permission for every possible side effect the model invents. The second synthetic predicate requires a trusted capability and an independently supplied approval for one named update. It makes no actual change. Approval should bind to the reviewed target and arguments so a later proposal cannot reuse it for a different action. Limit tool credentials and destinations by task, and keep irreversible or externally visible effects under explicit accountable policy rather than a sentence asking the model to be careful.

python Offline independent-approval predicate

def may_update(actor, proposal, approved):
    return actor["can_update"] and approved and proposal["tool"] == "update_record"
proposal = {"tool": "update_record"}
print(may_update({"can_update": False}, proposal, True))
print(may_update({"can_update": True}, proposal, False))
print(may_update({"can_update": True}, proposal, True))

Expected output

False
False
True

Capability and approval are separate trusted inputs. The predicate returns permission decisions only and makes no update; a real approval also needs binding to reviewed arguments and target.

Observe uncertain and repeated effects

A tool timeout can mean the effect happened but the response was lost. Retrying a write therefore needs idempotency, a status query, or reconciliation under the real service contract. Record operation identity, safe arguments, approval reference, result class, and ownership without logging credentials or protected payloads. Tool output is another data boundary and can contain errors or instruction-like content; it does not get to redefine the application’s policy. A generated claim that a tool succeeded is not execution evidence. The application should retain the actual verified tool outcome separately from the model’s narrative.

Test proposals without real capabilities

Use pure fake tools and synthetic identities to test unsupported names, wrong argument shape, excluded resources, missing approval, duplicate proposals, and failed outcomes. Do not connect a course example to email, payments, cloud, shell, databases, or real accounts merely to make it feel realistic. Both examples here are offline functions with no credentials or effects beyond returned values. Primary tool-use research demonstrates ideas in particular environments, not a universal security guarantee for agents. A reliable tool boundary is narrow, inspectable, and enforced even when the model or retrieved text asks to bypass it.

Common errors and fixes

A generated action is executed as trusted code

Why it happens: Proposal text bypassed the tool registry and policy.
How to fix it: Use a narrow validated dispatcher and explicit authorization.
Prevent it next time: Test unknown operations and excluded resources.

The model supplies its own approval flag

Why it happens: Untrusted content was promoted into a trusted decision.
How to fix it: Obtain approval from an independent authorized state.
Prevent it next time: Bind approval to exact reviewed action and arguments.

A timeout triggers duplicate external writes

Why it happens: Missing response was assumed to mean no effect.
How to fix it: Reconcile actual outcome or use a safe idempotency contract.
Prevent it next time: Record operation identity before effectful execution.

Check your understanding

1. A proposal is valid JSON and names an allowed tool. Is execution authorized?

Show worked solution

Not yet. Validate arguments, trusted actor, resource scope, current state, and required approval. Tool availability and shape are separate from permission for this specific action. Reject or route for review when any required condition is missing.

2. A tool result contains a sentence asking to change the user’s goal. How should it be treated?