In my July post on Logic Apps Automation, I promised a follow-up on the Sandbox. I also got something wrong. I called the Sandbox “built-in Python execution”, and a few paragraphs later I called the Code Interpreter “JavaScript only”, as if both were one toggle.

They are two features. This post explains the difference, and then reports what happened when I built a real scenario on the second one. It took an afternoon, eleven runs, and four failures before it worked.

Two features, not one

The designer now offers four agent types: Workflow agent, Managed agent, Foundry prompt agent, and Foundry Hosted agent. The first two run code in very different ways.

A Workflow agent has a Code Interpreter. It runs JavaScript in isolation, with no network, no file system, and a short timeout. It suits calculation and transformation.

A Managed agent runs inside a sandbox, with cloned repositories, skills, and input files. The documentation calls this a coding agent and describes the sandbox as a micro virtual machine. The portal calls it an “isolated container”. I quote both, because they are different isolation claims.

	Workflow agent	Managed agent
Runs code in	Code Interpreter	Sandbox
Runtime	JavaScript	Shell, scripts, build tools
File system and repositories	No	Yes
Harness	Workflow runtime	GitHub Copilot (only option)
Typical job	Transform and calculate	Read, build, and test code

The scenario: a contract-change reviewer

A backend team publishes a new OpenAPI specification. My integration repository holds the current contract and eight consumer contract tests. The agent must compare the two, run the tests, and report breaking changes.

The demo repository contains the contract, the tests, two sample specifications, and a SKILL.md file with the review rules.

Building it

1. Environment. I created one in UK South. Sweden Central appears in Microsoft’s announcement, but my portal did not offer it.
2. Sandbox. Under Sandboxes, I created contract-review with tier M (1 core, 2 GB RAM, 20 GB disk). At least one repository is required. For GitHub I used OAuth. Managed identity is available for Azure DevOps only.

1. Workflow. An HTTP trigger, a Managed agent, and a Response action.
2. Harness tab. I selected the sandbox and pointed Repository Skills at the skills folder.

1. Parameters tab. A system message, a user message, and one input file named new-spec.md with the content [confirm: ${triggerBody()}].
2. Run settings. Loop count 60 and a timeout.

The result

With the breaking sample, the agent returned:

Verdict: BREAKING. Test result: 5 passed, 3 failed. Breaking changes: amount was removed from the response and is no longer required, and status added a new enum value withdrawn.

With the compatible sample, it returned COMPATIBLE with 8 of 8 tests passed. It also listed the new optional currency field and a version change that no test checks. So it ran the tests and compared the specifications itself, as the skill instructs.

What went wrong on the way

A green tick on a dead connection. The connection dialog reported “Endpoint and key retrieved” for my Azure OpenAI resource. However, key authentication is disabled on that resource. The first run failed with HTTP 403 and a message naming COPILOT_PROVIDER_API_KEY, a variable no builder can set. Switching the connection to system-assigned managed identity fixed it.

A stale repository. A prebuilt sandbox is a snapshot. I pushed the contract and tests after the first build, and the agent kept seeing the old, empty repository. The runs still reported Succeeded.

Test data that overrides your payload. With “Run with test data” on, the trigger returns its stored sample and ignores the JSON in the test dialog.

A sub-agent without a connection. One run failed with “Agent connection not found”. The agent reported that a background review agent had failed. After I told it to do all work itself, the error did not return.

The agent stayed honest

This is the part I value most. In five runs the agent could not do the job: the repository was empty, the specification was empty, or a sub-agent had failed. Each time it said so and stopped. It never invented a test result. The system message asks for exactly that: “If the tests cannot run, say so and stop. Never guess a test result.”

What it costs

Run	Outcome	Total tokens
Sub-agent connection error	No review	203,283
Stale repository	Refusal	126,386
Successful review	BREAKING, 5 passed, 3 failed	74,722

Three things stand out. First, the successful run was the cheapest. A broken setup costs more than a working one, because the agent keeps searching. Second, no tokens were cached in any run. Third, the sandbox compute is negligible: 29 seconds of runtime is about 0.2 cents at list price.

Therefore, the model is the bill. I brought my own Azure OpenAI deployment, so these tokens land on that resource, outside the Automation meters and outside the free grant.

In addition, the loop count defaults to 5,000. In July I called the iteration bound “your cost fuse”. By default, that fuse is open. Moreover, the log showed “Iteration 1 of 1” for runs that made many tool calls, so the loop count does not limit what I expected it to limit.

Where this is the wrong answer

The task is a calculation. Use the Workflow agent and its Code Interpreter.

The steps are known up front. Running contract tests on every commit is a pipeline job. An agent adds cost and a non-deterministic failure mode.

The code must be current. The sandbox reviews the repository as it was at build time. For a check that should follow main, that is a real limitation.

You need a predictable bill. The same task cost between 75,000 and 200,000 tokens, depending on what went wrong.

You need long-running work behind an HTTP call. The request headers show a 30-minute ceiling on the platform’s ingress.

My take

The split is a good design: a cheap, locked-down tool for the common case, and a full workspace for the rare one. The secure setup also worked end to end, with no keys on the model resource, managed identity for the model, and OAuth for the repository.

However, the sandbox moves Automation into territory that build systems already own. So I would use a Managed agent where judgment is needed, such as reading a change and explaining its impact, and leave repeatable work to the pipeline. And I would set the loop count before the first run, not after the first invoice.