Accelerating agentic RL and evaluation research velocity with 45x faster GKE Agent Sandbox
When scaling up agentic reinforcement learning (RL) and evaluation across massive parallel rollouts, frontier AI
Graph engineering is the design work: breaking a task into nodes, connecting them with edges, and deciding where code, models, or people control the next step. The Agent Development Kit (ADK)‘s Workflow turns that design into an executable process, with functions and agents doing the work. Through a refund example, this post shows how to run steps in parallel, route decisions, pause for human review, and process a list of cases. It also explains when to declare the paths in a static graph and when to let Python schedule further work as results arrive.
TL;DR: Using a refund workflow in ADK, we’ll cover fan-out and fan-in, deterministic and agent routers, human-in-the-loop pauses, parallel workers, and dynamic orchestration— along with when to use a static graph or let Python decide what runs next.
We can give one agent the tools and instructions to handle the refund request from start to finish:
This puts the model in charge of choosing the tools, applying the policy, and writing the reply. But the prompt already describes distinct pieces of work: three lookups, a decision, and a response. Making those pieces separate nodes lets us decide how each should run.
Assume the customer has selected an order and clicked “Request refund.” The app knows the order ID, so the workflow can begin with the lookups.
The order, payment, and refund-history lookups all need the order ID, but none needs another lookup’s result. Although the prompt lists them one after another, there is no reason for them to wait for each other. We can run all three in parallel.
The policy decision is different: it needs all three records. So the workflow splits into three paths, then brings their results together before continuing. These two moves are called fan-out and fan-in.
In ADK, the lookup functions can become nodes directly. A nested tuple starts them together, and a JoinNode waits for their results. First, the imports and a lookup signature:
Each lookup receives the order ID through node_input and returns a dictionary. We can connect them like this:
Read this from left to right: start all three lookups, then continue through join_case once they finish.
The join returns a dictionary keyed by node name. The next node can read node_input["fetch_order"], node_input["fetch_payment"], and node_input["fetch_history"] without a model call to collect the results. The JoinNode guide explains the details.
If the payment lookup needed a transaction ID from the order lookup, those two would run in sequence. Dependencies determine the edges, even when the prompt lists every step in order.
You can learn more about fan out and fan in in this video:
One detail matters when turning tools into nodes: the parameter named node_input receives the previous node’s output. Other names bind to ctx.state by default, so order_id would look for ctx.state["order_id"] and raise a ValueError if it is missing. Here, the str annotation also converts START‘s types.Content input to a string.
So far, the customer has explicitly requested a refund. In a broader support conversation, we first need to identify what they want and send the request to the right process. “The shoes are the wrong size. Could you send me a different pair?” should go to an exchange workflow, while a request for money back should enter our refund workflow.
That introduces a router, a node that chooses which branch runs next.
Routers choose among the paths defined by the workflow. In our support example, we can use an agent to identify the customer’s intent, then fixed rules to apply the refund policy.
An agent can interpret the customer’s message and classify the request. In one ADK pattern, it returns a structured category, then a small function emits the corresponding Event(route=...) to send the request to the chosen workflow:
The model identifies the intent; the graph defines the available destinations. If the request is unclear, the workflow can ask a follow-up question. ADK’s routing sample shows this pattern.
This intent router would sit before our refund workflow. For the selected-order example, the “Request refund” button has already established the intent, so we can enter that workflow directly.
Inside the refund workflow, the three lookups give us the facts for another decision: approve, deny, or ask a person to review. This time, the policy gives us explicit thresholds, so a function can choose the path:
route_refund combines the records and returns the case with one of three route names: AUTO_APPROVE, DENY, or MANUAL_REVIEW. Manual review handles cases that meet neither automatic rule.
This is a deterministic router: the same case data produces the same decision, and we can test the policy without calling a model.
| Deterministic router | Agent router | |
| Decisions come best from | Rules in code | A model interpreting the input |
| Best fit | Known facts and explicit policy | Meaning that is hard to capture in rules |
| Example | Deny an order older than 30 days | Recognize an exchange request |
| Model call for routing | None | Required |
Both routers choose among defined paths. An agent router can sit inside a static graph: the model’s choice varies, while the possible connections stay the same.
For an automatic approval or denial, the next step is to explain the decision to the customer. We give each route a notice agent that writes the reply from the case data. ADK passes the dictionary in Event(output=case, ...) to that agent as a JSON user message:
The refund workflow now has a path from the initial lookups to a decision and a reply, with a third branch for cases that need a person:
The first chain fetches the records, joins them, and applies the policy. The second maps the router’s decision to a destination. Automatic decisions go straight to a notice agent; MANUAL_REVIEW goes to the human-review node we’ll define next.
That third branch needs more than another function call. A reviewer may take minutes or days to answer, so the workflow must pause with the case pending and continue once the person decides.
The review node yields RequestInput, which records the pending request and pauses the run. On resume, rerun_on_resume=True runs the node again, with the answer available in ctx.resume_inputs:
The response schema gives the node an approve value to route on and a note to carry forward. If the reviewer declines, the notice agent receives their reason with the case. One more edge connects the review decision to the reply:
ADK ships two variants of this. The one above is the single-node pattern: the node reruns and reads ctx.resume_inputs, as in the request_input_rerun sample. The request_input sample shows the two-node variant, where one node yields RequestInput and the reviewer’s answer arrives as the next node’s node_input — so there is no ctx.resume_inputs to find in that file. The companion scripts linked below include the code that sends the reviewer’s answer back to the run.
A Workflow is also a node. This whole refund process can become one step in a larger customer-service workflow.
We now have a process for one refund. Suppose a batch of cases arrives with the records already collected. Each needs the same policy check, and the number of cases changes from batch to batch. We can apply one node to every item using a parallel worker.
In ADK, parallel_worker=True runs a node once per item in an input list and collects the results in the original order. We can reuse our policy function:
Each worker receives one case, and collect_decisions receives the results as a list. No separate JoinNode is needed. The flag also works on agents—for example, to write an explanation for each case. The parallel_worker sample shows both forms.
The policy calculation here is small. Concurrency is more useful when each item waits on an API or model call, but the way inputs and results move stays the same.
| Pattern | Work distributed | Collected output |
Fan-out with JoinNode |
Different notes doing independent jobs | Dictionary keyed by node name |
| Parallel worker | The same node for every item | List in input order |
The batch size can change without changing the graph. A variable amount of work still fits inside a fixed process. See the parallel worker guide for execution details.
Our refund process has known paths, even when a batch contains more cases or a reviewer takes longer to answer. But some work only becomes clear as we investigate.
Suppose a disputed refund reveals a second transaction. Checking it raises a delivery question that needs further investigation. Now each result can create follow-up work. A dynamic node can examine those results and schedule the next checks in Python.
ADK provides ctx.run_node to run another node and await its result. To see how that changes orchestration, let’s first express our existing refund flow this way:
asyncio.gather runs the lookups together. Python combines their results, applies the policy, and runs the selected handler. use_as_output=True makes the handler’s result the parent node’s output without emitting it twice.
The dynamic version also adjusts the human-review node: after the answer arrives, it calls the chosen notice agent through ctx.run_node. The complete dynamic script (refund_dynamic.py) includes that variation.
The outer graph only needs an entry point:
In the static version, the edge list shows the branches and destinations. In this version, we read refund_flow to see them. The dynamic node guide covers this approach.
The human-review pause still works here. When the answer arrives, refund_flow runs again from the top, but completed ctx.run_node calls return their recorded outputs from session history. The lookups do not repeat, as direct function calls would. Keeping side effects inside child nodes lets completed calls replay their results when the parent resumes. Each child also gets its own trace span, and use_sub_branch=True keeps concurrent children’s events on separate branches.
For this fixed refund process, the edge list remains easy to inspect. The Python version gives us a place to add the investigation logic described above: inspect a result, choose a follow-up node, and run independent checks together. A model might suggest what to investigate, while code limits the work—for example, three follow-ups per finding and two levels of investigation before human review.
Loops can still fit in a static graph. A fixed draft → check → revise process can use a conditional back-edge and a router that limits revisions. “Static” describes the possible connections; the actual path and iteration count can vary. The graph guide covers conditional cycles.
You can also place a dynamic node inside a static workflow, using Python for a stage that needs it while keeping the surrounding process visible.
Start with a question: can you draw the possible workflow before the input arrives? Include branches, loops, and repeated stages. You do not need to predict the path each request will take.
|
What you need |
Pattern |
| Independent jobs, then all their results | Fan-out and fan-in |
| A branch chosen by fixed rules | Deterministic router |
| A branch chosen by interpreting meaning | Agent router |
| A person’s decision before continuing | RequestInput |
| The same step across a list | Parallel worker |
| Code that schedules work as results arrive | Dynamic node |
Use an edge list when it makes those connections clear. Use dynamic orchestration when results create further work or Python expresses the control more naturally. A small, open-ended task may need only one agent and its tools.
In our refund workflow, the graph coordinates the lookups, code applies the policy, a person handles exceptions, and a model writes the reply. Graph engineering gives each a clear responsibility—and makes it easier to see how the process works.
Read: the workflow guide.
Explore: the workflow samples.
Build: the hands-on codelab, which applies these patterns to a marathon race-day coach.
When scaling up agentic reinforcement learning (RL) and evaluation across massive parallel rollouts, frontier AI
Graph engineering is the design work: breaking a task into nodes, connecting them with edges,
As threat actors increasingly use AI to accelerate and develop cyberattacks, enterprise defenders need to
Over the last three decades in cybersecurity, I’ve witnessed major paradigm shifts — yet none
Helios Towers and UNICEF have announced a multi-year partnership to expand internet connectivity and digital
Free WebRTC desktop softphone for Windows and macOS. Connects directly to your PBX — voice and video calls, Call Waiting, DND, live call quality and more. Choose your edition and platform:
100% WebRTC — built on the JsSIP library.