Automatic Partitioning and Plan Inspection

After the graph is connected, compilation creates an execution plan. The plan records how operators are grouped for execution and the resources required by the graph. Most applications can start with the compiler's automatic partitioning and inspect the result when they need to understand placement or a compile failure.

The sequence is simple: compose the graph, compile it, check that the plan was accepted, and inspect the parts that matter to the application. Building Graphs and Queue Policies and Fan-Out, Fan-In, Mux, and Demux introduce the graph that the compiler receives.

Compile once, inspect the accepted plan

Compile after all operators, connections, contracts, and initial configuration have been added. Use holoscan::compile(), check plan.ok(), inspect diagnostics with plan.json(), and start an accepted plan with run_async():

const auto plan = holoscan::compile(graph);
if (!plan.ok()) {
  std::cerr << plan.json() << '\n';
  return 1;
}

holoscan::RunSession session = holoscan::run_async(plan);

The application should check plan.ok() before starting the run. plan.json() is useful when a graph is rejected because a port, queue, contract, or resource requirement is incompatible.

The exact run call depends on the application structure. Running and Stopping covers run control; this page focuses on the plan produced before execution.

Let the compiler partition the graph

With no authored partitions, the compiler examines the connected graph and groups operators into execution partitions. The graph's edges, branches, joins, device requirements, and activation rules all contribute to the result.

Automatic partitioning is a good starting point. It keeps the graph description focused on data flow and lets the compiler choose a valid layout. Different partitions can make progress concurrently when their inputs and resources allow it, but partition count alone does not determine throughput.

Author a boundary when placement matters

Create a named partition when a group of operators needs a clear placement boundary or a stable name for inspection. This example keeps preprocessing together and lets the compiler place the remaining operators:

graph.partition("preprocess").add(source).add(converter);
graph.auto_partition_remaining();

Graph::partition() adds the named group. auto_partition_remaining() asks the compiler to place operators that were not included in an authored group.

Use a manual boundary only when the application has a reason to own it. Each executable operator must belong to one accepted partition. For a single named group containing the complete graph, use include_all().

Inspect the groups that were accepted

After compilation succeeds, inspect the partitions when you need to explain a placement decision or compare two graph versions:

for (const auto& partition : plan.partitions()) {
  std::cout << "lane=" << partition.execution_lane() << '\n';
  for (const auto& path : partition.operator_paths()) {
    std::cout << "  " << path << '\n';
  }
}

plan.partitions() gives the accepted groups. execution_lane() identifies the serial execution lane for a group, and operator_paths() lists the operators in that group.

Use this information to answer practical questions: did the intended operators stay together, did a branch create an additional group, and did a graph change alter the accepted layout? For a deeper explanation of a boundary, the plan also exposes partition_decisions() and partition_placements().

Know when a new plan is required

The accepted plan governs the running graph. A supported scalar parameter update can change operator behavior within that plan, but adding an operator, changing an edge, increasing a compiled Tensor bound, or changing a placement request requires a new graph compilation. Dynamic Graph Behavior explains which changes can stay inside an existing plan.

After a topology or contract change, compile again and compare the new partitions and resource records before drawing performance conclusions. Keep the plan information associated with the application binary and the external endpoint catalog used to run it.

Related Examples

Automatic partitioning source composes a diamond graph and prints accepted partitions.
Graph composition source shows connection and queue decisions that affect the plan.
CUDA Green Context Example shows a case where a named partition supports an explicit placement request.

Continue the flow

Rates, Queue Sizing, and Derived Tensor Capacity explains the capacity evidence recorded with the plan.
Dynamic Graph Behavior explains updates that stay inside a plan and changes that need recompilation.

Further Reading

holoscan::compile() definition declares compilation entry points and options.
Execution plan definitions describe accepted partitions and inspection records.
NVIDIA Holoscan SDK User Guide provides the official application and deployment reference.

❯