Building Graphs and Queue Policies

A graph describes how operators exchange data. Connect an output to an input, compile the graph into an execution plan, and then run that plan.

The connection also determines how much input history can be retained while the receiving operator waits for its next activation. Choose the queue policy together with the temporal contract that selects the data.

Connect the operators

Assume source, transform, and sink are operator handles with compatible ports. Use Graph::add_flow() to connect them:

graph.add_flow(source->output, transform->input);
graph.add_flow(transform->output, sink->input);

The output and input types must agree. Each input has one upstream producer. To combine several producers, declare separate named inputs on a mux or fusion operator and connect one producer to each input. Fan-Out, Fan-In, Mux, and Demux covers those layouts.

Compile before running

After all connections and operator settings are in place, use holoscan::compile() to create an ExecutionPlan. Check plan.ok() before passing it to run():

const holoscan::ExecutionPlan plan = holoscan::compile(graph);
if (!plan.ok()) {
  std::cerr << plan.json() << '\n';
  return 1;
}
return holoscan::run(plan);

plan.ok() tells you whether the graph produced an accepted plan. Compilation checks port compatibility, temporal requirements, clocks, placement, and finite resource requests. Inspect the plan diagnostics before starting the run when compilation fails.

Once the plan is accepted, the graph connections and queue settings belong to that plan. Change the graph or its connection options, then compile a new plan.

Choose how a queue behaves

A queue keeps accepted input samples between publication and selection. The policy controls what happens when the queue reaches its finite depth:

Policy	Behavior	Use it when
`kBlocking`	Preserve accepted queued samples and apply backpressure when the queue is full.	Every accepted sample matters and the producer can tolerate waiting.
`kDropOldest`	Keep newer samples by replacing the oldest unread sample when the queue is full.	Freshness matters more than processing every sample.

A queue policy does not choose the sample for compute(). The receiving operator's temporal contract still decides which queued values are selected. A windowed or aligned contract needs the history required to form its selection and cannot use a policy that discards that history.

Set an explicit depth when the application needs one

Start with the default queue settings when the graph has no special history requirement. Set edge options when the application needs a known depth or a specific overflow policy:

graph.add_flow(
    source->output,
    transform->input,
    holoscan::ConnectionOptions{
        .queue_policy = holoscan::QueuePolicy::kDropOldest,
        .queue_depth = 1U,
    });

ConnectionOptions applies to that graph edge. queue_depth describes logical input history. It does not describe Tensor bytes or the total number of samples that may remain retained by readers and output pools.

Keep the depth large enough for the selected temporal contract and the expected producer burst. Rates, Queue Sizing, and Derived Tensor Capacity explains how rates, wait horizons, retained readers, and Tensor capacity affect that decision.

Separate topology from placement

Connections describe data flow. The planner decides where the accepted graph executes. Use Automatic Partitioning and Plan Inspection when you need to inspect the execution groups or an authored placement decision.

A new branch can add another reader, queue, and Tensor lifetime. Check Fan-Out, Fan-In, Mux, and Demux before adding branches to a memory-sensitive pipeline.

Continue the flow

Temporal Contracts and Input Selection explains when an operator becomes eligible and which samples it receives.
Fan-Out, Fan-In, Mux, and Demux explains branching, merging, and value-based routing.
Automatic Partitioning and Plan Inspection explains how to inspect the accepted plan.
Writing a Minimal Operator combines operators, connections, compilation, and run control in one application.

Related Examples

First pipeline example shows the basic graph construction and run path.
Graph composition example shows connections and per-edge queue choices.

Further Reading

Graph connections and graph-owned operator composition
Queue policies and edge options
Complete graph construction and run example
NVIDIA Holoscan SDK User Guide provides the official graph and queue reference.

❯