Writing Tensor Operators

Tensor processing adds storage requirements to the read-and-publish sequence. The operator declares the input representation and output capacity, reads the selected sample, acquires output storage, and publishes the completed result.

The host transform below scales a fixed 64 × 3 measurement array. The CUDA version follows the same ownership sequence, with writes and readiness ordered on the runtime stream.

Follow the acquire, fill, commit, publish path

Use this sequence whenever an operator creates a new Tensor result:

Stage	User decision	Result
Acquire	Receive the selected sample and check its representation, shape, readiness, and metadata.	The input is safe to read for this activation.
Allocate	Call `allocate_tensor()` using the output contract's admitted bounds.	A runtime-owned output loan is available, or allocation fails cleanly.
Fill	Use `write_host_and_commit()` for a host transform or `write(stream)` for CUDA work.	The output bytes are written and the write phase is committed.
Publish	Call `Output<Tensor>::emit(loan)` after commitment.	Downstream operators receive the completed Tensor and its declared metadata.

Commit is not publication. A failed acquisition, allocation, write, commitment, or publication must return an error from compute(). Allocation Ownership and Tensor Pools explains why a retained loan affects capacity; the example below shows the application code.

Declare the Representation and Capacity

Use TensorInputSpec for consumer requirements and TensorOutputSpec for producer promises. The TensorRepresentation names memory kind, element type, rank, and stride policy. A field omitted from that representation remains unconstrained.

A TensorOutputStorage::kRuntimePool output needs finite byte-span and rank capacity as well as concrete placement. Those bounds size the admitted storage; they do not promise an exact shape. Check the actual dimensions when the algorithm needs a specific geometry. For strided data, account for the addressable span instead of treating logical element count as the whole allocation.

Conversion Support

For same-process raw Tensor edges, Holoscan SDK rejects an effective non-default memory-conversion policy, including kDisable. Leave that policy omitted for compatible storage. Automatic host/device converter insertion is unavailable; add an explicit adaptation operator when the computation needs different storage. See the compiler check.

For a CUDA output, the application also supplies device placement before compiling; see Tensor Contracts and Memory Types. Changing a memory-kind enum does not make a CPU loop able to dereference device storage.

Select the Input that Drives the Transform

A one-input transform can use OnEach{input} to process each accepted sample on that input. This is the contract used by the SDK's Tensor inspectors. Receive the admitted sample and check the result before reading its Tensor; copying a handle retains the allocation and read claim without copying its numerical data.

More involved fusion operators need an explicit temporal selection and missing-input policy. Exact selects at the frozen event-time reference; it is not a synonym for ordinary FIFO reception. Combining OnAnyInput with an explicit Exact selector without a missing policy fails presence validation. Follow Temporal Contracts and Input Selection when adding independently timed inputs.

Follow a Checked Host Transform

MeasurementScale is an operator-class fragment for an application that supplies an upstream compatible host Tensor, a downstream consumer, graph wiring, and run control. It needs <cmath>, <cstddef>, <cstdint>, <utility>, and the SDK headers for Tensor, ExecutionContext, Operator, OperatorSpec, temporal contracts, and Tensor output loans. The complete SDK programs below provide graph and lifecycle examples.

The input contract requires a contiguous host float32 array. The compute method checks host placement, exact shape, byte count, typed pointer, and finite values. It rejects invalid samples, acquires a separate result allocation, and writes the scaled values directly into that output.

class MeasurementScale final : public holoscan::Operator<> {
 public:
  holoscan::Input<holoscan::Tensor> input;
  holoscan::Output<holoscan::Tensor> output;

  void setup(holoscan::OperatorSpec& spec) override {
    const holoscan::TensorRepresentation representation{
        .memory_kind = holoscan::MemoryKind::kHost,
        .dtype = DLDataType{kDLFloat, 32U, 1U},
        .rank = 2U,
        .strides = holoscan::TensorStridePolicy::kContiguousOnly,
    };
    spec.input(input, "input")
        .expects_tensor(holoscan::TensorInputSpec{
            .representation = representation,
            .bounds = holoscan::tensor_bounds(kBytes),
        });
    spec.output(output, "output")
        .max_emits_per_compute(1U)
        .produces_tensor(holoscan::TensorOutputSpec{
            .representation = representation,
            .bounds = holoscan::tensor_bounds(kBytes),
            .storage = holoscan::TensorOutputStorage::kRuntimePool,
        });
  }

  holoscan::Contract contract() const override {
    holoscan::Contract result;
    result.trigger(holoscan::OnEach{input});
    return result;
  }

  holoscan::expected<void, holoscan::Error> compute(
      holoscan::ExecutionContext&) override {
    auto sample = input.receive();
    if (!sample) {
      return holoscan::make_unexpected(std::move(sample).error());
    }
    const auto& tensor = sample->data;
    if (tensor.data() == nullptr || tensor.device().device_type != kDLCPU) {
      return holoscan::make_unexpected(
          holoscan::Error{holoscan::ErrorCode::kInvalidArgument});
    }
    const auto shape = tensor.shape_span();
    if (shape.size() != 2U || shape[0] != 64 || shape[1] != 3 ||
        !tensor.is_contiguous() ||
        tensor.nbytes() != static_cast<std::int64_t>(kBytes) ||
        (sample->metadata.flags & holoscan::SampleFlags::kInvalid) !=
            holoscan::SampleFlags::kNone) {
      return holoscan::make_unexpected(
          holoscan::Error{holoscan::ErrorCode::kInvalidArgument});
    }
    auto ready = tensor.synchronize_for_host();
    if (!ready) {
      return holoscan::make_unexpected(std::move(ready).error());
    }
    const float* source = tensor.data_as<float>();
    if (source == nullptr) {
      return holoscan::make_unexpected(
          holoscan::Error{holoscan::ErrorCode::kInvalidArgument});
    }
    auto loan = output.allocate_tensor(holoscan::TensorLoanRequest{
        .shape = shape,
        .dtype = DLDataType{kDLFloat, 32U, 1U},
    });
    if (!loan) {
      return holoscan::make_unexpected(std::move(loan).error());
    }
    auto committed = loan->write_host_and_commit(
        [source](const holoscan::TensorOutputWriteGuard& writer)
            -> holoscan::expected<void, holoscan::Error> {
          float* destination = writer.data_as<float>();
          if (destination == nullptr || writer.byte_size() != kBytes) {
            return holoscan::make_unexpected(
                holoscan::Error{holoscan::ErrorCode::kInvalidArgument});
          }
          for (std::size_t i = 0; i < kElements; ++i) {
            if (!std::isfinite(source[i])) {
              return holoscan::make_unexpected(
                  holoscan::Error{holoscan::ErrorCode::kInvalidArgument});
            }
            destination[i] = source[i] * 0.5F;
          }
          return {};
        });
    if (!committed) {
      return holoscan::make_unexpected(std::move(committed).error());
    }
    holoscan::EmitOptions options{};
    const auto capture = sample->metadata.capture_time();
    if (capture.valid()) {
      options.capture_time = capture;
    }
    if (sample->metadata.frame_id != 0U) {
      options.frame_id = sample->metadata.frame_id;
    }
    if (sample->metadata.trace_id != 0U) {
      options.trace = holoscan::TraceContext::root(sample->metadata.trace_id);
    }
    options.flags = sample->metadata.flags;
    return output.emit(std::move(*loan), options);
  }

 private:
  static constexpr std::size_t kElements = 64U * 3U;
  static constexpr std::size_t kBytes = kElements * sizeof(float);
};

The received sample stays alive throughout the fill callback, keeping source valid. write_host_and_commit() acquires the sole writer, invokes the callback, and commits only if the callback succeeds. A returned error abandons that write; no partially filled result is emitted. Its borrowed guard and pointer must not escape the callback.

The final emit() attempts publication separately from commitment. The example propagates every failure, including pool exhaustion and backpressure. A streaming application can instead choose a documented drop/retry policy, but a terminal loan emit consumes the loan on both success and error; retry needs a fresh allocation.

Preserve the Measurement's Meaning

The transform's numerical result still describes the original measurement. It keeps a valid acquisition time, nonzero frame identity and trace identity, and semantic flags through EmitOptions. Missing capture time stays missing. A new output receives its own runtime publication sequence and time, so consumers can distinguish acquisition from this transform's publication.

Tensor::data_as<T>() checks typed interpretation and alignment but does not establish host accessibility, readiness, or the algorithm's exact shape. The host contract and checked synchronize_for_host() supply the corresponding requirements here. The write guard's data_as<T>() is a pointer cast; its safety relies on the admitted loan request and the checked span.

Move the Compute Work onto CUDA

For device storage, borrow with allocate_tensor(), acquire write(context.cuda_stream()), enqueue the operation on that same runtime stream, check the CUDA return/launch result, and then call commit(). Publication remains explicit. A CUDA kernel must implement the transform; the host loop above needs a device-compatible implementation before it can process a device pointer.

The SDK's PatternSource::emit_one_frame() shows checked allocation, stream writing, cudaMemsetAsync, commitment, and publication. Its real implementation is preferable to a recipe with an undefined kernel hook. Check CUDA enqueue/launch errors immediately and surface asynchronous failures at the appropriate completion boundary; a successful launch alone does not prove execution succeeded.

A graph input is prepared for work on the runtime stream before compute. Private streams need an explicit readiness wait, registered release stream, and retained owner until work completes. These responsibilities are followed in Zero-Copy Data Flow and CUDA Stream Ordering.

Keep Scratch and External Interfaces Deliberate

Bound reusable private scratch independently from graph outputs. Its allocation, outstanding work, reuse, and final release belong to the operator or its retained resource owner. Use Allocation Ownership and Tensor Pools when planning capacity, and keep private scratch separate from graph outputs when deciding where to write results.

At a CPU or external interface, expose the real copy and completion obligation. Host synchronization does not turn device memory into host bytes. A readback sink needs bounded destination storage and checked transfer/completion; the SDK inspection tap in Tensors in Depth provides that full path. For backend-owned storage, follow External Memory: VPI, PVA, NvSci, and DLPack instead of inventing an owner-only factory overload.

Related examples and concepts

Examples

Tensors 101 demonstrates host loans, both writer forms, read-only reception, and pool exhaustion.
Tensors in Depth adds device placement, CUDA readiness, private-stream readback, and retirement.
Complete Operator Patterns follows these programs from graph construction to checked shutdown.

Concepts

Tensor Contracts and Memory Types defines the contract vocabulary.
Sensor Payloads and Schemas explains acquisition time, correlation, and sensor-specific metadata.
Copy Paths and Memory Conversion distinguishes sharing, transfer, and adaptation.

Source

Tensor contract definitions define the accepted representation, bounds, and storage.
Tensor output loan definitions define writing, commitment, and stream ordering.
Tensors 101 in the NVIDIA Holoscan SDK documentation provides the official Tensor workflow that this page extends.

❯