DeepStream 7.0 LIDAR Components
![]() |
|
DS3D Framework
Deepstream 7.0 has implemented a 3D framework, interfaces and custom-libs for DeepStream-3D processing. This DS3D framework adds support for LIDAR and RADAR with BEVFusion and V2X models. These interfaces are capable of different types of data fusion like: dataloader, datafilter and datarender.
DS3D framework recognizes ds3d/datamap as the data format for data buffers in GStreamer plugins. The type ds3d/datamap are key-value pairs where key is a string and value, a pointer, structure or tensor frame to data. DS3D Framework source code can be find in the path /opt/nvidia/deepstream/deepstream/sources/libs/ds3d
.
The 3D Lidar Sensor Fusion example use a series of DS3D components configured in YAML format to perform the following pipeline.

Follow the DeepStream 7.0 Lidar example guide to check out and test the Lidar example.
Each component is configured in YAML format and implemented on a custom library of types: ds3d::dataloader, ds3d::databridge. s3d::datamixer and ds3d::datafilter
.
Load LIDAR data
To load LIDAR data is required to use the data type type: ds3d::dataloader
which is implemented in the library libnvds_lidarfileread.so.
or libnvds_3d_dataloader_realsense.so
. The library can be found in the path: /opt/nvidia/deepstream/deepstream/sources/libs/ds3d/dataloader/lidarsource
.
LIDAR info can be load from a yaml file and output as a ds3d/datamap
data type using file loader custom library or read lidar data frame by frame and create a ds3d::datamap
per frame.
You can check more details in the Custom Dataloader libnvds_lidarfileread Configuration Specifications
LIDAR preprocess
LiDAR data preprocess generates features, coordinates, and point numbers (feats/coords/N) tensors. Which can be achieved with the library libnvds_3d_lidar_preprocess_datafilter.so
which process data for v2x sensor fusion model.
This library can be found in the path: /opt/nvidia/deepstream/deepstream/sources/libs/ds3d/datafilter/lidar_preprocess/
and you can check specifications on Custom Datafilter libnvds_3d_lidar_preprocess_datafilter Specifications
This ca also be perform with the gstreamer element nvds3dfilter with the corresponding configuration file.
LIDAR Bridge
This bridge is designed to loads custom libraries for data conversion to and from DS3D. For example to convert 2D video (video/x-raw(memory:NVMM) into ds3d::datamap
format use the library libnvds_3d_video_databridge.so
of type ds3d::databridge
.
This ca also be perform with the gstreamer element nvds3dbridge with the corresponding configuration file.
Data and LIDAR Mixer
The ds3d::datamixer
combines video data 2D and LIDAR into single data of type ds3d::datamap
.
This is implemented in the library libnvds_3d_multisensor_mixer.so
and also can be performed with the gstreamer element nvds3dmixer with the corresponding configuration file.
Data Alignment
The library libnvds_3d_alignment_datafilter.so
of type ds3d::datafilter
applies a series of transformations to align the LiDAR data with the camera image coordinate system.
This ca also be perform with the gstreamer element nvds3dfilter with the corresponding configuration file.
Data Inference
This library libnvds_tritoninferfilter.so
of type ds3d::datafilter
executes data inference using Triton Inference Server. It supports both Triton CAPI and gRPC modes. Custom pre-processing and post-processing might be required depending on the specific inference task.
You can check more details in the libnvds_tritoninferfilter Configuration Specifications
Data Render
You can render a 3D scene using OpenGL ES (GLES) with various elements such as textures, LiDAR points, bounding boxes within a single window. This is implemented in the library libnvds_tritoninferfilter.so
of type ds3d::datafilter
.
You can check more details in the Custom datarender libnvds_3d_gles_ensemble_render Configuration Specifications
Contact Us
For direct inquiries, please refer to the contact information available on our Contact page. Alternatively, you may complete and submit the form provided at the same link. We will respond to your request at our earliest opportunity.
Links to RidgeRun Resources and RidgeRun Artificial Intelligence Solutions can be found in the footer below.