RidgeRun Developer Wiki

AI Based Object Redaction/Examples/Library Examples: Difference between revisions

Page Discussion

AI Based Object Redaction/Examples/Library Examples (view source)

Revision as of 23:18, 29 December 2023

7,715 bytes added ,  29 December 2023
no edit summary
No edit summary
No edit summary
Line 7: Line 7:
First, we create the backend object. The backend provides factories to create the redaction algorithm and buffers. It provides the user the ability to select the desired backend for the execution of the algorithm for object redaction. The backend could be CPU or GPU.  
First, we create the backend object. The backend provides factories to create the redaction algorithm and buffers. It provides the user the ability to select the desired backend for the execution of the algorithm for object redaction. The backend could be CPU or GPU.  


<syntaxhighlight lang="cpp">
<syntaxhighlight lang=cpp>
std::shared_ptr<rd::IBackend> backend = std::make_shared<rd::gpu::Backend>();
std::shared_ptr<rd::IBackend> backend = std::make_shared<rd::gpu::Backend>();
/* HOLA */
</syntaxhighlight>
</syntaxhighlight>


Line 35: Line 34:


==== Resolution ====
==== Resolution ====
Resolution is a structure that consists of two parameters: width and height. The resolution of the input video/image may differ from the resolution accepted by the AI model. We should create both of these resolutions that will b
Resolution is a structure that consists of two parameters: width and height. The resolution of the input video/image may differ from the resolution accepted by the AI model. We should create both of these resolutions with the AI model resolution as 640x480 in case of ONNX face detector.


<syntaxhighlight lang=cpp>
#define CONVERT_WIDTH 640
#define CONVERT_HEIGHT 480
rd::Resolution input_resolution = rd::Resolution(INPUT_WIDTH, INPUT_HEIGHT);
rd::Resolution convert_resolution = rd::Resolution(CONVERT_WIDTH, CONVERT_HEIGHT);
</syntaxhighlight>
==== Format ====
Format is an enumeration of values for the supported formats, which are: RGBA, RGB, GREY and YUV. The format of the input video/image may differ from the format accepted by the AI model. We should create both of these formats with the AI model format as RGB in case of ONNX face detector.
<syntaxhighlight lang=cpp>
rd::Format format = rd::Format::RGB;
rd::Format input_format = rd::Format::INPUT_FORMAT;
</syntaxhighlight>
==== Allocate Buffers ====
With the resolution and formats defined, the buffer objects can be created.
With the cpu backend create an input and output buffers in CPU memory with the input video/image resolution and format. The input buffer must contain the image/frame data deploy in an array containing a pointer to each color component of the data.
<syntaxhighlight lang=cpp>
std::shared_ptr<rd::io::IBuffer> input = backend_cpu->getBuffer(imageData, input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> output_final = backend_cpu->getBuffer(input_resolution, input_format);
</syntaxhighlight>
With the gpu backend create an input and output buffers in GPU memory with the input video/image resolution and format. Also for the AI model to work properly create a GPU memory buffer with the supported resolution and format.
<syntaxhighlight lang=cpp>
std::shared_ptr<rd::io::IBuffer> input_gpu = backend->getBuffer(input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> output = backend->getBuffer(input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> input_convert = backend->getBuffer(convert_resolution, format);
</syntaxhighlight>
The CPU input buffer must be allocated to GPU memory when using GPU, to accomplish the allocation use the <code>copyFromHost</code> method to upload the input buffer to GPU memory.
<syntaxhighlight lang=cpp>
input_gpu->copyFromHost(input);
</syntaxhighlight>


=== Redaction Algorithm ===
The Object Redaction library its compound by the stages: convert, detect, track (optional) and redact. This stages can be perform in a single step using the apply method or in a step-by-step process.
The Object Redaction library use a vector of a structure Rectangle to save the detected and tracked faces coordinates in a images for the redaction algorithm to modify the output buffer. This vector must be initialize before performing the detect stage.
<syntaxhighlight lang=cpp>
std::vector<rd::Rectangle> faces;
</syntaxhighlight>
==== Step-by-step ====
* First step is to preprocess the input image to be accepted by the AI model.
<syntaxhighlight lang=cpp>
algorithm->convert(input_gpu, input_convert);
</syntaxhighlight>
* Second step is to detect the faces in the preprocess image and save the coordinates in the vector of rectangles.
<syntaxhighlight lang=cpp>
algorithm->detect(model, input_convert, &faces);
</syntaxhighlight>
* The final step is to redact the detected faces in the given coordinates.
<syntaxhighlight lang=cpp>
algorithm->redact(input_gpu, output, faces, rd::RedactionAlgorithm::BLURRING);
</syntaxhighlight>
==== Apply method ====
To apply the redaction algorithm in a single step use the apply method set by the algorithm. This method
<syntaxhighlight lang=cpp>
rd::IRedaction::apply(backend, model, input_gpu, output, &faces, rd::RedactionAlgorithm::BLURRING);
</syntaxhighlight>
=== Download buffer to CPU memory ===
When using GPU the output buffer must be allocated to CPU memory. To accomplish the allocation use the <code>copyToHost</code> method to download the output buffer to CPU memory.
<syntaxhighlight lang=cpp>
output->copyToHost(output_final);
</syntaxhighlight>
The output final buffer contains the modified image where the detected faces have been redact.
=== Full example ===
The full example script should look like:
<syntaxhighlight lang=cpp>
#include "rd/common/datatypes.hpp"
#include "rd/common/ibackend.hpp"
#include "rd/common/ibuffer.hpp"
#include "rd/common/imodel.hpp"
#include "rd/common/runtimeerror.hpp"
/*Backend*/
#include "cpu/backend.hpp"
#include "gpu/backend.hpp"
/*io*/
#include "cpu/onnxfacedetect.hpp"
#include "cpu/redaction.hpp"
#include "gpu/onnxfacedetect.hpp"
#include "gpu/redaction.hpp"
#include "rd/common/ivideoinput.hpp"
#include "rd/io/v4l2/v4l2capture.hpp"
#include <unistd.h>
#include <fstream>
#include <iostream>
#include <memory>
#include <string>
#define INPUT_WIDTH 1080
#define INPUT_HEIGHT 720
#define INPUT_BPP 2
#define CONVERT_WIDTH 640
#define CONVERT_HEIGHT 480
static void save_buffer(std::shared_ptr<rd::io::IBuffer> buffer,
                        std::string name) {
  /*Save the buffer*/
  std::vector<unsigned char*> data = buffer->data();
  uint size = buffer->stride()[0] * buffer->size().height;
  FILE* file = fopen(name.c_str(), "wb");
  fwrite(data[0], size, 1, file);
  fclose(file);
}
int main() {
  /* Open the image file using fstream */
  std::ifstream file(SEVEN_FACES, std::ios::binary);
  if (!file.is_open()) {
    std::cerr << "Error: Unable to open the image file." << std::endl;
    return -1;
  }
  /* Determine the file size */
  int file_size = INPUT_WIDTH * INPUT_HEIGHT * INPUT_BPP;
  /* Read the image data into a vector */
  unsigned char* data_ptr = new unsigned char[file_size];
  std::vector<unsigned char*> imageData;
  file.read(reinterpret_cast<char*>(data_ptr), file_size);
  imageData.push_back(data_ptr);
  /* Create GPU Backend */
  std::shared_ptr<rd::IBackend> backend = std::make_shared<rd::gpu::Backend>();
  /* Create CPU backend to save the final image */
  std::shared_ptr<rd::IBackend> backend_cpu =
      std::make_shared<rd::cpu::Backend>();
  /* Get Algorithm */
  std::shared_ptr<rd::IRedaction> algorithm = backend->getAlgorithm();
  std::shared_ptr<rd::IModel> model =
      backend->getModel(rd::Model::FACE_DETECTION);
  /* Buffers */
  rd::Resolution input_resolution = rd::Resolution(INPUT_WIDTH, INPUT_HEIGHT);
  rd::Resolution convert_resolution =
      rd::Resolution(CONVERT_WIDTH, CONVERT_HEIGHT);
  rd::Format format = rd::Format::RGB;
  rd::Format input_format = rd::Format::YUY2;
  /* Allocate Buffers */
  std::shared_ptr<rd::io::IBuffer> input =
      backend_cpu->getBuffer(imageData, input_resolution, input_format);
  std::shared_ptr<rd::io::IBuffer> output_final =
      backend_cpu->getBuffer(input_resolution, input_format);
  std::shared_ptr<rd::io::IBuffer> input_convert =
      backend->getBuffer(convert_resolution, format);
  std::shared_ptr<rd::io::IBuffer> output =
      backend->getBuffer(input_resolution, input_format);
  std::shared_ptr<rd::io::IBuffer> input_gpu =
      backend->getBuffer(input_resolution, input_format);
  input_gpu->copyFromHost(input);
  /* Preprocess image to be accepted by the face recognition model */
  algorithm->convert(input_gpu, input_convert);
  /* Detect Faces */
  std::vector<rd::Rectangle> faces;
  algorithm->detect(model, input_convert, &faces);
  /* Print out detected faces */
  std::cout << faces.size() << std::endl;
  for (size_t i = 0; i < faces.size(); i++) {
    std::cout << faces[i] << std::endl;
  }
  /* Redact detected faces */
  algorithm->redact(input_gpu, output, faces, rd::RedactionAlgorithm::BLURRING);
  /* Download buffer to CPU memory */
  output->copyToHost(output_final);
  /* Save redacted image */
  save_buffer(output_final, "output_final");
  std::cout << "Exit!!" << std::endl;
  return 0;
}
</syntaxhighlight>


<noinclude>
<noinclude>
{{AI Based Object Redaction/Foot|Examples|Examples/GStreamer Pipelines}}
{{AI Based Object Redaction/Foot|Examples|Examples/GStreamer Pipelines}}
</noinclude>
</noinclude>
Kcarvajal
335

edits

Retrieved from "https://developer.ridgerun.com/wiki/index.php/Special:MobileDiff/52205"