AI Based Object Redaction - Examples - Library Examples
| AI Based Object Redaction |
|---|
|
| Overview |
| Getting Started |
| GStreamer Plugin |
| Examples |
| Performance |
| Troubleshooting |
| FAQ |
| Contact Us |
 | Note: Please refer to the API Documentation for detailed documentation. |
In this section will be explained an example for face redaction running on GPU.
The first thing you want to do is to include the library header as follows:
#include <rd/redaction.hpp>
Backend
First, we create the backend object. The backend provides factories to create the redaction algorithm and buffers. It provides the user the ability to select the desired backend for the execution of the algorithm for object redaction. The backend could be CPU or GPU.
std::shared_ptr<rd::IBackend> backend = std::make_shared<rd::gpu::Backend>();
In case the input buffer is not already in GPU memory, we also need to create a CPU backend to allocate the buffer in CPU memory.
std::shared_ptr<rd::IBackend> cpu_backend = std::make_shared<rd::cpu::Backend>();
Get algorithm
The GetAlgorithm method is used to obtain the redaction algorithm to process the input buffer.
std::shared_ptr<rd::IRedaction> algorithm = backend->GetAlgorithm();
Get Model
The getModel method is used to obtain the AI model that will be used for the detection of the desired object. In this case: faces.
std::shared_ptr<rd::IModel> model = backend->getModel(rd::Model::FACE_DETECTION);
Buffers
Buffers are the structures used to manipulate and load the data corresponding to the video frames. A buffer consists of a resolution and a format.
Resolution
Resolution is a structure that consists of two parameters: width and height. Here we define the input resolution as 1080x720.
#define INPUT_WIDTH 1080 #define INPUT_HEIGHT 720 rd::Resolution input_resolution = rd::Resolution(INPUT_WIDTH, INPUT_HEIGHT);
Format
Format is an enumeration of values for the supported formats, which are: RGBA, RGB, GREY and YUV. Here we define the input format as YUY2.
rd::Format input_format = rd::Format::YUY2;
Allocate Buffers
With the resolution and formats defined, the buffer objects can be created.
With the cpu backend create an input and output buffers in CPU memory with the input video/image resolution and format. The input buffer must contain the image/frame data deploy in an array containing a pointer to each color component of the data.
std::shared_ptr<rd::io::IBuffer> input = backend_cpu->getBuffer(imageData, input_resolution, input_format); std::shared_ptr<rd::io::IBuffer> output_final = backend_cpu->getBuffer(input_resolution, input_format);
With the gpu backend create an input and output buffers in GPU memory with the input video/image resolution and format.
std::shared_ptr<rd::io::IBuffer> input_gpu = backend->getBuffer(input_resolution, input_format); std::shared_ptr<rd::io::IBuffer> output = backend->getBuffer(input_resolution, input_format);
The CPU input buffer must be moved to GPU memory when using GPU, to accomplish this, use the copyFromHost method to upload the input buffer to GPU memory.
input_gpu->copyFromHost(input);
Redaction Algorithm
The Object Redaction library comprises the following stages: detect, track (optional), and redact. These stages can be performed in a single step using the apply method or in a step-by-step process.
The Object Redaction library uses a vector of a structure Rectangle to save the detected and tracked faces coordinates in an image for the redaction algorithm to modify the output buffer. This vector must be initialized before performing the detect stage.
std::vector<rd::Rectangle> faces;
Step-by-step
- The first step is to detect the faces in the input image and save the coordinates in the vector of rectangles.
algorithm->detect(model, input_gpu, &faces);
| NOTE: Since the detection model usually works with a fixed resolution and format, an internal conversion to the resolution and format expected by the model. This is transparent so the user doesn't have to worry about any conversion, however, a convert method is also available in case the user wants to do this conversion before calling the detect method. |
algorithm->convert(input_gpu, input_convert);
- The second and optional step would be to perform tracking (this is a work in progress)
- The final step is to redact the detected faces in the given coordinates.
algorithm->redact(input_gpu, output, faces, rd::RedactionAlgorithm::BLURRING);
Apply method
To apply the redaction algorithm in a single step use the apply method set by the algorithm. This method
rd::IRedaction::apply(backend, model, input_gpu, output, &faces, rd::RedactionAlgorithm::BLURRING);
Download buffer to CPU memory
When using GPU the output buffer must be moved back to CPU memory. To accomplish this, use the copyToHost method to download the output buffer to CPU memory.
output->copyToHost(output_final);
The output final buffer contains the modified image where the detected faces have been redacted.
Full example
The full example script should look like:
#include <rd/redaction.hpp>
#include <unistd.h>
#include <fstream>
#include <iostream>
#include <memory>
#include <string>
#define INPUT_WIDTH 1080
#define INPUT_HEIGHT 720
#define INPUT_BPP 2
static void save_buffer(std::shared_ptr<rd::io::IBuffer> buffer,
std::string name) {
/*Save the buffer*/
std::vector<unsigned char*> data = buffer->data();
uint size = buffer->stride()[0] * buffer->size().height;
FILE* file = fopen(name.c_str(), "wb");
fwrite(data[0], size, 1, file);
fclose(file);
}
int main() {
/* Open the image file using fstream */
std::ifstream file(SEVEN_FACES, std::ios::binary);
if (!file.is_open()) {
std::cerr << "Error: Unable to open the image file." << std::endl;
return -1;
}
/* Determine the file size */
int file_size = INPUT_WIDTH * INPUT_HEIGHT * INPUT_BPP;
/* Read the image data into a vector */
unsigned char* data_ptr = new unsigned char[file_size];
std::vector<unsigned char*> imageData;
file.read(reinterpret_cast<char*>(data_ptr), file_size);
imageData.push_back(data_ptr);
/* Create GPU Backend */
std::shared_ptr<rd::IBackend> backend = std::make_shared<rd::gpu::Backend>();
/* Create CPU backend to save the final image */
std::shared_ptr<rd::IBackend> backend_cpu =
std::make_shared<rd::cpu::Backend>();
/* Get Algorithm */
std::shared_ptr<rd::IRedaction> algorithm = backend->getAlgorithm();
std::shared_ptr<rd::IModel> model =
backend->getModel(rd::Model::FACE_DETECTION);
/* Buffers */
rd::Resolution input_resolution = rd::Resolution(INPUT_WIDTH, INPUT_HEIGHT);
rd::Format input_format = rd::Format::YUY2;
/* Allocate Buffers */
std::shared_ptr<rd::io::IBuffer> input =
backend_cpu->getBuffer(imageData, input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> output_final =
backend_cpu->getBuffer(input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> output =
backend->getBuffer(input_resolution, input_format);
std::shared_ptr<rd::io::IBuffer> input_gpu =
backend->getBuffer(input_resolution, input_format);
/*Copy input buffer to GPU*/
input_gpu->copyFromHost(input);
/* Detect Faces */
std::vector<rd::Rectangle> faces;
algorithm->detect(model, input_gpu, &faces);
/* Print out detected faces */
std::cout << faces.size() << std::endl;
for (size_t i = 0; i < faces.size(); i++) {
std::cout << faces[i] << std::endl;
}
/* Redact detected faces */
algorithm->redact(input_gpu, output, faces, rd::RedactionAlgorithm::BLURRING);
/* Download buffer back to CPU memory */
output->copyToHost(output_final);
/* Save redacted image */
save_buffer(output_final, "output_final");
std::cout << "Exit!!" << std::endl;
return 0;
}