Getting Started Guide for RidgeRun iMX6 Graphics-Server
(Redirected from Getting Started Guide for Graphics-Server)
 |
|
|
Template:Eval SDK Download, Demo Image download and Contact Us buttons |
Keywords: IMX6 Nitrogen, OpenGL, GPU, Graphics-Server, rendering, hardware acceleration
Demo Video
Introduction
Graphics-Server is an API based on OpenGL 2.0 that allows you to easily make GPU hardware accelerated video, image, and text renderings. It allows you to create graphic objects such as planes or cubes with different textures, for example pictures and USB video. Also, RidgeRun's Graphics-Server gives you the ability to customize the created graphics objects and add animation effects.
RidgeRun provides a C API header and library for the interaction between your application and the Graphics-Server. The interprocess communication is done via DBUS.
Key Features
General:
- Based on OpenGL 2.0.
- Provides a C API for the interaction via DBUS with the graphics-server.
- The dimensions and position of the objects are normalized values, meaning that a value of 1 indicates the whole screen width or height (depending on the parameter) regardless of the screen resolution. This allows to easily keep the graphics object aspect ratio independent of the display device.
- Auto detect full screen resolution if the output resolution value is not set.
- If a change in the screen resolution occurs, all what the user has to do is to call the graphicsRestore() function with the new resolution standard in order to adjust the graphics rendering to the new screen resolution.
- Customizable alpha, position, size and rotation degrees of each graphics object.
- Mask frames: This feature consists of a filter that give the possibility to only render a specific section in a particular shape of a video or an image. The shape and section of the rendered video/image is determined by a PNG picture who act as a mask. The alphas of the PNG mask are assigned to the video/image, so only the non-transparent pixels of the mask are rendered with the video/image, the transparent pixels of the mask are removed from the video/image (rendered as transparent pixels).
- Chroma-Key: This feature consists of a filter that removes pixels from an image that are under the range of a specific RGB signature + RGB Threshold configured by the user. The pixels that fits in that range do not get rendered (become transparent pixels).
- Alpha Blending: This feature allows to display frame-buffer 0 and frame-buffer 1 outputs superimposed one over the other. The Graphics-Server render its output on frame-buffer 1, so frame-buffer 0 is available for other purposes. If the Alpha blending feature is activated and the rendered output do not have a solid background, you could see the Graphics-Server output displayed on top of the content of the frame-buffer 0.
Text Rendering:
- Text rendering supports different colors and fonts.
- Text rendering uses a "text chart" graphic object. This automatically adjusts the size of the text string to fill the maximum possible area of the text chart, thus automatically adjusting the text size to fit in the text chart size.
Animation:
- Available animations for any graphic object:
- Shift (x,y plane).
- Rotate (x,y,z axes).
- Scale (up/down scale).
- Fade.
- Auto-Movement, Auto-Scaling, Auto-Rotation animations.
- Configurable rotation rate in rotate animations.
- Configurable movement interval (time from start to end position) and acceleration in shift animations.
- Configurable scaling time interval in scale animations.
- Configurable fading time interval, initial and final alpha values in fade animations.
- Animation group: This feature consists in a group of individual and diverse elements treated as one entity as far as control tasks are concerned. This allows you to control the animation execution flow of a group of elements. For example: This make possible to start or stop the available animations of all the elements under the group at the same time.
Limitations and Future Enhancements
Limitations:
- Only works on Vivante GPU units.
- Currently supports display video from a USB source captured with V4l2src and using the common 640x480 resolution.
- Currently supports to display one video object at a time.
- Chroma-Key feature works fine with images that has well-defined colors, but has some problems with video (Display some pixels of the undesired color configured in the chroma-Key). For example: If you configured the green color to be blocked by the chroma-key feature (RGB color signature + rgb threshold), the chroma-key feature will only remove the pixels that matches exactly with the configured color signature plus threshold, if there are different green color tones (darker or lighter) in the picture that get out of the RGB range defined by the RGB color signature + threshold, those pixels will be rendered in the final image.
Future enhancement:
- Add support to display video from different video sources (could be video streams from a file or even the network) and different standard resolutions.
- Add support to multiple video objects at a time.
- Improve Chroma-Key implementation to fix issues when it is applied to a video element and to be less sensitive to a specific RGB color signature.
- Add support to render video in a cube element.
- Add support to create different 3D figures (not only a cube, mesh loading).
- Add shaders to support several video transformations.
System Requirements
To run Graphics-Server you need to meet the following system requirements:
- Linux Kernel 3.0 or above version
- OpenGL version 2.0
- GStreamer 1.0 or above version
- DBus
- Vivante GPU Drivers
Installation Guide
Using the RR-SDK
The Graphics-Server has been tested using the IMX6-Nitrogen RidgeRun SDK. Following you will find the installation steps under the RR-SDK:
- 1) Uncompress the Graphics-Server app tarball into the proprietary directory of the SDK
- 2) Run "make config" command on the SDK root (DEVDIR)
- 3) Select the following elements:
-> Proprietary software
-> [*] Ridgerun Graphics Server
-> [*] Ridgerun Graphics Library
-> [*] Ridgerun Graphics Library demo
- 4) Run "make" command on the SDK root (DEVDIR), or in the Graphics-Server directory run: "make build install"
Using any other Linux distribution
The Graphics-Server has only been installed and tested using the RidgeRun SDK for IMX6-Nitrogen platform, but it should work fine in other platforms that meet the requirements by just make a quick and easy adjustment in the configuration files, and then just run "make" command. If you are interested in running the Graphics-Server in another platform with a custom Linux distribution please feel free to contact us and we will help you in the process.
Graphics-Server API Documentation
Graphics-Server has a very complete and descriptive documentation generated by Doxygen. To get access to the API documentation go to $Graphics-Server-Installation-DIR/Doc/html/ path and open the index.html file with any browser. For example:
google-chrome Doc/html/index.html
which will open the documentation in the browser window.
Graphics-Server Screen Plane Coordinates
In the below picture you can see a diagram that specifies the coordinates of the Graphics-Server Screen Plane workspace. I it is important to recall that Graphics-Server dimensions and position of the objects are normalized values, meaning that a value of 1 indicates the whole screen width or height (depending on the parameter) regardless of the screen resolution. This allows to easily keep the graphics object aspect ratio no matter the screen resolution.
Graphics-Server Demo
We made a basic demo that expose all the Graphics-Server features to demonstrate its capabilities.
Execution commands
Graphics-Server execution commands:
- 1) (Optional) Run in background a GStreamer pipeline that plays a video display it on frame-buffer 0. This allows showing the alpha blending capabilities of the Graphics-Server on IMX6 Nitrogen6X. The video is rendered on frame-buffer 0 and the Graphics-server output is rendered on frame-buffer 1. With alpha blending capabilities the Graphics-Server output (which does not have a solid background) is rendered on top of the playing video, allowing us to see both buffer-frames superimposed one over the other.
FB_MULTI_BUFFER=2 gst-launch-1.0 -v filesrc location=big_buck_bunny_480p_h264.mov \ ! qtdemux ! queue ! vpudec ! imxeglvivsink native-display=0 &
- 2) Run the graphics-server graphics-demo-app:
graphics-demo-app
- 3) Stop the demo-app and Kill the graphics-server:
Ctrl c killall graphics-server
- 4) Restart the graphics-server:
./etc/init.d/graphics-server start
Demo Execution Results
Refer to the first section of this wiki: "Demo Video"
CPU Performance Statistics
Here is some example output from running the top -d1 command while also running the Graphics-Server demo and the background video play GStreamer pipeline:
//Before running the graphics-demo-app: 4% gst-launch-1.0 -v filesrc location=big 2% graphics-server 6% gst-launch-1.0 -v filesrc location=big 3% graphics-server 6% gst-launch-1.0 -v filesrc location=big 3% graphics-server 5% gst-launch-1.0 -v filesrc location=big 3% graphics-server //After running the graphics-demo-app: 6% graphics-server 4% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 8% graphics-server 5% gst-launch-1.0 -v filesrc location=big 1% graphics-demo-app 5% gst-launch-1.0 -v filesrc location=big 2% graphics-server 0% graphics-demo-app 6% gst-launch-1.0 -v filesrc location=big 4% graphics-server 0% graphics-demo-app 8% graphics-server 6% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 10% graphics-server 6% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 10% graphics-server 6% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 8% graphics-server 5% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 9% graphics-server 5% gst-launch-1.0 -v filesrc location=big 1% graphics-demo-app 8% graphics-server 4% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 6% graphics-server 4% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 7% graphics-server 5% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 6% graphics-server 5% gst-launch-1.0 -v filesrc location=big 0% graphics-demo-app 8% graphics-server 5% gst-launch-1.0 -v filesrc location=big 1% graphics-demo-app
Demo Code
You can find the demo source code under: $GRAPHICS_SERVER_INSTALL_DIR/src/src/demo/main.c
//--------------------------------------------------------------------------------------
// File: main.cpp
//--------------------------------------------------------------------------------------
#include <iostream>
#include <stdio.h>
#include <assert.h>
#include <string.h>
#include <fcntl.h>
#include <malloc.h>
#include <math.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <termios.h>
#include <unistd.h>
#include <time.h>
#include <pthread.h>
#include <sys/time.h>
#include "graphics-client.h"
#define DEBUG 1
#include "debug.h"
#include <sys/ioctl.h>
#include <fcntl.h>
#include <unistd.h>
#include <linux/videodev2.h>
#include <linux/mxcfb.h>
#include <stdio.h>
#include <termios.h>
#include <unistd.h>
#include <fcntl.h>
bool fade_in;
/* Returns an integer in the range [0, n).
*
* Uses rand(), and so is affected-by/affects the same seed.
*/
int randint(int n) {
if ((n - 1) == RAND_MAX) {
return rand();
} else {
// Chop off all of the values that would cause skew...
long end = RAND_MAX / n; // truncate skew
assert (end > 0L);
end *= n;
// ... and ignore results from rand() than fall above that limit.
// (Worst case the loop condition should succeed 50% of the time, so we can
// expect to bail out of this loop pretty quickly.)
int r;
while ((r = rand()) >= end);
return r % n;
}
}
//Callback function that evaluates move/fade animations end flag in an asynchronous way (execute actions in a separated tread)
void on_graphicsMessageCallback(Graphics *object, gint type, const gchar *tag){
switch(type){
case ANIMATION_MOVING_END:
/* Upscale animation of logo plane picture */
graphicsStartAutoScaling("logo2");
graphicsSetVisible("logo2", true);
sleep(3);
/* Fade-in animation of logo plane picture */
graphicsSetFadingInterval("logo2", 230, 0, 3000);
graphicsEnableFading("logo2", true);
graphicsEnableEmitMessage("logo", false);
graphicsEnableEmitMessage("logo2", true);
graphicsStartFading("logo2");
fade_in = true;
break;
case ANIMATION_FADING_END:
/* Fade-in/fade-out continuous animation of logo plane picture */
if (fade_in==true){
graphicsSetFadingInterval("logo2", 0, 230, 3000);
graphicsStartFading("logo2");
fade_in = false;
}
else{
graphicsSetFadingInterval("logo2", 230, 0, 3000);
graphicsStartFading("logo2");
fade_in = true;
}
break;
default: break;
}
debug_info("Getting message (%d) from (%s)\n", type, tag);
}
int main (int argc, char **argv)
{
//Enable alpha blending of frame-buffer 1. Displayed output will be a blend of fb0 and fb1.
int alpha_blend = alpha_pixel_enable(1);
/*INIT*/
graphicsInit();
/*Rotary and movable RidgeRun Logo picture cube element*/
graphicsNewPicture("logo", "/usr/share/ridgerun/media/RR-Logo.png", GRAPHICS_SURFACE_CUBE);
graphicsSetRotationRate("logo", 0, 90, 0);
graphicsSetSize("logo", 0.15, 0.15, OBJECT_TOP_LEFT);
graphicsSetMoveAcceleration("logo", 0.2);
graphicsSetMoveInterval("logo", 0.1, 0.9, 0.125, 0.15, 6000);
graphicsEnableAutoRotation("logo", true);
graphicsEnableAutoMovement("logo", true);
graphicsEnableEmitMessage("logo", true);
graphicsSetVisible("logo", true);
graphicsStartAutoRotation("logo");
graphicsStartAutoMovement("logo");
/*Rotary, masked and movable video element*/
graphicsNewVideo("video", "auto", 0.0, 0.0);
graphicsSetVisible("video", false);
graphicsSetSize("video",0.35,0.35, OBJECT_TOP_LEFT);
graphicsSetMask("video", "/usr/share/ridgerun/media/gear.png");
graphicsSetMoveAcceleration("video", 0);
graphicsSetMoveInterval("video", 0.0, 0.0, 0.5, 0.5, 6000);
graphicsSetRotationRate("video", 45, 45, 0);
graphicsEnableMask("video", true);
graphicsEnableAutoMovement("video", true);
graphicsEnableAutoRotation("video", true);
graphicsSetVisible("video", true);
graphicsStartAutoMovement("video");
graphicsStartAutoRotation("video");
/* Fading and scalable logo plane picture */
graphicsNewPicture("logo2", "/usr/share/ridgerun/media/RR-Logo.png", GRAPHICS_SURFACE_PLANE);
graphicsSetPosition("logo2", 0.2, 0.7);
graphicsSetScalingCenter("logo2", OBJECT_CENTER);
graphicsSetScalingInterval("logo2", 0.05, 0.05, 0.25, 0.15, 3000);
graphicsEnableAutoScaling("logo2", true);
graphicsSetAlpha("logo2", 230);
/*Text chart element*/
graphicsNewText("text", "/usr/share/fonts/truetype/tlwg/TlwgTypewriter-BoldOblique.ttf");
graphicsSetSize("text", 0.2, 0.1, OBJECT_TOP_LEFT);
graphicsSetPosition("text", 0.5, 0.1);
graphicsSetVisible("text", true);
graphicsSetText("text", "GRAPHICS SERVER");
usleep(4*1000*1000);
/*Text chart element -> Changing text content*/
char text[16];
for(int i = 0; i < 10; i++){
sprintf(text, "CHANGING TEXT (%d)", i);
graphicsSetText("text", text);
sleep(1);
}
/* Changing Text Font */
char fonts[6][64];
sprintf(fonts[0],"/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-BoldOblique.ttf");
sprintf(fonts[1], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono-Bold.ttf");
sprintf(fonts[2], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansCondensed.ttf");
sprintf(fonts[3], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Italic.ttf");
sprintf(fonts[4], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerifCondensed-BoldItalic.ttf");
sprintf(fonts[5], "/usr/share/fonts/truetype/tlwg/TlwgTypewriter-BoldOblique.ttf");
for (int i = 0; i < 6; i++){
graphicsSetFont("text", fonts[i]);
graphicsSetText("text", "CHANGING FONT");
sleep(2);
}
/* Changing Text Color */
graphicsSetText("text", "CHANGING COLOR");
for (int i = 0; i < 10; i++){
graphicsSetFontColor("text", randint(255), randint(255), randint(255));
sleep(2);
}
graphicsSetText("text", "DONE!!");
// Continuous demo loop
while(1){
usleep(100000);
debug_info("Continuous demo loop.\n");
}
debug_info("Exit\n");
// Destroy objects
graphicsDestroy();
return 0;
}
Demo Code Overview
Rotary and movable RidgeRun Logo picture cube element
This are the necessary functions to create the rotary and movable cube picture element of the demo:
// Creates a new picture element from a given path. GRAPHICS Surface Cube picture element with RidgeRun Logo
graphicsNewPicture("logo", "/usr/share/ridgerun/media/RR-Logo.png", GRAPHICS_SURFACE_CUBE);
// Set Rotation Rate = 90 degrees/second in Y axis
graphicsSetRotationRate("logo", 0, 90, 0);
// Set element size -> 0.15x0.15 (Width x Height Normalized) Scaled from Top Left corner
graphicsSetSize("logo", 0.15, 0.15, OBJECT_TOP_LEFT);
// Set moving animation acceleration
graphicsSetMoveAcceleration("logo", 0.2);
// Set time duration, initial and final position of the moving animation
graphicsSetMoveInterval("logo", 0.1, 0.9, 0.125, 0.15, 6000);
// Enable element Auto-rotation animation
graphicsEnableAutoRotation("logo", true);
// Enable element Auto-movement animation
graphicsEnableAutoMovement("logo", true);
// Enable element to emit end-animation flag.
graphicsEnableEmitMessage("logo", true);
// Set visible the element on screen. Enable element rendering
graphicsSetVisible("logo", true);
// Start picture cube element rotation animation
graphicsStartAutoRotation("logo");
// Start picture cube element movement animation
graphicsStartAutoMovement("logo");
Rotary, masked and movable video element
This are the necessary functions to create the rotary, masked and movable video element of the demo:
// Creates a new video object
graphicsNewVideo("video", "auto", 0.0, 0.0);
// Set invisible the element on screen. Disable element rendering
graphicsSetVisible("video", false);
// Set element size -> 0.3x0.3 (Width x Height Normalized) Scaled from Top Left corner
graphicsSetSize("video",0.35,0.35, OBJECT_TOP_LEFT);
// Set Mask frame (The video will be only rendered in the non-transparent pixels of the png mask picture).
graphicsSetMask("video", "/usr/share/ridgerun/media/gear.png");
// Set moving animation acceleration
graphicsSetMoveAcceleration("video", 0);
// Set time duration, initial and final position of the moving animation
graphicsSetMoveInterval("video", 0.0, 0.0, 0.5, 0.5, 6000);
// Set Rotation Rate = 45 degrees/second in X and Y axis
graphicsSetRotationRate("video", 45, 45, 0);
// Enable Mask frame on the element
graphicsEnableMask("video", true);
// Enable element Auto-movement animation
graphicsEnableAutoMovement("video", true);
// Enable element Auto-rotation animation
graphicsEnableAutoRotation("video", true);
// Set visible the element on screen. Enable element rendering
graphicsSetVisible("video", true);
// Start element Auto-movement animation
graphicsStartAutoMovement("video");
// Start element Auto-rotation animation
graphicsStartAutoRotation("video");
Fading and scalable logo plane picture element
This are the necessary functions to set the fading and scalable animation to the logo picture element of the demo:
// Creates a new picture element from a given path. GRAPHICS Surface Cube picture element with RidgeRun Logo
graphicsNewPicture("logo2", "/usr/share/ridgerun/media/RR-Logo.png", GRAPHICS_SURFACE_PLANE);
// Set element position -> Element normalized top-left corner=(0.2,0.7) Note:Top left corner of the screen (0,0) is the plane start point
graphicsSetPosition("logo2", 0.2, 0.7);
// Set scaling animation center point
graphicsSetScalingCenter("logo2", OBJECT_CENTER);
// Set time duration, initial and final size of the scaling animation
graphicsSetScalingInterval("logo2", 0.05, 0.05, 0.25, 0.15, 3000);
// Enable the auto-scaling animation
graphicsEnableAutoScaling("logo2", true);
// Set element initial alpha
graphicsSetAlpha("logo2", 230);
==
==
//Callback function that evaluates move/fade animations end flag in an asynchronous way (execute actions in a separated tread)
void on_graphicsMessageCallback(Graphics *object, gint type, const gchar *tag){
switch(type){
case ANIMATION_MOVING_END:
/* Upscale animation of logo plane picture */
// Start the auto-scaling animation
graphicsStartAutoScaling("logo2");
// Set visible the element on screen. Enable element rendering
graphicsSetVisible("logo2", true);
sleep(3);
/* Fade-in animation of logo plane picture */
// Sets the fading interval. Initial_alpha=230, Final_alpha=0, transitiom_time(ms)=3000
graphicsSetFadingInterval("logo2", 230, 0, 3000);
// Enable the fading animation for the element
graphicsEnableFading("logo2", true);
// Disable element to emit end-animation flag.
graphicsEnableEmitMessage("logo", false);
// Enable "End animation" message and flag. The user can use this flag to determine when moving or fading animation ends
graphicsEnableEmitMessage("logo2", true);
// Start fading animation
graphicsStartFading("logo2");
fade_in = true;
break;
case ANIMATION_FADING_END:
/* Fade-in/fade-out continuous animation of logo plane picture */
if (fade_in==true){
graphicsSetFadingInterval("logo2", 0, 230, 3000);
graphicsStartFading("logo2");
fade_in = false;
}
else{
graphicsSetFadingInterval("logo2", 230, 0, 3000);
graphicsStartFading("logo2");
fade_in = true;
}
break;
default: break;
}
debug_info("Getting message (%d) from (%s)\n", type, tag);
}
Text chart element
This are the necessary functions to create the text chart element of the demo:
// Creates a new text object with the given font path
graphicsNewText("text", "/usr/share/fonts/truetype/tlwg/TlwgTypewriter-BoldOblique.ttf");
// Set element size -> 0.2x0.1 (Width x Height Normalized) Scaled from Top Left corner.
graphicsSetSize("text", 0.2, 0.1, OBJECT_TOP_LEFT);
// Set element position -> Element normalized top-left corner=(0.5,0.1) Note:Top left corner of the screen (0,0) is the plane start point
graphicsSetPosition("text", 0.5, 0.1);
// Set visible the element on screen. Enable element rendering
graphicsSetVisible("text", true);
// Set text content of the text element
graphicsSetText("text", "RIDGERUN");
/*Text chart element -> Changing text content*/
char text[16];
for(int i = 0; i < 10; i++){
sprintf(text, "CHANGING TEXT (%d)", i);
graphicsSetText("text", text);
sleep(1);
}
/* Changing Text Font */
char fonts[6][64];
sprintf(fonts[0],"/usr/share/fonts/truetype/ttf-dejavu/DejaVuSans-BoldOblique.ttf");
sprintf(fonts[1], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansMono-Bold.ttf");
sprintf(fonts[2], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSansCondensed.ttf");
sprintf(fonts[3], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerif-Italic.ttf");
sprintf(fonts[4], "/usr/share/fonts/truetype/ttf-dejavu/DejaVuSerifCondensed-BoldItalic.ttf");
sprintf(fonts[5], "/usr/share/fonts/truetype/tlwg/TlwgTypewriter-BoldOblique.ttf");
// Changing font loop
for (int i = 0; i < 6; i++){
graphicsSetFont("text", fonts[i]);
graphicsSetText("text", "CHANGING FONT");
sleep(2);
}
/* Changing Text Color */
graphicsSetText("text", "CHANGING COLOR");
// Changing color loop. The color is assigned in an aleatory way
for (int i = 0; i < 10; i++){
graphicsSetFontColor("text", randint(255), randint(255), randint(255));
sleep(2);
}
graphicsSetText("text", "DONE!!");
Set Debug Level
To change the debug level of the Graphics-Server, you have to change the DEBUG_LEVEL_CONFIG variable value on the principal Makefile accordingly to the following description and recompile the app. By default this variable is set to 0 (No debug).
#~ DEBUG_LEVEL: #~ * 0: print only errors #~ * 1: print errors and warnings #~ * 2: print errors, warnings and info messages #~ * 3: print errors, warnings, info and debug messages #~ * 4: print errors, warnings, info, debug and log messages #~ * 5: print errors, warnings, info, debug, log and trace messages DEBUG_LEVEL_CONFIG=0