KRIA SOM based Intelligent Applications
Introduction
In this article we will introduce the reader to the new Xilinx KRIA SOM. So, we will now detail how to take advantage of KRIA SOM in Edge AI application.
Xilinx made a big step towards the Software Engineers with its new Vitis Environment. The new KRIA SOM is another step towards also for ML engineers. KRIA SOM ahs an already built Platform and Image, such that the Engineer just needs to adjust its AI application.
Now, we are going to detail the characteristics of KRIA SOM and what are its features.
KRIA SOM Technical Features
This SOM is a 16nm UltraScale+ architecture, specifically a ZU5 device. This architecture enables the user to run complex AI based applications, which is the aim for which the KRIA SOM has been created.
The KRIA SOM has these features:
You receive KRIA SOM with a carrier board with the most common interfaces for Video Analytics, such as USB or HDMI. The hardware engineers can manage those interfaces directly from the PL by modifying the provided Vitis platform or the Vivado design.
KRIA SOM is compatible with the Xilinx Deep Learning Unit up to the most powerful B4096.
MakarenaLabs takes advantage of this possibility by integrating its Musebox Framework into the new KRIA.
Musebox supports 4 different tasks of Machine Learning:
- Face Analysis;
- People Analysis;
- Object and Text Analysis;
- Medical Segmentation;
- (and more incoming, stay tuned!);
GStreamer Pipeline and IVAS
The new KRIA development process integrates custom GStreamer plugins in order to enhance the management of the video pipeline processing. So, this allow the user to represent the problem as a queue of events and create a cascade structure of executors.
This new mode of development flow needs only the knowledge of standard C++ without having prior experience as hardware engineers or hardware knowledge at all.
Also, KRIA SOM integrates the IVAS library, which provides a set of methods and kernels to enable the Software Engineer to create its Image Processing Pipeline. So, with this library, the Software Engineer has an interface compatible both with GStreamer and the Xilinx Deep Learning Processing Unit.
How MakarenaLabs Re-Engineered MuseBox on KRIA SOM
As the last step in this article we want to show how it is possible to re-create an ML application. So, follow these 4 simple steps:
- First of all, create the Vitis AI model: The first step is to compile our AI model using the classic flow of Vitis AI, which generates both the .xmodel file. This file is the equivalent representation of our AI model, which has been trained with SPFP values into INT8 precision, maximizing the performances.
- Then, create the configuration JSON file (here an example): this file contains all the informations which GStreamer uses to setup the Queue of commands (i.e. the AI model requires pre-process, where can the executable can find it etc…).
- After that, create an “IVAS like library”: where our AI models and pre or post processing functions. Those libraries are all software based and you can find an example here. The IVAS library ensure that the user has a lot of libraries, so the application will be a software dependencies without the necessity of re-creating the Hardware design.
- Finally, create a queue of kernels in the main: GStreamer will be able to schedule the operations (the example here). In this main GStreamer will initialize the environment and will create the queue of kernels to be scheduled for execution.
Final Remarks and Conclusions
In this article we have introduced the new KRIA SOM and its new software based workflow with examples provided by Xilinx in its public repository.
The reader may find useful the following links to our MuseBox Framework for ML application and Xilinx resources of KRIA:
