Object Detection Using Deep Neural Networks – AI from HPC to the Edge

Project reference: 1917

Deep neural networks (DNNs) are evaluated and used in many areas today, to replace or complement traditional technologies. In this project, the student will develop a DNN to detect and localize selected objects in images. The work comprises the selection of the right network topology, the training, and validation strategy, as well as collecting and pre-processing training and validation data. The project does not stop with the training of the model in the data center but considers the complete lifecycle, including deployment and optimization. An application should demonstrate the correct object detection of the trained DNN on an edge device with limited computing resources (e.g. Intel’s Movidius Neural Compute Stick). The final outcome should be the object detection of a single type of object in a 720p video stream with ca. 24fps (real-time).

Training of the DNN will take place on NVIDIA V100 GPUs (e.g. NVIDIA DGX-2 system) with Tensorflow as deep learning framework. Depending on the selected edge device, deployment and optimization of the DNN will be done with either NVIDIA TensorRT or Intel’s OpenVino toolkits. As a stretch goal, for better portability, the final DNN should be using the ONNX model interchange format.

Training of a Deep Neural Network over multiple epochs to decrease loss and increase accuracy [illustration]

Project Mentor: Georg Zitzlsberger, M.Sc.

Project Co-mentor: Martin Golasowski

Site Co-ordinator: Karina Pešatová, Martina Kovářová

Participant: Pablo Lluch Romero

Learning Outcomes:
The student will be guided through the principles of DNNs, the state of the art networks and Google’s TensorFlow. With the project, the student also gets accustomed to the latest NVIDIA technologies around V100 that will be used for training, and different deployment and optimization toolkits.

Student Prerequisites (compulsory):

• Knowledge in Python is needed to create the DNN.
• Knowledge of Linux as the development environment.
• The project includes training for the student on mandatory DNN basics (training material from NVIDIA and Intel is used).

Student Prerequisites (desirable):
Tensorflow, Nvidia TensorRT, Intel OpenVino

Training Materials:
Introduction to:

• https://www.tensorflow.org/api_docs/
• https://developer.nvidia.com/tensorrt
• https://software.intel.com/en-us/openvino-toolkit
• https://software.intel.com/en-us/movidius-ncs

Workplan:

• 1^st  week: training week.
• 2^nd week: internal student training on DNN technologies and networks incl. first exercises on TensorFlow
• 3^rd week: Selection of network, training strategy and identification of training data
• 4^th week: Training and validation of DNN on NVIDIA V100 (finding right hyper-parameters)
• 5^th week: Deployment to the edge device
• 6^th week: Demoing final application of DNN on edge device (e.g. via video stream)
• 7^th week: Optimization for edge device (e.g. TensorRT/OpenVino) to improve the DNN performance for the edge device
• 8^th week: Discussing full processing pipeline (HPC to edge) with quality analysis

Final Product Description:
An environment (processing pipeline) will be created to train a DNN for detecting and localizing a specific object on the HPC side and to optimize and deploy it to an edge device for use.

Adapting the Project: Increasing the Difficulty:
Retrain the DNN to detect more different types of objects.

Adapting the Project: Decreasing the Difficulty:
The exact type and number of objects to detect will be selected by the student depending on his/her skills. Also, the deployment to the edge is variable in terms of which device to use. Using ONNX is a stretch-goal that might be sacrificed to favor faster solutions vs. using standards.

Resources:
Access to all hardware mentioned in the proposal (IT4I’s clusters, DGX-2 system and Movidius Neural Compute Stick)

Organisation:
IT4Innovations National Supercomputing Center at VSB – Technical University of Ostrava