Neural Nework - Object Detection

Introduction

In this example, we will be using Ameba Pro2 development board to detect 80 different kinds of objects such as person, bicycle, car, laptop etc using on-board camera sensor (JXF37).
The following examples show how Object Detection and On-Screen Display is used. The main difference between the two examples is where the Object Detection results are processed. “ObjectDetectionCallback” uses a callback function, while “ObjectDetectionLoop” deals with the results in the loop function.
1. ObjectDetectionLoop
2. ObjectDetectionCallback

Procedure

Open one of the Object Detection examples in “File” -> “Example” -> “AmebaNN” -> “ObjectDetectionLoop”.

Since the camera video output will be used, ensure that the “Video ISP Bin” option in “Tools” is set to enabled. Also ensure that “Neural Network Bin” is enabled to upload a flash binary that includes the required neural network binary file.

In the highlighted code snippet, fill in the “ssid” with your WiFi network SSID and “pass” with the network password.

Compile the code and upload it to Ameba. After pressing the Reset button, wait for the Ameba Pro 2 board to connect to the WiFi network. The board’s IP address and network port number for RTSP will be shown in the Serial Monitor.
The result of detected objects can be validated using VLC. You may download VLC media player from the link (here).
Upon the completion of the software installation, open VLC media player, and go to “Media” -> “Open Network Stream”.

Make sure your PC is connected to the same network as the Ameba Pro2 board for streaming. Since RTSP is used as the streaming protocol, key in “rtsp://{IPaddress}:{port}” as the Network URL in VLC media player, replacing {IPaddress} with the IP address of your Ameba Pro2 board, and {port} with the RTSP port shown in Serial Monitor (e.g., “rtsp://192.168.1.154:554”). The default RTSP port number is 554.

Next, click “Play” to start RTSP streaming to see the result. The video stream from the camera will be shown in VLC media player.

A bounding box with its object name and confidence score will be drawn on the RTSP video stream to highlight the recognised objects.

The pretrained model can recognise 80 different types of objects in total. The objects can be found in ObjectClassList.h. The index number for each object is fixed and should not be changed. To deactivate the detection of certain objects, set the filter value to 0. For example, set the filter value to 0 to exclude the detection of bicycles.