create new tag
view all tags

Person detection with!MicroPython

Even though the final goal of this demo is to get images from a camera and to decide on the spot if there is a person in sight, the first step is to test the model with well known test images. We use a pre-trained and quantized model. Re-creating the model and training it is non-trivial because the model provided has been built with an outdated version of TensorFlow. It would be an interesting task to build a new Keras model and train it.

You can find an example of the person detection demo in the sources of TFLM and a different version for the esp-idf and the Arduino SDK. For MicroPython I tried to port both versions and compared the results with their C++ originals.

The TFLM version

Here only 2 test images are used: a person and a giraffe:

person.png no_person.png
person no person
The result from the invocation of the model returns a score value of -128 .. 127, which can be converted to a probability value through de-quantization. The microlite module provides the quantizeInt8ToFloat which accomplishes this task.

Here is the result from the MicroPython program:


As you can see, the results are correct and the confidence levels are pretty high.

The esp-idf and Arduino version of the person detection model

These versions are pretty complex and they incorporate

  • inference on test images
  • camera readout
  • display on a TFT screen
all in a single multi-tasking program. As the hardware used is different from my boards I decided to disentangle these components and to provide a separate program for the use of test images and the use of the camera. The treatment of the test images can be done on both CPU boards, while the camera version is of course on possible with the FreeNove board incorporating the OV2640 camera.

Here are the test images used in the example:

image0.png image1.png image2.png image3.png image4.png
image 0 image 1 image 2 image 3 image 4
image5.png image6.png image7.png image8.png image9.png
image 5 image 6 image 7 image 8 image 9
The images are provided as binary files with 96x96 uint8_t gray scale pixel values. Th visualize them I wrote a short Python script (show-test-image.py) and another one (pixel2png.py) to convert the binary files into png images.

Before passing the pixels into the input tensor for the model invocation the pixel values must be converted from uint8_t to int8_t expected be the model.


The program supplies its results in the above summary table. You may judge yourself, how well the person detection works. It seems that after all the difference of a person and a monkey is not that great!

-- Uli Raich - 2023-11-08


Topic attachments
I Attachment History Action Size Date Who Comment
PNGpng image0.png r1 manage 4.3 K 2023-11-08 - 20:52 UliRaich  
PNGpng image1.png r1 manage 5.1 K 2023-11-08 - 20:52 UliRaich  
PNGpng image2.png r1 manage 4.4 K 2023-11-08 - 20:52 UliRaich  
PNGpng image3.png r1 manage 5.4 K 2023-11-08 - 20:52 UliRaich  
PNGpng image4.png r1 manage 4.9 K 2023-11-08 - 20:52 UliRaich  
PNGpng image5.png r1 manage 7.3 K 2023-11-08 - 20:53 UliRaich  
PNGpng image6.png r1 manage 3.5 K 2023-11-08 - 20:53 UliRaich  
PNGpng image7.png r1 manage 6.6 K 2023-11-08 - 20:53 UliRaich  
PNGpng image8.png r1 manage 4.9 K 2023-11-08 - 20:53 UliRaich  
PNGpng image9.png r1 manage 3.6 K 2023-11-08 - 20:53 UliRaich  
PNGpng mp_person_detect.png r1 manage 38.6 K 2023-11-08 - 20:46 UliRaich  
PNGpng no_person.png r1 manage 6.8 K 2023-11-08 - 20:26 UliRaich  
PNGpng person.png r2 r1 manage 5.7 K 2023-11-08 - 20:26 UliRaich  
PNGpng person_detect_10imgs.png r1 manage 45.1 K 2023-11-08 - 21:04 UliRaich  
Topic revision: r1 - 2023-11-08 - UliRaich
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2024 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback