create new tag
view all tags



Machine learning (ML) algorithms usually work on huge datasets and require an enormous amount of resources. On the other hand many mobile phone use face recognition to improve photographs or speech recognition for natural language translation. This can be achieved by training machine learning models, in particular neural networks, on the big computer, which may use a GPU (Graphical Procession Unit) available on many of them, while the trained model is scaled down and used on the mobile phone.

Not long ago Google has released TensorFlow, the Artificial Intelligence library it uses for its own algorithms, as OpenSource and therefore made it available to everybody. The full Tensorflow library is used to create and train models and the TFLite library allow the down-scaling and use on smaller machines. Only very recently TFLite-micro (tflm) has been released which strips down the system even further to make it available of embedded systems.

TinyML with MicroPython on the ESP32

The TinyML library is described in a book published by O'Reilly. This book demonstrates how to use tflm models on three different micro-controllers:

  • Arduino Nano 33 BLE Sense
  • Sparkfun Edge
  • ST Microelectronics STM32F746G Discovery
and uses the C++ programming language.

It describes 4 different projects:

  • A hello world example which uses a regression algorithm to predict values of a sine function
  • Wake word detection, a model which can detect the spoken words "yes" and "no". The on-board microphone is used to sample the audio data.
  • People detection, a system that gets images from a camera and determines if a person is in sight of the camera
  • Magic Wand, a system that recognizes gestures using an accelerometer
Michael O'Cleirigh has started a project on github trying to port these examples to MicroPython (https://github.com/mocleiri/tensorflow-micropython-examples). Since there are several ESP32 modules that include a simple camera, I2S microphones are supported in MicroPython and several accelerometers with I2C interface are available, it should be possible to run these examples on the ESP32.

Setting up the PC and MicroPython for TensorFlow

The following pages show how to set up the PC to run TensorFlow, how to create models and how to down-scale them for use on the ESP32. Then a custom version of MicroPython must be created including the esp32-camera driver and its MicroPython interface as well as ulab, a stripped down version of numpy and scipy as well as the tflm library and its MicroPython access routines.

Finally the installation of the TinyML examples on the ESP32 is explained.

-- Uli Raich - 2022-01-31


Edit | Attach | Watch | Print version | History: r4 < r3 < r2 < r1 | Backlinks | Raw View | Raw edit | More topic actions
Topic revision: r4 - 2022-09-23 - UliRaich
This site is powered by the TWiki collaboration platform Powered by PerlCopyright © 2008-2022 by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback