Preparing a custom version of MicroPython with TensorFlow

Introduction

For work with ML algorithms I use the esp32-cam module because it is small, cheap and has a camera installed on it. To get it ready for ML models we must first create a custom version of MicroPython. Michael O'Cleirigh's github repository (https://github.com/mocleiri/tensorflow-micropython-examples) contains all the tools to do just this.

Here are the steps to build this custom microPython version:

• Download the repository:
  git clone https://github.com/mocleiri/tensorflow-micropython-examples tflite-micro-micropython
• The steps to build the firmware can be found in tflite-micro-micropython/.github/workflows/build_esp32.yml
  In my case the Espressif development environment espidf has already been downloaded and set up earlier. I use espidf version 4.3.1, the latest stable version at time of writing.
  
  Activate the virtual Python environment needed for espidf (if venvwrapper is installed: workon espidf)
  Make sure that the modules Pillow and Wave have been installed on this virtual environment. If not, install them with pip:
  • pip3 install Pillow
  • pip3 install Wave
• Setup the sub-modules needed for tflm:
  • cd tflite-micro-micropython
  • git submodule init
  • git submodule update --recursive
• Regenerate the microlite/tfm directory
  • cd tensorflow
  • ../micropython-modules/microlite/prepare-tflm-esp.sh
• Setup the sub-modules for the ESP32 port of MicroPython
  • cd ../micropython
  • git submodule update --init lib/axtls
  • git submodule update --init lib/berkeley-db-1.xx
• Get the esp32-camera driver from Espressif
  • cd ..
  • cd tflm-esp-kernels
  • git submodule update --init examples/person_detection/esp32-camera
• Build the MicroPython cross compiler
  • cd ../micropython/mpy_cross
  • make
• cd .. (the micropython folder of tflite-micro-micropython)
  • git apply ../boards/esp32/MICROLITE_SPIRAM_CAM/micropython.patch !!! This is not needed any longer and the patch does not work anymore !!!
    MicroPython allocates the full SPIRAM installed on the esp32-cam module for its heap. However, the esp32-camera driver needs some SPIRAM for its image buffer. The patch modifies main.c in micropython/ports/esp32 such that part of the SPIRAM is kept free for the camera driver.
• Due to recent changes in MicroPython some modifications to the micropython-modules are needed:
  • The last parameter in MP_REGISTER_MODULE (enable flag) must be removed in modcamera.c and tensorflow-microlite.c
  • mp_obj_type_t must be modified according to https://github.com/micropython/micropython/wiki/Build-Troubleshooting#define_const_obj_type in tensorflow-microlite.c
• cd boards/esp32/MICROLITE_SPIRAM_16M, when using the TTGO T7 Mini32 V1.5 ESP32-WROVER-B CPU board or
  cd boards/esp32/MICROLITE_SPIRAM_CAM, when using the esp32-cam module
  
  • idf.py clean build (this builds the MicroPython firmware)
• flash the custom MicroPython interpreter
  • idf.py flash

Since all the necessary drivers and libraries are now ready in MicroPython we can go ahead and try the TensorFlow Lite Micro examples.

-- Uli Raich - 2022-01-31

Comments