MicroPythonExamples < IoT_Course

Tags: view all tags

---+ Running the !TensorFlow examples in !MicroPython [[MicroPython][<br />]]
---++ The !MicroPython interface to !TensorFlow

The !MicroPython interface to !TensorFlow is implemented as a module written in C. It contains several Python classes:
   * interpreter: gives access to the tflite-micro runtime interpreter
   * tensor: allows to define and fille the input tensor and to interpret the output tensor
   * audio_frontend: needed for the wake words example.
---++ Running inference on the ESP32

After having defined and trained the model with !TensorFlow, it must be converted into a !TensorFlow Lite model, which is an optimized !FlatBuffer format identified by the .tflite extension. This file must be transfered to the !MicroPython file system. In my examples I create a "models" folder on the ESP32 into which I save the models. You can easily find the size of the model with the ls -l command on the PC. The model is read into a bytearray:

<img alt="readModel.png" height="153" src="%ATTACHURL%/readModel.png" title="readModel.png" width="413" />

Once the model is loaded we can create the runtime interpreter:

<img alt="interpreter.png" height="39" src="%ATTACHURL%/interpreter.png" title="interpreter.png" width="859" />

<img alt="invoke.png" height="44" src="%ATTACHURL%/invoke.png" title="invoke.png" width="175" />

input_callback is called when you invoke the interpreter. It gives access to the input tensor and allows to fill it using the setValue method of the tensor class. In the example below, the input tensor is filled with pixel values on an image that has previously been read from the ESP32 file system.

<img alt="input_callback.png" height="193" src="%ATTACHURL%/input_callback.png" title="input_callback.png" width="709" />

Finally the output_callback is called in which you access the output tensor and interpret it.
---++ The classes and their resources
---+++++ tensor

The tensor class has the methods:
   * getValue(index)
   * setValue(index,value)
   * getType()
   * quantizeFloatToInt8()
   * quantiteInt8ToFloat()
---+++++ The interpreter

The interpreter has methods to get the input and output tensors and for invocation:
   * getInputTensor(tensor_number)
   * getOutputTensor(tensor_number)
   * invoke()
Here is a description of the !MicroPython version of the examples:
   * [[hello world !MicroPython]]
   * [[person detection !MicroPython]]
   * [[wake word detection !MicroPython]]
   * [[magic wand MicroPython]]
-- %USERSIG{UliRaich - 2023-09-07}%
---++ Comments

<br />%COMMENT%

Attachments

Topic attachments
I	Attachment	History	Action	Size	Date	Who
png	input_callback.png	r1	manage	29.9 K	2023-10-12 - 08:37	UliRaich
png	interpreter.png	r1	manage	7.2 K	2023-10-12 - 08:27	UliRaich
png	invoke.png	r1	manage	3.0 K	2023-10-12 - 08:39	UliRaich
png	output_callback.png	r1	manage	22.4 K	2023-10-12 - 08:37	UliRaich
png	readModel.png	r1	manage	22.6 K	2023-10-12 - 08:23	UliRaich

Topic revision: r3 - 2023-11-08 - UliRaich