TFTDisplay < IoT_Course_English

Tags: view all tags
---+ TFT Displays
---++ Introduction

A wide range of displays is available for use with the Arduino system. For the !WeMos D1 mini system I found 3 display boards. Each of them is based a on different controller chip and has different resolution. Here they are are:
   * 64x48 pixel 0.66" TFT screen with SSD1306 controller. The SSD1306 uses the !I2C bus
   * 128x128 pixel 1.4" display based on the ST7735 controller. The ST7735 uses the SPI bus
   * 320x240 pixel 2.4" display based on the ILI9341 display and the XPT2046 touch controller.
| 64x48 pixels with SSD1306 | 128x128 with ST7735 | 320x240 pixel touch screen with ILI9341 |
| <img alt="ssd1306.png" height="362" src="%ATTACHURL%/ssd1306.png" title="ssd1306.png" width="459" /> | <img alt="tftScreen.png" height="422" src="%ATTACHURL%/tftScreen.png" title="tftScreen.png" width="328" /> | <img alt="lolon2.4inch.png" height="471" src="%ATTACHURL%/lolon2.4inch.png" title="lolon2.4inch.png" width="347" /> |
Each of them has its advantages and disadvantages. The 0.66" display is really tiny and its resolution is very limited allowing to write just a few characters. In addition it is monochrome only. However, the reference display driver included in !MicroPython uses this controller chip. There are also higher resolution and color screens using this controller but I did not find one with the !WeMos D1 mini bus interface.

The 1.4" display is already quite a bit better even though 128x128 bit resolution is still not enough for sophisticated graphics. It is enough however to write a few lines of text and produce simple graphics, e.g. creating a graph of sensor readout results.

The 2.4" display is clearly the most powerful device, adding touch functionality. It is however so big that it is difficult to plug it onto the !WeMos D1 bus, even if the connector is available. The better solution is an adapter board and a connecting cable.This display board also features a touch screen and is one of the boards supported by the [[lvgl]] Graphical User Interface (GUI) library.

Since the 1.4" display is enough for most our applications, it can be nicely plugged onto the bus just like any other sensor or actuator shield and it is substantially cheaper than the 2.4" solution, this is the device we chose for the course.
---++ Preparing the hardware

Before using the display its solder jumpers on the back must be configured. There are 4 solder fields for configuration
   * TFT LED. his allows to control the display back light. I leave it unconfigured which results in the back light LED being permanently on
   * TFT_CS: D0. TFT_CS and TFT_DC can be selected onto D0, D3, D4 or D8 corresponding to GPIO 26, 17,16, 5. Since D3 and D4 cannot be used on the CPU with PSRAM the only selections possible are D0 and D8. I select D0 for CS and D8 for TFT_DC
   * TFT_DC: D8
   * TFT_RST is connected to the system reset
Here is a photo of the jumper settings:

<img alt="tftJumpers.png" height="322" src="%ATTACHURL%/tftJumpers.png" title="tftJumpers.png" width="1024" />

---++ The driver software

Searching the Internet I found many display drivers, none of which really satisfied me.
   * The ssd1306 reference driver included in !MicroPython. This driver is very basic and it relies very heavily on its [[http://docs.micropython.org/en/latest/library/framebuf.html][framebuffer superclass]].
   * A pure Python version of a ST7735 driver for the ESP32: [[https://github.com/boochow/MicroPython-ST7735][https://github.com/boochow/MicroPython-ST7735. ]]This a port of a ST7735 driver written by Guy Carver for stm32 processors (https://github.com/GuyCarver/MicroPython/blob/master/lib/ST7735.py). At first glance, this is exactly what I was looking for. However, Guy Carver himself was not too happy about its speed and, compared to other drivers, it has limited functionality. <br />There is a demo program putting images up onto the screen and another one exercising the driver's graphic functions like
      * drawing line
      * drawing rectangles
      * drawing filled rectangles
      * drawing text
   * A C-module of the above driver also written by Guy. Finding this code was not so easy. It is hidden in a fork of !MicroPython, made some years back, and it was again developed for stm32 processors. Since the method of integrating C-modules into !MicroPython has been made substantially easier in recent versions of !MicroPython quite a bit of work was needed to adapt this driver. When trying it, it was even slower than the Python code! I figured out that the reason was the way the SPI commands are sent to the controller. If you send commands and data in large chunks then you can speed up graphics considerably. This however is true for the Python and the C code.
   * A driver for displays based on the SSD1351 controller: [[https://github.com/rdagger/micropython-ssd1351][https://github.com/rdagger/micropython-ssd1351 ]]The display module used has 128x128 resolution just like the module we have. The driver comes with a big number of demo programs, some of which are rather sophisticated. It also provides many more functions than the ST7735 driver. A very nice tutorial with Youtube video is also available. Unfortunately loading fonts is extremely slow.<br />I decided to try porting this library to the St7735 chip in such a way that the demo programs would run with a minimum of modification. Speeding up font loading and a port of the demo programs from the above ST7735 driver should be part of the process.
---++ Developing our own driver

The first goal for our own driver was to be able to run all the demo programs on it. Since the calling sequences of the drivers are different (different method names, different way to pass parameters) some modifications to the demo code was of course necessary. In addition the method names and parameters defined in the framebuffer should be available.

Comparing the original ST7735 driver to the SSD1351 one we see the following differences
| *ST7735* | *SSD1351* |
| clips drawing elements when they exceed display boundaries | drops drawing elements if any pixel would exceed display boundaries |
| Provides 3 fonts as Python files | Provides many different fonts as C files<br />These fonts must be loaded before use, which takes a long time |
| uses .bmp files for images | uses binary rgb565 files for images |
| does not support sprites | allows use of sprites |
| Comes with 2 demo programs for graphics primitives<br />and image display | Comes with man demo programs, some of which are rather sophisticated<br />Even supplies simple Arcade games |
Finally I decided to come up with a mixture of both because I prefer drawing element clamping fro just dropping the element. In addition to the original naming convention of the methods additional names were introduced following the convention in the framebuffer driver.
---++ Fonts

Fonts are normally loaded into RAM before use. However, this takes time and uses a big amount of RAM memory, a very scarce resource on the standard ESP32. (We are using an ESP32 with additional PSRAM where the problem is much less critical).

Since font loading in the SSD1351 driver takes ages (the demo-font program using 9 different fonts took ~ 30s before the first character could be written. After that text writing was fast) I searched for another method and ended up asking the !MicroPython forum. Peter Hinch sent me a link to his [[https://github.com/peterhinch/micropython-font-to-py][micropython-font-to-py]] git repository explaining how the font data can be kept in flash and you can nevertheless get fast access to the them. The fonts are implemented as Python source files providing not only the font data but also access routines to the pixel data. These font files can be frozen into !MicroPython and are thus stored in flash. When a character is written, only the pixel data for this character is loaded into RAM.

The problem was now how to convert the font files in C supplied with the SSD1351 driver to Python font files compatible with micropython-font-to-py. I wrote a Python script:

%ATTACHURL%/convertFont.py.txt

doing exactly this. The output is actually executable (on the PC) and will print the pixel data of the letter "A" for this font.

<img alt="font.png" height="219" src="%ATTACHURL%/font.png" title="font.png" width="438" />

Finally I copied the fonts into the modules/fonts directory of the !MicroPython sources and froze them into the binary.

You can use the fonts by importing the !MicroPython module:

<literal>from&nbsp; ST7735 import Display, color565 import fonts/sysfont as sysfont sck = Pin(18)<br />miso= Pin(19)<br />mosi= Pin(23)<br />SPI_CS = 26<br />SPI_DC = 5<br />spi = SPI(2, baudrate=32000000, sck=sck, mosi=mosi, miso=miso) display=Display(spi,SPI_CS,SPI_DC)<br />display.draw_text(0, 0, 'Hello World!', sysfont, color565(255, 0, 0)) &nbsp; &nbsp;</literal>

In addition to font handling I received a link to [[https://github.com/peterhinch/micropython-nano-gui][micropython-nano-gui]] a lightweight and minimal !MicroPython GUI library for display drivers based on the framebuf class. This library contains a few widgets to display sensor data and includes a plot library.
---++ The frame buffer

There are two different approaches to the display driver:
   * Every graphics primitive results in a direct write to the display controller
   * All graphics primitives write to frame buffer memory. When all drawing is done the frame buffer memory as a whole is copied to the hardware
The SSD1351 driver uses the first approach while the reference driver for the SSD1306 in !MicroPython uses the second method. Peter Hinch's libraries depend on the underlying framebuf class.
---++ The goal

I now had:
   * a simple demo program I wrote for the SSD1306 module
   * 2 demo programs running on the original ST7735 driver
   * a bunch of demo programs from the SSD1351 driver
   * Peter Hinch's library for font handling, including his writer to display the fonts, and his _nanogui_ library including the _fplot_ plot library
I set myself the goal to write a driver which was capable of running all of these demo programs.

I ended up writing a class with methods implementing both design principles. There is a method
   * _draw_line(x0,y0,x11,y1,color) drawing_ the line directly on the hardware
   * and a framebuf compatible method <i>line(x0,y0,x11,y1,color) <br /> </i>which draws the line in the frame buffer memory
When using the frame buffer method you have to call the _show()_ method to copy the framebuf memory to the hardware to make the drawing visible.

Here are the methods:

These first methods write to the hardware directly
   * size(): returns width and height of the display as a tuple
   * clear(): clears the display
   * display_on(), display_off() switches the display on or off
   * cleanup(): clears the display, switches it of and deinits the SPI bus
   * draw_pixel(x,y,pixel): draw a single pixel
   * draw_hline(x,y,w,color): draw a horizontal line of length w, starting at the point x,y
   * draw_vline(x,y,w,color): you guess!
   * draw_line(x0,y0,x1,y1,color): draw a line from point x0,y0 to point x1,y1
   * draw_lines(coords,color): draw lines where the point coordinates are given as a list of coordinate pairs
   * draw_rectangle(x,y,w,h,color): draw a rectangle of width w and height h starting at point x,y
   * draw_polygon(sides,x0,y0,r,color,rotate=0)
   * draw_circle(x0,y0,r,color): draw a circle of radius r with center point x0,y0
   * draw_ellipse(x0,y0,a,b.color): draw an ellipse with center point x0,y0 and half axis a,b
   * draw_filledCircle(x0, y0, r, color): same as draw:circle but the circle is filled
   * draw_filledEllipse(x0, y0, a, b, color)
   * draw_filledPolygon(sides, x0, y0, r, color, rotate=0)
   * draw_letter(self, x, y, letter, font, color, background=0,landscape=False): draw the glyph for _letter_ at position x,y using the font <i>font. </i>If landscape is false the letter is drawn left to right, if landscape is true, the letter is rotated counter clock wise by 90° and is drawn from bottom to top
   * draw_text(self, x, y, text, font, color, background=0,landscape=False, spacing=1, nowrap = False): I think you can guess! If nowrap is true and the text is larger than the length of a text line it is wrapped to the next line.
   * draw_image(path,x0,y0,w=128, h=128): draw an image from a rgb565 raw image file to the display
   * load_sprite(path, w, h): load a sprite of width w and height h from a rgb565 raw image file
   * draw_sprite(buf, x, y, w, h): draw the sprite, which was previously loaded into buf
and these pass through the framebuffer:
   * fill(color): sets the entire frame buffer to color<br />fill(0): corresponds to clear() as all pixels in the frame buffer are set to 0 (BLACK)
   * pixel(x,y,pixel): draw a single pixel
   * hline(x,y,w,color): draw a horizontal line of length w, starting at the point x,y
   * vline(x,y,w,color): you guess!
   * line(x0,y0,x1,y1,color): draw a line from point x0,y0 to point x1,y1
   * lines(coords,color): draw lines where the point coordinates are given as a list of coordinate pairs
   * rect(x,y,w,h,color): draw a rectangle of width w and height h starting at point x,y
   * polygon(sides,x0,y0,r,color,rotate=0)
   * circle(x0,y0,r,color): draw a circle of radius r with center point x0,y0
   * ellipse(x0,y0,a,b.color): draw an ellipse with center point x0,y0 and half axis a,b
   * fill_circle(x0, y0, r, color): same as draw:circle but the circle is filled
   * fill_ellipse(x0, y0, a, b, color)
   * fill_polygon(sides, x0, y0, r, color, rotate=0)
   * letter(self, x, y, letter, font, color, background=0,landscape=False): draw the glyph for _letter_ at position x,y using the font <i>font. </i>If landscape is false the letter is drawn left to right, if landscape is true, the letter is rotated counter clock wise by 90° and is drawn from bottom to top
   * text(self, x, y, text, color, font=sysfont, background=0, landscape=False, spacing=1, nowrap = False): Please note the different parameter sequence as compared to draw_text
   * image(path,x0,y0,w=128, h=128): draw an image from a rgb565 raw image file to the display
   * sprite(buf, x, y, w, h): draw the sprite, which was previously loaded into buf
   * show(): copies the frame buffer data to the hardware
---++ The demos

From my tests with the SSD1306 driver I had
   * demo_ssd1306_fb.py: This program demonstrates
      * the drawing of lines
      * rectangles
      * filled rectangles
      * circles
      * the plot of a sine function with a text
   * demo_ssd1306.py has the same functionality but instead of passing through the frame buffer it writes to the hardware directly

Here are a few photos of some of the screen images
| <img alt="img_7968.png" height="373" src="%ATTACHURL%/img_7968.png" title="img_7968.png" width="300" /> | <img alt="img_7996.png" height="375" src="%ATTACHURL%/img_7996.png" title="img_7996.png" width="300" /> | <img alt="img_7994.png" height="367" src="%ATTACHURL%/img_7994.png" title="img_7994.png" width="300" /> |
The next demo program originated from the original ST7735 driver. It again comes in two versions:
   * graphicstest.py and
   * graphicstest_fb.py
This program has a text writing demo in addition, where a long text is displayed, wrapped to the next line once the end of the line is reached. The text is displayed in portrait mode in green font color and in white when showing off text drawing in landscape mode.

This program is quite similar in nature as demo-ssd1306.py but produces slightly different graphics and it integrates a text writing demo in addition, where a long text is displayed, wrapped to the next line once the end of the line is reached. The text is displayed in portrait mode in green font color and in white when showing off text drawing in landscape mode.

Again a have a few photos for you:
| <img alt="img_7972.png" height="372" src="%ATTACHURL%/img_7972.png" title="img_7972.png" width="300" /> | <img alt="img_7973.png" height="375" src="%ATTACHURL%/img_7973.png" title="img_7973.png" width="300" /> | <img alt="img_7964.png" height="377" src="%ATTACHURL%/img_7964.png" title="img_7964.png" width="300" /> | <img alt="img_7965.png" height="371" src="%ATTACHURL%/img_7965.png" title="img_7965.png" width="300" /> |
Then we have all the demo programs that came with the SSD1351 driver, some of which are quite sophisticated.
   * demo_bouncing boxes.py: shows a number of differently colored filled rectangles that bump of the the edges of the screens
   * demo_colored_squares.py: shows a number a filled rectangles in different colors
   * demo_color_palette.py:
| Bouncing Boxes | Color squares | Color palette |
| <img alt="img_8003.png" height="375" src="%ATTACHURL%/img_8003.png" title="img_8003.png" width="300" /> | <img alt="img_7952.png" height="369" src="%ATTACHURL%/img_7952.png" title="img_7952.png" width="300" /> | <img alt="img_8014.png" height="364" src="%ATTACHURL%/img_8014.png" title="img_8014.png" width="300" /> |
As explained above, text writing in different fonts is not exactly easy. While the original SSD1351 driver loaded its fonts from C files before using them, a procedure which could take several tens of seconds, the new version uses the same fonts but converted into Python code and stored in flash together with access methods to get the pixel information. Now, no font loading is necessary any more and text drawing becomes very fast. The text drawing demo<br /><br />
   * demo_fonts.py
only needed minor modification to reflect this change. However, a utility script to convert the original C font to a Python font is needed.

| <img alt="img_7958.png" height="370" src="%ATTACHURL%/img_7958.png" title="img_7958.png" width="300" /> | <img alt="img_7959.png" height="369" src="%ATTACHURL%/img_7959.png" title="img_7959.png" width="300" /> | <img alt="img_7960.png" height="373" src="%ATTACHURL%/img_7960.png" title="img_7960.png" width="300" /> | <img alt="img_7961.png" height="369" src="%ATTACHURL%/img_7961.png" title="img_7961.png" width="300" /> |
You can find the source code of the demos on the [[https://github.com/uraich/IoT-Course/tree/master/modules/display/demos][github repository]] for the course.
---++ Video games

In the list of demos for the SSD1351 driver you even find 2 simple video games:

Both games need additional hardware: a joystick in case of Super Mario and a linear potentiometer in the case of Arkanoid. demo-mario orriginally used a Bluetooth based joystick, which we don't have. I therefore replaced it with the analogue joystick from our sensor kit. Of course the software needed to be adapted correspondingly.
| demo-mario | arkanoid |
| <img alt="img_8026.png" src="%ATTACHURL%/img_8026.png" title="img_8026.png" /> | <img alt="img_8030.png" height="364" src="%ATTACHURL%/img_8030.png" title="img_8030.png" width="300" /> |

---++ GUI elements in nanogui

Peter Hinch's nanogui library contains a few widgets that can be used to graphically display sensor results. Their number is fairly limited (that is why Peter calls it <strong>nano</strong>gui) but extensions are possible if needed. nanogui contains:
   * a label
   * a meter
   * an led
   * a dial
I managed to get all of Peter's demo programs:
   * color15.py
   * color95.py
   * aclock.py
to work with the ST7735 driver. I did not try alevel.py, written for the pyboard and making use of its accelerometer, which is not available on our ESP32 board. The color95.py program showing the meters, was adapted to present temperature and humidity values from SHT30 measurements and I created a "Scale" widget to be used as a voltmeter in conjunction with the ESP32 ADC.Particularly interesting is also the fplot.py library allowing to plot data in cartesian or polar coordinate and which has the facility to take data in real time as they are produced by the sensor. For more details please refer to <a href="https://github.com/peterhinch/micropython-nano-gui" target="_blank">https://github.com/peterhinch/micropython-nano-gui<img alt="" border="0" height="12" src="http://localhost/pub/TWiki/TWikiDocGraphics/external-link.gif" width="13" /></a>

Here again some pictures
| Clock | SHT30 meters | Voltmeter |
| <img alt="img_7982.png" height="366" src="%ATTACHURL%/img_7982.png" title="img_7982.png" width="300" /> | <img alt="img_8016.png" height="362" src="%ATTACHURL%/img_8016.png" title="img_8016.png" width="300" /> | <img alt="img_8025.png" height="364" src="%ATTACHURL%/img_8025.png" title="img_8025.png" width="300" /> |
The clock is Peter's original clock program adapted to the ST7735. The SHT30 meters are meter widgets from nanogui connected to real measurements from the SHT30. Yes, also in Europe we can have Ghanaian temperatures! even though relative humidity is substantially lower.

The voltmeter meter application uses a Scale widget, not part of nanogui. Peter has supplied a superclass DObject (displayable object) from which the Scale widget is derived. The demo connects to the ESP32 ADC to display measured signal levels.
---++ Plotting

Peter's nanogui repository also contains a plotting package called fplot. This package allows to produce cartesian and polar plots to be generated. Particularly interesting is a feature that allows to add real time measurements to plots as they come in.

Again a few examples:

| Cartesian Plot | Polar Plot | Damped Oscillator |
| <img alt="img_7984.png" height="367" src="%ATTACHURL%/img_7984.png" title="img_7984.png" width="300" /> | <img alt="img_7990.png" src="%ATTACHURL%/img_7990.png" title="img_7990.png" /> | <img alt="img_8020.png" src="%ATTACHURL%/img_8020.png" title="img_8020.png" /> |
-- %USERSIG{UliRaich - 2020-07-04}%

---++ Comments

%COMMENT%
Attachments
Topic attachments
I	Attachment	History	Action	Size	Date	Who
txt	convertFont.py.txt	r1	manage	8.6 K	2020-07-21 - 11:26	UliRaich
png	font.png	r1	manage	2.7 K	2020-07-21 - 09:22	UliRaich
png	img_7952.png	r1	manage	205.7 K	2020-07-27 - 13:30	UliRaich
png	img_7958.png	r1	manage	211.5 K	2020-07-27 - 14:03	UliRaich
png	img_7959.png	r1	manage	220.4 K	2020-07-27 - 14:03	UliRaich
png	img_7960.png	r1	manage	238.6 K	2020-07-27 - 14:03	UliRaich
png	img_7961.png	r1	manage	244.8 K	2020-07-27 - 14:03	UliRaich
png	img_7964.png	r1	manage	238.0 K	2020-07-27 - 13:48	UliRaich
png	img_7965.png	r2 r1	manage	241.3 K	2020-07-27 - 13:42	UliRaich
png	img_7968.png	r1	manage	263.5 K	2020-07-27 - 12:36	UliRaich
png	img_7972.png	r1	manage	245.8 K	2020-07-27 - 13:03	UliRaich
png	img_7973.png	r2 r1	manage	236.6 K	2020-07-27 - 13:05	UliRaich
png	img_7982.png	r1	manage	196.5 K	2020-07-30 - 09:49	UliRaich
png	img_7984.png	r1	manage	221.3 K	2020-07-30 - 09:49	UliRaich
png	img_7990.png	r1	manage	204.3 K	2020-07-30 - 09:49	UliRaich
png	img_7994.png	r1	manage	187.9 K	2020-07-27 - 12:36	UliRaich
png	img_7996.png	r1	manage	237.4 K	2020-07-27 - 12:36	UliRaich
png	img_8000.png	r1	manage	231.5 K	2020-07-27 - 13:30	UliRaich
png	img_8003.png	r1	manage	165.4 K	2020-07-27 - 13:53	UliRaich
png	img_8014.png	r1	manage	227.4 K	2020-07-28 - 12:03	UliRaich
png	img_8016.png	r1	manage	191.6 K	2020-07-30 - 09:49	UliRaich
png	img_8020.png	r1	manage	204.1 K	2020-07-30 - 09:49	UliRaich
png	img_8025.png	r1	manage	181.8 K	2020-07-30 - 09:49	UliRaich
png	img_8026.png	r1	manage	402.3 K	2020-07-30 - 09:54	UliRaich
png	img_8030.png	r1	manage	187.2 K	2020-07-30 - 11:24	UliRaich
png	lolon2.4inch.png	r1	manage	218.8 K	2020-07-04 - 09:45	UliRaich
png	ssd1306.png	r1	manage	158.9 K	2020-07-04 - 09:44	UliRaich
png	tftJumpers.png	r1	manage	626.4 K	2020-07-17 - 16:16	UliRaich
png	tftScreen.png	r1	manage	144.4 K	2020-07-04 - 09:45	UliRaich
Topic revision: r13 - 2020-10-23 - UliRaich