Screen Interface Prototype

While I have been rather negligent with my blog updates I have been kicking goals with progress on the Grid-Eye thermal sensor, BananaPi screens and the user interface. Of these it is user interface that I am going to blather about here.

My primary goal of this project is to improve the microwave user interface, shifting away from four character seven segment displays to a more interactive fully featured display.

Everything else, most of the work, is to support and implement this improvement. As I am now up to playing with the screens, selecting screen sizes and touch interfaces it seemed worthwhile to spend a bit of time considering what the GUI will actually look like.

I did a few pages of pencil sketches, useful to work through a wide variety of ideas.

Then I did a prototype implementation in HTML to iterate the better ideas. This prototype is below, it is a fully interactive webpage – some bits pretend to work, other bits are stubbed out.

I would really appreciate it if people would play and give feedback. It is designed for a fixed screen size of 480×800 pixels, the primary user interface will be finger on a touchscreen, there will also be general web browser access for PCs and portable devices.

View in new tab/window

The source code is available on github, it is prototype code so there are more than few shortcuts and lots of poor practices.

My current design guidelines are:

  • Intuitive to use
    • No instruction manual or help pages
    • No advanced touch interfaces (sliding, multitouch etc.)
    • Large, obvious, touch targets
    • Accessible colour scheme and aria support
  • Forgiving of mistakes
  • Not too hideously ugly

Choosing a touchscreen

When selecting a touchscreen there is actually an interesting choice that I have to make. There are two common touch technologies, resistive and capacitive, many people probably know it as old and new touchscreens. (Interesting aside, capacitive screens actually came first in 1965 but weren’t widely adopted, then resistive screens became common before capacitive screens resurged with phones and became the “new” style.)

Resistive screens

A resistive screen works by letting you push one side closer to the other and detects this change.

Pro:

  • Electrically simple, 4 wire support built in to BananaPi A20 chip.
  • Cheaper to manufacture.
  • More accurate detection than capacitive screens.
  • Will work if wet.
  • Will work if wearing gloves, using elbow or poked with a stick.
  • Plastic is scratch resistant.
  • Can optionally detect pressure.

Con:

  • Easier to damage screen.
  • Damaged screen is useless, minor damage typically causes complete failure.
  • Only supports two simultaneous touch points.

Capacitive screens

A capacitive screen detects power loss to a nearby conductor such as a finger. The modern standard is a projected capacitive technology (PCT) panel which means a direct touch isn’t required, this allows a protective glass layer to be inserted on the top of the screen assembly.

Pro:

  • More familiar to people.
  • Easier to clean.
  • Sensitive PCT devices can detect fingers through insulating gloves.
  • Supports complex multitouch patterns.

Con:

  • Fails if coated in a conductive material, such as sweat or food liquids.
  • Sensitive to mechanical design, the metal chassis is a conductor which can significantly impact on screen operation and sensitivity.
  • Conductivity varies on the person, the person’s hydration, ambient humidity etc.

Further pontificating

So for something portable, like a phone, going capacitive is obviously the better choice.

For a kitchen appliance it is far less clear. Wearing dishwashing gloves or smearing tomato sauce across the controls is going to be more common.

The big factor I keep coming back to is the familiarity. People expect to have a solid capacitive touch screen and attach a higher quality value to it than a resistive screen. And that the limitations of the capacitive screen are well understood so people seem happy to work around them.

Basically the engineer in me leans towards resistive, the marketing orphan I recently adopted says capacitive.

So for now I’m getting both. I am already familiar with capacitive touch panels and have one coming as part of one of my screens. So I just need to acquire a resistive panel to play with and understand.

Shopping

Buying a resistive panel is far harder than it should be… mostly due to limitations with the wondrous site alibaba.

<rant>
Alibaba is fantastic for interacting with component suppliers but the search engine is absolute rubbish, pre-Google Yahoo rubbish. (As Yahoo used to own a substantial chunk of Alibaba maybe it is literally the pre-Google Yahoo rubbish.) Essentially you enter some words, Yahoo searches for those words in the title of each product, the product with the most words in common is the top of the list. Which means that the retailers create new duplicate product entries with different titles trying to hold the bingo card that matches your search words. One touchscreen manufacturer had a single 4 wire resistive touch product, available in about 15 different sizes. They had roughly 1800 bingo cards/product entries trying to attract eyeballs.

(A nice trick with Alibaba direct is that they don’t keep these all updated, so you can often get up to 20% off by finding a differently priced entry of the same product from the same manufacturer.)

Once I find a manufacturer I try to figure out if I can trust them. A warning trigger for me is anything that they are obviously lying about in their marketing, if there are obvious lies I can see then there are probably subtle lies that I can’t and will bite me later. With resistive touchscreens however EVERYBODYalmost everybody lies, possibly this is related to the above Alibaba is crap rant. And it doesn’t look like lies that help them, a 4 wire touchscreen has four analog passive resistive wires, they state this, then in the interface section they state “RS232 or USB interface”… which I don’t understand. It clearly is a 4 wire FPC connector, there is no RS232 or USB plug in the picture, no mention in the description. I suspect there is a series of checkboxes box for interface and they feel the need to tick something… Another odd one is they list a resolution, numbers vary but all are wrong. It is an analog device, any resolution limitations come from the quantization performed by the controller, which they don’t sell. Why provide a fake number and risk it being smaller than somebody else’s fake number when the real answer is infinite and therefor better? *sigh*
</rant>

Buying

I have reached out to three suppliers for further information:

I also found some interesting capacitive touch manufacturers in case I go that route:

State of displays

Teeing up a display for an embedded system is a little more complex than buying a monitor.

At the core of it is one of my favorite sayings, “The wonderful thing about standards is that there are so many to choose from.”

Starting from the screen end of things, I want a screen that is roughly 150x90mm in size. Because I want it viewable from the side and ideally all angles, I have a strong preference for an IPS display. I found a screen that is really nice,Topfoison’s 6″ IPS display.

This display uses MIPI DSI for its video signal, specifically a 4 lane MIPI DSI interface (because standards restrict choice too much many standards allow incompatibility within themselves).

MIPI DSI is an interesting standard, it is a mobile industry standard (Mobile Industry Processor Interface – pronounced lowercase as mipi, like nbn) which essentially specifies a number of half duplex SERDES links for throwing video around. Many mobile phones seem to have adopted the standard as it removes the need for a video converter – reducing part count/cost, simplifying cabling and reducing internal EMI. The display and chip manufacturers have shifted with the market. However as it is fairly new mostly it is new/expensive hardware with support.
If you are meant to be doing something else here is some further reading, nice overview, detail with pretty scope pictures.

So, looking at hardware to plug in to this we have:

  • BananaPi M1, the board I have been planning on using.
    Is not listed as supporting MIPI DSI, does support relatively generic LVDS output.
    Graphics processor is the ARM MALI400 MP2, the newer ARM MALI cores include a display controller but I believe this wasn’t the case for the MALI400. I haven’t been able to find documentation on what display controller Allwinner used in the A20 chipset.
    Some websites do claim that BPI has a MIPI DSI output, I think they are wrong and are simply assuming that the RaspberryPi look-alike video header is a function-alike video header.
  • BananaPi M2
    Uses the Allwinner A31s (datasheet) chipset which does not support MIPI DSI output.
    The Allwinner A31 does, but it seems they removed it when pluralising chip.
    The SinoVoip website lists the output as “LVDS/RGB/CPU display interface (DSI) for raw LCD panels” on the product page. However the DSI gets dropped when getting into the detailed LCD connection information.
  • BananaPi M3
    Uses the Allwinner A83T chipset
    Specifically states support for MIPI DSI and MIPI CSI interfaces, with dedicated headers.
    However this isn’t available yet, when contacted SinoVoip suggested it should be out around the end of this year.
  • Raspberry Pi
    The Raspberry Pi uses a Broadcom chipset which supports and has a header for 2 lane MIPI DSI output.
    However due to driver issues this output isn’t generally useable.
    There are complex projects made with custom hardware to convert the HDMI output to MIPI DSI.
    The Raspberry foundation produced a screen which uses the MIPI DSI output with kernel support. Apparently they managed to do it in such a way that generic boards aren’t supported, just theirs.
  • Lemon Pi
    Uses an Actions S500 chipset
    Specification states support for MIPI DSI but doesn’t specify how many lanes and the S500 documentation doesn’t mention DSI.
    Not yet available, indications that things are happening so may ship this year.
    Includes (optional) built in bootable permanent storage.
  • Other boards
    There are other systems out there which aren’t suitable for various reasons.
    For example the H4 Hummingbird uses the Allwinner A31 with MIPI DSI output but at $70 isn’t competitive.
  • Making choices

    I have ordered the official SinoVoip 7″ LCD touchscreen. This is a non-IPS 7″ display connected to the LCD connector via a dedicated adaptor board. It also includes a touchscreen, no details so am assume it is a capacitive panel. The main reason for getting this screen is that as it is distributed by SinoVoip it should be well supported, the price is rather high but I haven’t inquired about production quantities.

    I am also negotiating an order of the Topfoison 6″ IPS screen with HDMI adaptor. The adaptor seems to add about $5-10 USD per unit and I expect a HDMI cable will be a few dollars, so the wiring ends up being a substantial percentage of the screen costs. Currently using MIPI DSI directly isn’t an option but I will definitely reevaluate down the track once the next generation of boards comes out.

Return top