Monday, November 23, 2015

Efficiently developing multimodal user interfaces

Matthias Hampel

HMIs for medical devices

New car owners love the exciting features built into their vehicles: comfort, driver assistance, and infotainment functions that make the car a joy to spend time in. However, with the number of new functions increasing, it can be difficult even for experienced users to find and operate those features.

For automotive manufacturers, this means that designing good user interfaces is becoming more challenging. Drivers need to be able to access a wide range of information and functions easily and quickly, and with as little driving distraction as possible. Car manufacturers are tasked with developing HMIs on limited budgets and tight schedules—while still making them simple, easy to use, and manageable despite covering that wide range of functions.

To address the problem, manufacturers are focusing on two pillars for HMIs: digital instrument clusters and multimodality.

Digital instrument clusters

The right HMI enables rendered displays on digital instrument clusters, allowing various pieces of information to appear on the same screen. Users can adjust the size of the display, within the specifications of the system, to meet their needs. Digital instrument clusters improves both the range of functions available and safety, since it allows important warnings to appear at any time and any irrelevant information to be faded out during the journey.
Multimodality involves linking different control concepts such as buttons and switches, touch displays, and gesture and voice command systems.


Multimodality allows the driver to select different options for communicating with the software and to switch between them. This means drivers and passengers can use the most suitable and convenient operating mode for the given driving situation.

For example, while driving, the user can use speech commands to specify a destination, rather than a touch interface. However, since it’s easier to select a route from the display, the driver can switch to touch mode for that step. Of course, this requires that information be linked together so that the control option can be switched in a way that is intuitive and understandable for the user.

Cloud connection

Intelligently linking information offers even more options when the interface is connected to the Cloud. Many future comfort functions, from streaming and smartphone integration to navigation are greatly improved by augmenting them with information from the Cloud. Systems can better support drivers and offer them intelligent suggestions based on their selections, like offering music similar to what was just played or finding an alternative route with a good French restaurant on the way.

An intelligent voice command system also requires an online connection. For example, natural language systems process huge amounts of data and have to understand the intentions of the speaker and apply the relevant context. Here’s where hybrid solutions can be successful, ensuring offline recognition and working in situations like tunnels or a dead spot.

Development tools

Developing these kinds of multimodal systems presents many challenges for car manufacturers. Firstly they need a continuous development strategy for their HMIs. Tools like EB GUIDE support these strategies, letting manufacturers manage and maintain many HMI versions to accommodate different countries, brands, and models via a single platform.
That saves a lot of time and development expense.

Selecting the right development tool also has an impact on the efficiency of the development process and the quality of the user interfaces. It’s important, for example, to seamlessly integrate graphic and touch control elements as well as the latest speech technology, requiring comprehensive support for different modalities. Developers have to determine suitable commands systematically and right from the start of design, because changing commands later can be time-consuming and expensive. Ideally, a development system should have a speech control component that allows for parallel workflows and a direct link for modeling voice input as well as touch and graphic control elements.

Testing the software can also add significant time and cost. As user interfaces become more and more complex, tests must be carried out often, early, and regularly. In today’s rapid development environment, development and tests aren’t carried out one after the other, but in parallel. A development tool, such as EB GUIDE, that uses a model-based approach provides live simulation, saving on test time. Information from tests can be easily derived and the tests can be carried out automatically and on an ongoing basis, so that the current product status is known at all times.

Lastly, a large user base is important to developing a solid product. The wider the range of experience and feedback from users, the quicker and more effectively software can be optimized. This is why, for example, EB provides the basic version of EB GUIDE free of charge for non-commercial use. Students, customers, and others can test your HMI software if you make it available to them. This approach is beneficial for developers from a variety of industries, including ones like mechanical engineering or medical technology, where multimodal user interfaces are being used more and more.

In the long term, multimodal user interfaces will play an increasingly important part in developing and operating complex technical applications. And the automotive industry will continue to assume a pioneering role, as vehicle autonomy evolves, requiring more functions and more displays in the car. The automotive industry is already on the cutting edge with driver assistance applications that visualize the vehicle environment in order to display potential hazards and respond to them, or continuously analyze the physical condition of the passengers to detect fatigue and other issues.

Intelligently linking the widest variety of control elements plays a key role in allowing consumers to safely use the latest technology. EB is delighted to play a lead role in intelligent linking.

You can try EB GUIDE for free by downloading our Community Edition