To what extent has the coronavirus pandemic determined trends in the HMI sector?

Stephan Meyer-Loges: The corona pandemic has meant that hygiene aspects have come more to the fore. Operating units must be completely disinfectable. Customers are paying more attention to seamless installation, which means that as many dirty edges as possible should be avoided. Fortunately, we have been developing our HMIs for years in such a way that they can be seamlessly integrated. In addition, much more thought is being given to alternative operating concepts.

How do hygiene aspects influence the development of new operating units?

Meyer-Loges : It has a particularly strong influence on materials. It actually increases the amount of documentation required, because it has to be explained how materials are to be cleaned. For example, visible, painted plastics are generally not resistant to disinfectants. In addition, the seamless installation concept still leaves gaps: this issue becomes increasingly relevant when integrating an HMI due to the hygiene aspect. With a view to the needs of our customers in the food processing and medical sectors, we already integrated a robot-supported, automated jointing system into our production last year. We are now benefiting from this.

Stephan Meyer-Loges, Head of Product Management at Garz & Fricke in Hamburg.

© Garz&Fricke

What role does software play in new operating concepts?

Meyer-Loges: I firmly believe that software will play an increasingly important role in the area of operating units and HMIs. Smartphones have taught us that devices can be customized via software; this trend is currently also very much in evidence in the automotive sector. And especially in areas where products have a longer life expectancy, it is a huge competitive advantage for manufacturers if devices can be repeatedly adapted to the circumstances using software. This means that the appearance and the way the device is used can be adapted and modernized. This also means that any errors and weak points can be rectified during operation. Much more important, however, is the fact that these concepts offer the possibility of adapting the scope of delivery via software, i.e. adding, changing or removing functions or only allowing them for a limited time.

Software would also make it possible to control machines and systems using your own smartphone, so that you no longer need a local HMI...

Meyer-Loges: It is possible to a certain extent; we have shown how it works with the remote touch system: HMIs equipped with this can be operated using a smartphone without the user having to download a proprietary app. However, operation with a mobile device remains an additional benefit. Our HMIs are used in a wide variety of areas - from medical and building applications to industrial automation - and many of these locations are not suitable for being completely reduced to personal devices in terms of operation. Local control has advantages, for example if you have forgotten your cell phone, if the Internet connection cannot be established properly or in terms of important safety aspects - the operator must be present during operation.

Beyond these points, what differentiating features are created by software?

Meyer-Loges: I think the functional scope of software differentiates very few devices. In the end, software theoretically allows everyone to obtain a very similar range of functions from different manufacturers. The real differentiation lies in the simplicity of integrating this software solution, and that is where our strength lies.

Are you talking about interoperability?

Meyer-Loges: Interoperability, sure, but the real differentiation for manufacturers like Garz & Fricke is not the scope of the software solution, but the way in which the system can be easily integrated by the device manufacturer. If in doubt, they can also obtain the software from a third-party provider. With us, however, he gets everything from a single source: a complete interface with operating system, documentation, etc.

What business models are created by and with software?

Meyer-Loges: First and foremost, you have to see software as an independent product that is delivered with our HMI: two components from which a complete system is created. In the end, it offers us an incredible number of possibilities: Hardware has to be replaced if you want to adapt features, software can be used to retrofit and upgrade features - provided this was also planned in the hardware from the outset. And of course, this results in new business models: the software services are available, but if you want to use them, you have to pay for them.

New operating concepts - top or flop?

Will gesture and voice controls set the trends in the future?

Meyer-Loges: The variety of interaction options continues to increase: from the touchscreen, which I have to touch, to a remote touch system, where I can operate it with my personal smartphone, to voice and gesture control. This doesn't necessarily have to do with trends, but ultimately with the ability to choose the best way of interacting with the user for a specific end device: Voice control makes little sense in a noisy environment and if you don't have your hands free, you can do relatively little with gesture control. Ultimately, the application and its environment will determine the operating concept.

What influence do such trends and software have on computing capacity and hardware development?

Meyer-Loges: Computing power is increasing in all areas, but there is no fundamental 'higher, faster, further' to be observed. What I mean by this is that the market is not demanding general-purpose hardware that can do everything. The range of components is wide: from simple microcontrollers to microcontrollers with special features such as industrial Ethernet with real-time capability or with special camera engines, from simple processor systems to high-performance SoCs. However, this diversity also increases the demands on developers: components with newer technologies and higher bandwidths require incredible attention in PCB design in order to function flawlessly and stably over the entire temperature range. This requires much more preparatory work than was perhaps the case six years ago with the technology available in the industrial sector.

External interfaces also have an impact on the hardware design: compared to USB 2, USB 3 offers more functions than just connecting a memory stick or a mouse, for example power delivery. However, these features bring with them new EMC requirements that affect not only the hardware design of the HMI, but the entire device design. As a result, approval and certification issues are also becoming increasingly complex.

Stephan Meyer-Loges, Head of Product Management at Garz & Fricke in Hamburg.

© Garz&Fricke

Irrespective of corona - what are the biggest trends in this area?

Meyer-Loges : Things are continuing as they have in recent years: screen resolution is increasing, as is the performance of processors and therefore the demands on software development. Alternative interaction concepts such as gesture and voice control will also definitely occupy us in the coming years and with them the questions: Where and how will a product be used? How do people ultimately want to interact with the product - with natural speech or by means of keyword-based voice control? How can the concept be implemented? How deep is the integration - already in a preliminary component such as an HMI or only in the end device? The possibility of connecting cameras and microphones will also become increasingly important. But wherever the data evaluation will take place, in the device or in the cloud, it must first be clarified which goal is to be achieved.

Can you process data on your HMIs at the edge?

Meyer-Loges: Of course! New processors such as the i.MX 8M Plus from NXP feature a Neural Network Engine, for example. This means that part of the processor is only there to execute machine learning algorithms. The topic is still relatively new for industrial systems, but it offers many opportunities. In other words, the world of thought is slowly starting to revolve around it: What can I do? What are the advantages? And what effects might it have? For example, is it possible to place a coffee machine in a public space that uses artificial intelligence to evaluate faces and then offers the user a double espresso because they look very tired? - Does this work under data protection law? There is a lot to consider here and we are still at the very beginning of an exciting development.

How does Garz & Fricke use artificial intelligence, now and in the future?

Meyer-Loges: Semiconductors with a dedicated AI unit won't come onto the market until next year, and from then on we will definitely be using them. But even without hardware acceleration, we have already shown application examples that map AI functions in the CPU, for example object recognition in a video stream. The opportunity for us is great: we can enter into discussions with our customers and demonstrate the benefits of a neural network engine in an HMI. For example, a rotating component in the machine could be monitored via microphone and AI and, if necessary, error and warning messages could be issued via the operating unit. This has advantages in terms of the service life of such devices, increases the efficiency of a system and ultimately has an impact on customer loyalty.

AI naturally goes hand in hand with the issue of data protection. I believe that development activities will have to be much broader in the future than they have been in the past. Developers will have to think much more about the impact of an innovation and the legal boundaries that may be reached. Whereas in the past you had to think about machine safety, today you have to consider which data protection guidelines could potentially be breached. There are many gray areas and regulatory challenges.

Six or seven years ago, we were dealing with capacitive touchscreens, but today we are thinking beyond that: the possibilities that arise are simply increasing and this is an incredibly exciting time.

You just have to learn to accept the complexity...

Meyer-Loges: ...or reduce it in the sense of conscious simplification, especially in areas where it can no longer be controlled. Perhaps in the end there will no longer be one person who masters everything, but you will have to focus on what suits you best.