TSN series part 15
Distributed real-time applications based on TSN
Time-Sensitive Networking (TSN) is one of the key technologies for future distributed real-time applications. In addition to the actual TSN-based communication, the ecosystem and its components must be considered for successful implementation.
The extensions to the Ethernet standards known as TSN are communication technology on the lower layers of the OSI model. However, the cross-industry interest in the technology and the associated technological visions has its origins in the distributed (real-time) applications enabled by TSN. These are characterized by the fact that they are distributed across several local applications on physically separate devices and are connected via a real-time capable network. In the industrial environment, these are typically control or monitoring tasks; in the automotive sector, for example, vehicle control or infotainment. Several independent applications can potentially share an infrastructure - both network and endpoints.
Use case from industry
What does a practical application scenario of a distributed real-time application in a TSN-based ecosystem that uses TSN and other current technologies such as virtualization using containers look like?
Assume three assembly stations, each of which has a two-axis manipulator and is controlled by PLC controllers. Unlike today, the PLC controllers are not to be implemented as individual devices, but as virtual controllers (vPLC) on an edge platform. A Linux PC with real-time patch (Preempt-RT) is used as the platform. The individual drive controllers are designed as embedded devices. Real-time critical target and actual values are exchanged between the vPLCs and the embedded devices at 1 ms intervals. In addition, further data can be transported sporadically in the network, such as parameters or status data for use in a production monitoring system.
OPC UA PubSub is used on the upper protocol layers for real-time data and Client Server for other data. The results of the FLC initiative could potentially be used for this in the future.
TSN is used to transport the various data in the same network, with real-time being achieved by using a time-based schedule with streams and data classes. Among other things, 802.1Qbv is used for this. In the case described here as an example, this schedule consists of a total of three classes: One for the isochronous exchange of target and actual values (ISO), one for the prioritized transport of parameters (PRIO) and another class for sending monitoring data according to the best-effort principle (BE).
The application is to be configured, deployed, managed and monitored by an engineering tool. A central task is the configuration of the TSN-based communication, which is realized by the CUC (Central User Configuration). This has the task of forwarding the communication connections defined by the user to the central network management and then transferring the resulting configurations to the end devices - in this case the vPLCs and embedded devices. Communication between the end devices and the CUC is application-specific, which is why a solution based on OPC UA is also used in this case.
The network as such is managed using the central configuration approach of TSN, for which a central network management - Central Network Controller (CNC) - is used to configure the infrastructure components (e.g. switches).
Components of a distributed real-time application
As shown in the representative use case, the ecosystem of a TSN-based real-time application is based on a large number of components, which are considered individually below.
The application
The first component is the distributed real-time application as such. It is made up of several local applications and the communication between them. The local applications correspond to typical real-time applications, but are synchronized to a common master time. The development as such, as well as the usable system environments, hardly differ from classic real-time applications. The operating or runtime systems used, such as Linux with Preempt-RT Patch and potentially real-time-capable container solutions, are particularly decisive.
The interfaces between the individual local applications must be specified both in terms of the data to be transferred and their properties. The boundary conditions for data transmission must be defined in the form of so-called Quality of Services (QoS). These include the transmission interval, the packet size and the earliest and latest possible offset of the interval start time for sending a message.
OPC UA, for example, is suitable for the transmission of messages, in particular due to the introduction of the Publish-Subscribe (PubSub) specification Part 14, which enables the application to provide (publish) data at a definable interval. Other applications in turn subscribe to this message and thus receive new values at regular intervals. UDP, AMQP and MQTT are used as transmission protocols. In the future, the Part 22 specification will include a description of TSN data streams and QoS as a nodeset for OPC UA.
Various solutions are available for integrating an OPC UA stack into applications, such as the open-source library open62541.
|
Focus on the big picture |
|---|
|
In the previous parts of our TSN series, we have looked at various aspects of TSN: from time synchronization and configuration to endpoint solutions. Technologically, these topics are highly relevant, but they have not yet resulted in a usable product or a functioning business case. Nevertheless, TSN is seen as a key technology for future-oriented innovations across all industries. As with any basic technology, the overall system around it is crucial for success. In this issue, this system approach will be analyzed and structured. In addition to gaining an understanding of the system for TSN applications, interested parties will gain an entry point for their own planned implementations. We hope to gain additional clarity on how such ecosystems can be designed, as well as information on specific applications, from various sources this year: On the one hand - as reported in the 9/2021 issue of Computer&Automation - from the profiles and on the other hand also on the application side from the work within the FLC group of the OPC Foundation and other organizations. As always, we welcome any feedback, comments or suggestions on this series of articles! Write to us at: redaktion[at]computer-automation.de Meinrad Happacher and Stefan Frick |
Endpoint and infrastructure
The individual local applications are each distributed to their own (physical) end devices. The entire infrastructure, consisting of the end devices and the communication infrastructure in between, is another aspect to be considered here.
The endpoints, in the use case described, the edge platform, the embedded devices and the switches (and other infrastructure components) must be able to be configured according to the requirements of the application and the communication so that the data is forwarded from the device at the right time.
Various queuing disciplines (qdiscs) exist under Linux for this purpose. Data communication can be controlled (in terms of time) by distributing the traffic across several buffers and using other mechanisms.
For strict timing requirements, it is advisable to use TSN-capable hardware. In addition to supporting time synchronization via hardware clock, special transmission mechanisms, such as those offered by the Intel i210 or i225 chips, can be particularly advantageous for precision. A comprehensive endpoint solution for the use of TSN and OPC UA on end devices already exists: it was developed in the Access-TSN project and is available as open source.
Engineering of the distributed real-time application
The actual distribution of the application in the network and the definition of the communication routes is carried out by means of engineering. This can be done by a user in a suitable tool, for example. In a first step, the network can be searched for connected devices (topology determination) in a similar way to familiar engineering tools from industrial control technology. The individual applications - in this example, the applications of the controllers and the individual drives - are then mapped to the respective end devices in the network. Once the application has been distributed, the streams still need to be defined. One possible way of doing this is to visually display the network in the engineering tool and define a connection between two end devices using drag-and-drop. These streams are further specified by the user using additional settings, such as priorities.
Suitable tools for this have so far been proprietary in-house developments. An open solution that either provides a complete tool or at least offers the necessary functions for extending existing engineering tools is conceivable and, above all, desirable in the future.
The author: Stefan Oechsle is a research assistant at the Institute for Control Engineering (ISW) in Stuttgart.
© ISWDeployment
The transfer of applications to the respective end devices and their final execution can be done manually, but modern concepts - keyword: containers and virtualization - offer some advantages over the conventional approach. Instead of buying suitable hardware to run a special application, such an application can also be run in containers on standard hardware. Virtualization allows different systems and hardware to be emulated according to requirements. Such approaches greatly increase modularity and, in particular, flexibility.
By encapsulating an application in containers, it is largely independent of the rest of the system. In addition, rolling out the application on a device (deployment) is relatively simple: a few command line commands are sufficient to load the desired container from a repository, instantiate it on the device, install the necessary dependencies and start the container and the application.
Common and above all open systems for containerization are Docker and Kubernetes.
The network configuration
However, the steps taken so far will only lead to a functional, distributed real-time application if the network is configured correctly. In terms of network configuration, TSN provides for three basic approaches in the IEEE 802.1Qcc standard: Centralized, decentralized and hybrid. The centralized approach provides for centralized network management using CNC. This is connected to all switches in the network and therefore knows the topology. The CUC, on the other hand, can be seen as an intermediary between the CNC and the end devices. The end devices - or triggered by the user or the tool - transmit their requirements to the CUC. From there, they are transferred to the CNC for calculation. The resulting configurations are then sent back to the end devices by the CUC and made available there. The CNC also configures the switches involved accordingly.
In the decentralized approach, there is no central network controller with knowledge of the infrastructure. The configuration is carried out using appropriate protocols, such as the SRP (Stream Reservation Protocol) or the RAP (Resource Allocation Protocol). The information from the streams is transmitted to all participants in the infrastructure using the corresponding protocol.
IEEE 802.1Qcc already defines the basic structures of the information to be transmitted for the configuration between CUC and CNC. In the standard 802.1Qdj, which has not yet been finalized, these structures are extended to a complete interface using RESTCONF.
The hybrid approach represents a mixture of the two variants, but is the least widely standardized to date and is therefore not considered here.
Monitoring and diagnostics
The ControlTSN approach: The modular structure is designed to ensure individual use of the framework.
© ISWThe components described for developing, distributing and deploying the application and configuring the network lead to an executable system.
To ensure that the system functions correctly, it is advisable to monitor the system. The synchronization of the devices and the properties of real-time communication are of particular importance here. The accuracy of the synchronization between the end devices can be determined using the reverse sync method. Individual slaves - end devices - are each defined as grandmasters in their own domain. By sending sync messages to the primary grandmaster, the grandmaster can calculate the time offset of the end devices. The jitter of the offsets in turn determines the accuracy of the time synchronization.
When monitoring real-time communication, the latency of the transmitted data should be considered. This can be done by calculating the difference between the send timestamp (Tx) and the receive timestamp (Rx) of a stream. However, jitter should also be monitored.
Open reference architecture
The efficient development of functional and stable TSN-based distributed applications requires a combination of the components described. Only through the interaction of these components can the three necessary views - network engineering, container engineering and application engineering - be covered.
In order to provide developers of future applications and the associated engineering tools with a basis, a modular framework for implementing these aspects is being developed in the ControlTSN project. The focus is equally on central engineering and local endpoints of this open solution. The components listed are reflected in individual modules, which are integrated by a framework core. A central data and function interface ensures the necessary inter-module communication. The focus here is explicitly on the design of a kind of modular system and not a single application or a single product. The components can thus be used for future projects according to individual requirements.
You might also be interested in
Focus on interoperability
Testbeds play a decisive role in the development of TSN towards a cross-manufacturer ecosystem. What progress has the TSN testbed at ISW made in recent years and what current challenges are currently being discussed there?
read more...First steps into practice
Is Time Sensitive Networking still grey theory or can TSN-based networks with controllers from different manufacturers already be implemented today? A setup with B&R and Beckhoff controllers was created at Kempten University of Applied Sciences....
read more...Call for Papers - submit presentations by April 29
The TSN/A Conference has become 'the' international information and discussion platform for real-time Ethernet par excellence. Become part of the community and submit your presentation to the presence event on September 28 and 29!
read more...Multi-vendor demo celebrates its premiere
Three years after its launch, the OPC Foundation's Field Level Communications (FLC) initiative has now completed the second release candidate - a multi-vendor installation demonstrates the progress made.
read more...From TSN testbed to convertible production
With the TSN Testbed, the ISW in Stuttgart is supporting the establishment of a deterministic and open network standard in production facilities. Florian Frick describes the status quo.
read more...TSN profiles in comparison
Work on TSN profiles is in full swing in a wide range of sectors. This article in the TSN series highlights the ongoing work on the profiles for the industrial, ProAV, automotive and aerospace sectors.
read more...It's the combination that does it!
Although TSN can be used to implement cross-manufacturer and cross-platform networking, it is only in combination with 5G that end-to-end real-time communication is also possible for mobile applications and cloud connectivity.
read more...An evolution update
TSN is, soberly speaking, a further development of the IEEE Ethernet standards. On the other hand, TSN is the basis for revolutionary changes in industrial communication. What is the status quo of TSN developments? What challenges still need to be...
read more...Virtualized control systems using TSN backbone
The major challenges in implementing virtualized control systems are scalability and connectivity to the field level. An implementation at the ISW shows that this can already be achieved today using a TSN network.
read more...