Part 2 of the TSN series
Open solutions for TSN
Time-Sensitive Networking (TSN) is regarded as the key technology for convergent IT and OT networks. Open solutions can contribute to the development of the ecosystem, but which software functions are actually required and which solutions are already available?
TSN, as part of IEEE 802.1 Ethernet, is a network standard and correspondingly relevant for the communication infrastructure. However, TSN is even more important as a basic technology for convergent IT/OT networks and therefore for the endpoints on which future applications will be implemented. Just as is the case with Ethernet today, the greatest potential - and therefore also business and revenue - lies in the applications running over the network. No business model on the Internet is differentiated from the competition by a better Ethernet driver - and this will not be the case with TSN for innovative industrial applications either. Interoperability, stability, robustness, rapid development of the ecosystem and market penetration are much more important for all participants in a TSN network than their own 'better' implementation.
Open solutions can make a decisive contribution to this. Because they make it possible:
- Directly deployable implementations,
- 'Inspiration' and reference for proprietary solutions,
- Support standardization through early testing and validation,
- the basis for developing innovative applications and business models before the entire ecosystem is available.
Many different manufacturers and initiatives are therefore actively contributing to open source solutions. These include projects such as AccessTSN (Linutronix, Hirschmann, ISW and HS Offenburg), organizations such as OSADL (with IOSB and Kalycito), Linaro as a supporter of various hardware manufacturers and individual companies such as Intel.
Open source is uncharted territory for many in the automation industry and is often still viewed very critically. Currently, even basic technologies are often proprietary solutions and reliable cash cows for some companies. Furthermore, in the automation industry, open source is often seen as a community driven by idealistic hobby programmers. A look at other industries - whether IT or mobile - shows that open source is definitely an enabler for innovative, efficient, reliable and sustainable applications and business models. Accordingly, all those involved in the digital transformation of the automation industry should endeavor not to get lost in the details of basic technologies, but to focus on innovative applications.
TSN endpoints
Open source solutions are primarily highly relevant for TSN endpoints. However, there is no such thing as a typical endpoint. Instead, the degree of TSN functions supported by the respective endpoints can vary greatly depending on the application. Endpoints can be categorized according to various aspects:
- Significant differences in the available computing power result in the decision as to whether an operating system or a bare-metal implementation is used.
- The deterministic requirements can range from 'just connectivity' to the network to nanosecond-precise synchronization.
- There are also major differences in the hardware support of the TSN functions and in the number of ports.
The image above shows a generalized overview of the structure and functions of a TSN endpoint. But which functions exactly are required in the TSN endpoints and which open solutions are already available for these functions today?
The real-time operating system
In industrial applications in particular, the endpoints must fulfill deterministic requirements in addition to the communication network. If sufficient resources are available - so that no bare-metal solution or a minimal operating system has to be used - Linux with the real-time extension Preempt-RT is a good option. Thanks to the constantly growing ecosystem, a large community and the increasing integration into mainline Linux, solutions for many tasks are already available, broad hardware support is guaranteed and long-term support is ensured.
Time synchronization
A common time base is essential for distributed real-time applications in a TSN network. There are open implementations for the synchronization protocols used for this, IEEE 1588, the IEEE 802.1AS based on it and the revision coming out this year. Linux PTP (ptp4l) is of central importance here. This fully supports IEEE 1588 and 802.1AS-2011 for endpoints. Missing features for full support of bridges and the new functions in the context of 802.1AS-2020 are currently being discussed and developed. Linux PTP can work both as a pure software solution or with hardware support. Likewise, thanks to appropriate mechanisms, the local real-time system can be synchronized with the synchronized clock, which can be located in the network card.
Traffic management
To ensure deterministic communication, various mechanisms are used to control data traffic in the context of TSN. The most relevant are the time-controlled transmission of data, for example for streams, and the class-based transmission of data according to a schedule (IEEE 802.1Qbv).
The basic communication architecture for Ethernet remains the same in Linux even with the addition of TSN features: Applications use sockets to transfer data to the communication stack, which transfers the data to the network card using hardware-specific drivers. Structures for sorting and prioritizing packets already exist within the stack. Queueing disciplines (qdiscs) are used for this purpose, which are organized hierarchically and can be configured depending on the application.
Two mechanisms in particular are of central importance for the time-controlled sending of data: Firstly, a new socket option has been introduced in Linux, which allows packets to be given a send timestamp (launch time). The newly introduced qdisc etf (earliest txtime first) is used to ensure that packets are sent at the desired time. This sorts packets to be sent according to the time stamp and ensures deterministic communication. If available, hardware-based time-controlled transmission mechanisms can also be used (hardware offloading).
The new qdisc taprio can be used for class-based transmission in accordance with an IEEE 802.1Qbv schedule. This allows the Linux priority classes to be mapped to network priority classes and time slots. The image above shows an example configuration of the mechanism according to the schedule shown. Hardware offloading is also possible here. Efficient configuration and thus utilization of the new mechanisms is currently the focus of various activities.
The multiport support
Devices with multiple ports are particularly widespread in the automation industry and will also be found in TSN-based networks. In the past, these were usually proprietary and hardware-dependent solutions. For TSN, there are already open approaches for abstracting and using this device class in a standardized way. For details, see also page 5 "Linux-based switched endpoints".
Protocols on higher layers
Florian Frick is group leader for real-time communication and control hardware at ISW Stuttgart.
© ISWAs part of Ethernet, TSN covers layers 1 and 2 of the OSI model as well as time synchronization. Applications therefore always require protocols on the higher layers. For classic IT applications, the established standards such as TCP, IP, http and similar are still used, for which there are well-known open solutions.
OPC UA plays a central role in the automation industry. While there is industry-wide agreement on the use of OPC UA for communication from controllers to IT and usually also from controller to controller, there are two competing camps with regard to the field level: some favour standardization up to the application level with OPC UA, while others strive to use existing fieldbus protocols via TSN. In addition to other open implementations, there is a project for OPC UA that is supported by a broad industrial community and also supports the pub sub-standards: open62541. An introduction to the project can be found in the report on page 6 "OPC UA PubSub via TSN".
Ready to get started?
In summary, it can be said that many TSN mechanisms are already supported today and integrate seamlessly into the existing ecosystem. For initial experimental use or as a foundation for future projects, it is possible to work with a wide range of PC and embedded hardware that is already available. To make the start as easy as possible, a sample configuration is offered for free use as part of the Access TSN project(more information).
TSN in Corona times
There is no question that the TSN community has also been hit hard by the effects of the pandemic: The number of canceled meetings of IEEE, IEC, OPC Foundation, FLC Group and the testbeds is increasing daily and many tasks to be completed will therefore certainly take a little longer than hoped. Nevertheless, progress is being made, whether in terms of standards such as AS, configuration or available products.
Nevertheless, there is no reason for potential users of the technology to delay the development of TSN solutions. The fundamentals are sufficiently stable and solutions are already available to start developments.
In line with this, this second part of this series of articles now looks at open solutions for TSN. If you are already working on a prototype yourself or are inspired to do so, you are probably wondering how you can test the latest developments in a meaningful way. We will address this soon in the next part of the series.
Yours, Florian Frick and Meinrad Happacher
Linux-based switched endpoints
A switched endpoint is a device that usually has two ports, rarely more. Such systems are often found in industrial environments, as the devices can be connected to each other without the need for additional switches. In the past, this class of devices was usually implemented using special, proprietary solutions. From a device-internal perspective, these endpoint and switch functions must be implemented. In the context of open solutions for TSN, the question therefore arises: To what extent can switched endpoints be implemented with Linux?
To answer this question, two aspects need to be considered: Switch and Endpoint. As far as the endpoint is concerned, the Linux kernel currently already supports both the implemented TSN standards and the extensions for deterministic network communication. All these implementations integrate seamlessly into the Linux network stack and the existing programming interfaces. The support of end devices for TSN networks is therefore a given.
Kurt Kanzenbach is a Linux system software development engineer at Linutronix
© LinutronixWhat about the switch functionality of the endpoints? The Linux kernel has been providing frameworks for connecting Ethernet switches for some time. These frameworks are the Distributed Switch Architecture (DSA) and switchdev. What both have in common is that each switch front port is mapped as a regular Linux network interface. This makes it possible to use each port separately. This means that in the standard configuration, no traffic is routed between the ports. If this is to be the case, a bridge must be created. A bridge is also a network interface that makes it possible to connect the switch ports with each other. As the switch configuration in Linux works on the basis of network interfaces, the existing TSN endpoint implementation can be reused 1:1.
The resulting Linux switched endpoint architecture is shown in the image above. The individual applications, regardless of whether real-time requirements are present or not, participate in the network communication via the bridge interface. The switch decides via which physical port the packets are sent.
A TSN-capable switch also includes time synchronization protocols such as the Precision Time Protocol (PTP) or management protocols such as the Spanning Tree Protocol (STP). Here it is necessary to be able to send packets specifically to individual switch ports. The port interfaces can be used for this purpose. The TSN configuration is carried out using the standard Linux utilities.
The architecture shown allows the realization of switched endpoints based on the Linux operating system. There is already support for TSN-capable switched endpoints in the Linux kernel. These include switches from Texas Instruments or NXP. As the existing interfaces and frameworks are generic, more will follow.
OPC UA PubSub via TSN
The example of a generated binary code from the JIT compiler for the update of OPC UA PubSub messages in open62541.
© Fraunhofer IOSBThe open62541 project is a community-driven implementation of the OPC UA protocol. OPC UA, standardized as ISO/IEC 62541, defines a client-server protocol for interacting with a server-side object-oriented information model. The possible interactions include reading/writing variables, calling methods, instantiating/destroying objects, subscribing to notifications about variable changes and events, query mechanisms. In addition to the client-server protocol, an extension of OPC UA was published at the beginning of 2018 to include the publish-subscribe communication paradigm. OPC UA PubSub means that content from the information model is bundled into telegram messages and sent cyclically to subscribers.
PubSub can also be configured at runtime using an OPC UA client. The distribution of telegram messages to subscribers can be implemented using a range of different protocols. So far, the mapping has been defined for MQTT, AMQP, UDP/multicast and Ethernet. At the Ethernet level, TSN now makes it possible to achieve hard real-time for OPC UA PubSub.
But how is 'OPC UA PubSub' implemented in open62541 in order to guarantee PubSub real-time capability and still achieve a tight coupling of the data sources and configuration to a (non-real-time capable) OPC UA server? This development was funded by a consortium of industrial partners as part of the 'Open Source in Automation Development Lab' (OSADL). It was implemented by Fraunhofer IOSB - one of the co-initiators of open62541 - and the Indian embedded service provider Kalycito.
Julius Pfrommer is Group Leader Cyberphysical Distributed Systems at Fraunhofer IOSB.
© Fraunhofer IOSBOn the open62541 core library side, it has been added that the PubSub configuration can be 'frozen' at runtime and 'unfrozen' for adjustments. With a stable configuration, the structure of the PubSub message can be precalculated. Type checking in the information model is used to ensure a fixed length of the protocol messages and known offsets of the data fields. In a final step, known offsets and memory addresses from the OPC UA information model are used to generate binary code for updating the PubSub messages using a JIT compiler (based on GNU Lightning).
This binary code can be bound to an interrupt handler that is cyclically triggered by a network-synchronized time signal. The execution time is minimal. Not even case distinctions or jump instructions are necessary to update the PubSub message. Nevertheless, the full flexibility of the PubSub configuration remains at runtime via the information model of the OPC UA server.
Two new developments from the real-time Linux kernel can be used for integration at system level: Firstly, the ETF extension (Earliest TxTime First) to send messages at predefined times. And on the other hand, XDP (eXpress Data Path) for resource-efficient processing of network packets.
It is important to note that the control flow of interrupts is determined for the synchronization of the application to the TSN network clock. Interacting with the remaining non-real-time capable application logic without locks - for hard real-time capability - is a challenge for potential users of TSN. The open62541 library thus simplifies the implementation of TSN-enabled applications; the API and developer examples guide the user in applying best practices to TSN.
Sample code and comprehensive documentation for implementing your own applications from OPC UA PubSub via TSN can be found on the website of the open 62541 project. All that is required for use is an environment with real-time Linux and freely available network hardware with TSN capabilities (IEEE 802.1 ASRev and Qbv). Long-term round-trip measurements for operation with cycle times of around 100 μs demonstrate the performance of the approach.





















