Listing 2: Starting the LinuxPTP time synchronization in IEEE 802.1AS configuration

© ISW

In the example, time synchronization according to IEEE 802.1AS is used. LinuxPTP already includes a suitable configuration file. This is copied from the installation directory to the configuration directory after installation. For time synchronization, ptp4l is started with the 802.1AS configuration file on the desired network interface (in the example an Intel i210) (see Listing 2).

Listing 3 Configuration UTC-TAI Offset

© ISW

The keyword -2 specifies the OSI layer for PTP communication; Ethernet is specified for IEEE 802.1AS in this example. The parameter step_threshold=1 ensures faster convergence.

LinuxPTP synchronizes the hardware clock in the network interface (PHC) with the other network participants. To be able to refer applications to this synchronized network time, the system clock must be synchronized to the PHC. As the PHC uses international atomic time (TAI) and the system clock uses coordinated universal time (UTC) as the time base, the offset between the two time bases must be configured. The command in Listing 3 can be used for this purpose.

Listing 4: Starting time synchronization between PHC and system clock

© ISW

The next step is to start the synchronization of the system clock with the phc2sys tool (Listing 4). To do this, the PHC to be used must be specified as the master and the system clock as the slave. The parameter step_threshold=1 can also be used with phc2sys to ensure faster convergence. In addition, the keyword w can be used to wait for the PHC to be synchronized by ptp4l.

The Network Time Protocol (NTP) must be deactivated on the system to avoid interference with phc2sys. Likewise, only one instance of ptp4l and phc2sys may be executed at a time.
The two time synchronization tools mentioned can also be executed as system services; to do this, the service configurations must be changed in accordance with the start commands shown.
Depending on the system configuration and workload, the execution priorities of the time synchronization services should be increased so that these services receive sufficient computing time. An example of this can be found in the 'Pub/Sub_realtime' example from open62541.
As the network interfaces have to be restarted when configuring the network queues and this can lead to time jumps, the network configuration should be completed before starting the time synchronization.

Time-controlled data transmission

Philipp Neher is a research assistant at the ISW in Stuttgart.

© ISW

In Linux, support for controlled data transmission of TSN is implemented in the Linux Traffic Control (TC) system. Various queuing types and principles (queuing disciplines, qdiscs) ensure the sorting and transmission times of the data packets to be transmitted. Several of these can be used for TSN. For the most part, the qdiscs and the settings shown relate to the transmission path, so they are not important for pure data receivers.

Send at a specific time (ETF)

Time-controlled transmission can be used in TSN to implement cyclical streams, for example. A data packet is sent at a specified time. The application must determine this sending time before sending the packet and pass it to the Linux network layer together with the packet content. The Linux network layer uses the ETF-Qdisc (Earliest TXTime First) to ensure that the data packet is sent correctly. Hardware support can also be used to achieve greater precision. This is supported by the Intel i210 network controller, for example. The function is called 'LaunchTime'.

Listing 5 Configuration of queues and assignment of network priorities

© ISW

The ETF Qdisc works separately for each queue and can only be activated or configured for individual queues. To use ETF, the queues and the assignment of Linux socket priorities to the queues must therefore be configured first. This is done via the Qdisc MQPRIO. This realizes the assignment of the Linux socket priorities totraffic classes and finally to hardware queues (if available). Listing 5 shows a configuration example using an Intel i210 with four queues.
The command line tool tc is used for the configuration.

The command shown creates an MQPRIO Qdisc at the lowest position of the i210 network interface with the ID 6666. Furthermore, the num_tc 3 command defines three traffic classes (class 0 to class 2). The 16 available Linux socket priorities (0-15) must then be assigned to the traffic classes. To do this, the corresponding traffic class is specified after the 'map' keyword for each socket priority.

In the example shown, socket priority 3 is assigned to traffic class 0 and socket priority 2 is assigned to traffic class 1. All other socket priorities are assigned to traffic class 2. The traffic classes are then assigned to the queues using the keyword 'queues'. The traffic classes must be configured in strict ascending order. The first entry therefore applies to traffic class 0 and so on. The respective entry specifies the number of queues with offset. In the example, the first queue (i.e. queue Q0) is assigned to traffic class 0 and the second queue (i.e. Q1) to traffic class 1. In the third entry, two queues from offset 2 (i.e. queue Q2 and Q3) are assigned to traffic class 2. Finally, hardware offloading must be activated via 'hw 1'.

Listing 6 Configuration ETF Qdisc

© ISW

Once the queues have been created, the ETF-Qdisc can be configured (see Listing 6). The example shown creates an ETF-Qdisc on the first queue created in the previous example (ID 6666:1). The clockid parameter defines the clock to be used for time-controlled sending. The CLOCK_TAI used is the only one supported. To function correctly, this clock must be synchronized (see section Time synchronization).

The parameter 'delta' specifies the time span by which a packet is transferred to the hardware before the specified transmission time. This time period must be determined separately for each system. It can be estimated using the cyclictest tool. Details can be found in the "TSN Documentation Project for Linux". The example uses a time span of 500 µs. Finally, the keyword offload specifies the activation of hardware support.

Sending in traffic classes according to Qbv (TAPRIO)

Listing 7 Configuration TAPRIO Qdisc

© ISW

In addition to time-controlled transmission, Linux supports the TSN functionality "Enhancements for Scheduled Traffic" (IEEE802.1Qbv). Each traffic class is provided with a time slot for transmission. Under Linux, the functionality is implemented in the TAPRIO-Qdisc (Time Aware Priority). This is configured in a similar way to the MQPRIO Qdisc (see ETF section). In addition, the time slots are parameterized as a gate control list. Due to a lack of freely available network interfaces with Qbv support, the example in Listing 7 does not use hardware offloading.

The 'base-time' parameter specifies the start time of the schedule in nanoseconds. The 'sched-entry' entries configure the time slots. Each entry represents a time slot. The keyword 'S' opens the traffic classes specified afterwards. The traffic classes are specified as hexadecimal parameters, where each bit represents a traffic class. The time slot entry is specified by entering the time slot length in nanoseconds.

In the example, three time slots are therefore parameterized. Traffic class 0 is opened during the first 300 µs time slot, traffic classes 0 and 1 are opened during the second 300 µs time slot and traffic class 2 is opened during the 400 µs time slot. The 'flags' and 'txtime-delay' parameters activate a TxTime-assisted mode and the associated time period. The 'clockid' parameter specifies the clock to be used for the time stamps.

The implementation of the application

Listing 8: Setting the socket priority for TAPRIO (as in the example to priority 3) in C

© ISW

The development of an application that is to be executed in real time on the described system and use the described TSN functionalities can be carried out on the basis of the source code examples in the Realtime Linux Wiki. The described cyclic real-time thread should always use the CLOCK_TAI as a clock for synchronization. Before the start of the real-time cycle, the application may have to wait for the start of the schedule (for TAPRIO) using clock_nanosleep(). For the application with a network schedule, the application can open a normal socket with the desired type and then adjust the socket priority according to the Qdisc configuration (Listing 8).

Listing 9: C example of time-controlled sending: Using the SO_TXTIME socket option

© ISW

To use time-controlled sending, the 'SO_TXTIME' option must be activated for the socket. A previously calculated transmission time must be transferred with each data packet. When calculating the send time from the current time, a certain period of time for the execution of the Linux network stack must be taken into account. CLOCK_TAI should also be used here as a clock. The execution priorities of the Linux network stack and the real-time application must also be adjusted accordingly (Listing 9).

A basis for many applications

Simple, distributed real-time applications based on TSN can already be implemented today using the tools described. What exactly these look like is very application-specific and must be decided on a case-by-case basis. A complete example, which includes several endpoints and illustrates a typical automation scenario, can be found on the website of the 'AccessTSN' project [2].

References:

[1] https://wiki.linuxfoundation.org/realtime/documentation/howto/applications/start

[2] https://www.accesstsn.com/

[3] https://tsn.readthedocs.io/

[4] https://wiki.linuxfoundation.org/realtime/documentation/howto/applications/start

[5] https://github.com/open62541/open62541/tree/master/examples/pubsub_realtime