Parasoft
MQTT-based test adapter for industrial protocols
One approach to testing control systems uses an API test technology that sends MQTT-based messages directly to components that support industrial protocols such as ModBus, OPC UA. The resulting framework communicates directly with IIoT and backend IT systems.
Testing in industrial automation is often a manual task. However, increasing complexity and changing operating conditions require new testing approaches. This is where the methodology and technology from regular software engineering with its automated tests could be helpful in creating more robust and mature test processes. The Institute for Automation and Communication Ifak and the software company Parasoft are pursuing this approach in a joint project: they are combining their existing tools to enable testing in the context of industry-relevant use cases based on communication via Modbus and OPC UA.
The test concept
Figure 1: Illustration of the concept with a model-based test generator, SOAtest for test execution and a test adapter for communication with the system to be tested.
© ParasoftThe basic idea is to import abstract test cases from the model-based test tool MBTCreator developed by ifak into SOAtest, a tool developed by Parasoft for the execution and management of tests. In addition, SOAtest's MQTT capabilities are used to communicate via Modbus and OPC UA using a modular test adapter(Figure 1).
Model-based testing
In various research projects, ifak has focused specifically on the development and evaluation of a prototype toolchain for model-based testing(Figure 2). The toolchain uses methods of model synthesis, model-based test generation and test prioritization to implement an automated test process. This allows the systematic generation of a test suite from suitable test cases in order to validate all system requirements. The concept is based on formalized behavioral requirements, which are modeled on the basis of UML flowcharts using a specially developed requirements notation and serve as the basis for model synthesis. The result of the model synthesis is a graphics-based specification model that contains all relevant behavioral requirements and forms the basis for model-based test generation. The specification model is mapped using an extended Petri net and the known methods of Petri net theory are applied to automatically generate test cases using suitable graph-based test targets (all paths, nodes and edges). Depending on the complexity of the specification model and the selected test objectives, the test generation results in a test suite that comprises a large number of test cases and is characterized by a correspondingly high test coverage with regard to the relevant behavioral requirements.
The test concept - test adapter, SOAtest and workflow
The test adapter
The generated test cases are executed using a procedure that provides a protocol-independent interface for connecting the test tool to the system under test (SUT)(Figure 3). As the range of available protocols is extremely heterogeneous, particularly in the field of industrial automation, a generalized 'agent' is required to connect the test suite. The generalized test adapter developed for this purpose provides the test system with a uniform interface that decouples the communication structure from the test system. Communication with the SUT is realized in both directions using different communication protocols, e.g. OPC UA or Modbus. From the SUT's point of view, the test scenario is therefore no different from integration into a real operating environment. The approach to automated test execution is presented as a prototype using a demo application.
The test adapter uses MQTT as the medium for communication with MBTCreator. It was designed in such a way that the essential components of a test execution can be broken down into a common series of actions with a common language. In conjunction with a configuration that maps the specific characteristics of the system, communication with the various SUTs can be handled via the protocol used by the respective SUT.
In order to create a standardized language for test execution, each individual sub-step of test execution must in principle be mapped to a common language element. These sub-steps thus implement an action of the SUT that cannot be further subdivided. Such sub-steps would be, for example, the instructions 'Read sensor value X' or 'Stop'. If we examine test steps of this type more closely, we see that an action to be carried out by the system can be mapped to three types of actions as part of the test execution, namely executing a function, reading in values or writing values. Even if the steps required for this are heterogeneous and more complex in terms of communication with the system, this part of the execution is still transparent for the executing toolchain. In any case, reducing the chain to these three basic functions considerably simplifies communication on the test generation side, as the toolchain only needs to know these three operations.
The test adapter uses MQTT precisely for the reduction of actions just described. These elementary messages, which represent these basic actions, are implemented as various messages from and to the SUT. These messages are then converted into the protocol used by the SUT and used transparently as a communication element between the Test Suite and the SUT.
Test solution SOAtest
Parasoft SOAtest is a generic test solution for functional tests. It includes API testing for multiple interfaces (UI, REST & SOAP APIs, web services, microservices) and simplifies automated end-to-end testing (including databases, MQ, JMS, EDI, Kafka). The range of supported features and protocols can be expanded with plug-ins. Communication with the test adapter developed by ifak is possible via the MQTT interface, so that SOAtest can communicate via industry protocols such as Modbus and OPC UA.
The workflow
The model-based concept for test generation developed by ifak creates an abstract test suite in which a test oracle and the necessary test steps are defined in addition to the test data. The latter are built from a precisely defined set of actions (Read, Write, Assert) and can be converted into an equivalent test case representation in SOAtest for the purpose of test execution. This conversion can either be performed manually or by an algorithm.
In the first phase of the collaboration, a manual conversion of the abstract test cases was chosen(Figure 4). An abstract test case is shown on the left as a flow chart. Messages are transferred between the test system and the SUT. The corresponding test case in SOAtest is shown on the right. In a real test execution scenario, direct communication between the test system and SUT is often not possible because, for example, the test system does not support the necessary communication protocol or the communication has to be adapted to a very specific, SUT-dependent behavior.
SOAtest supports various interfaces and protocols, including MQTT, which enables it to communicate with the test adapter. With the help of the test adapter, tests for various communication protocols are possible with SOAtest. All relevant test steps can be implemented by using the native MQTT commands "publish" (for sending) and "subscribe" (for receiving) as well as the SOAtest-specific function "assert" for evaluating the content of a message. Another advantage of this concept is that the test suites can be used for different communication protocols if the underlying behavior of the SUT remains unchanged, e.g. if an SUT supports several different interfaces.
The communication protocols and their application
The following is a brief overview of the various communication protocols supported by the test adapter and their application. Based on the workflow outlined in the previous section, examples of integration in SOAtest for Modbus and OPC UA can be found here.
MQTT
MQTT is a lean machine-to-machine communication protocol that uses a publish-subscribe mechanism. Although originally intended for TCP, it can also be used with other lossless bidirectional protocols. MQTT supports encryption and authentication as well as various quality of service options. It guarantees the message sequence within a subscription topic and supports message persistence. A broker is also required as a centralized connection endpoint responsible for distributing the messages.
MQTT is widely used as a protocol for IoT devices, in the home automation sector and in other scenarios with small devices with a small footprint. In the test execution workflow, all communication between the test system (MBTCreator) and the test adapter takes place via MQTT. All relevant information for a test step is encoded in a JSON message, interpreted via the test adapter and converted into the required protocol. The data read and/or received by the SUT is in turn encoded by the test adapter as JSON messages and sent back to the test system.
Modbus
Modbus is a serial communication protocol for connecting electronic devices in the industrial sector. The protocol, which was specially developed for industrial applications, is relatively easy to set up and maintain compared to other standards and has few restrictions apart from the format of the data to be transmitted. Modbus uses RS-485 as the bit transmission layer.
- MQTT commands can be used to issue the following instructions to a Modbus device:
- Change the value in one of its registers, which is written to coil and holding registers.
- Read an I/O port: Read data from a discrete or coil port.
- Command the device to return one or more values contained in its coil and holding registers.
A Modbus command contains the Modbus address of the device for which it is intended (1 to 247). Only the device addressed in this way reacts to the command and executes it, even if other devices receive the command. (Exceptions to this are special broadcast-capable commands that are sent to node 0. These are executed but not confirmed). All Modbus commands contain checksum information, which the receiver can use to recognize any transmission errors.
Modbus is currently supported as a protocol for communication between an SUT and the presented test adapter.
OPC UA
OPC UA (OPC Unified Architecture) is a machine-to-machine communication protocol for industrial bus systems - an open, cross-platform protocol based on the principle of service-oriented architecture. It is divided into two main components, namely the information transport and the metamodel, which describes the information including the semantics. Various protocol implementations are supported for the transport, including security aspects.
The information model is a so-called full mesh network of nodes, with which meta and diagnostic information is represented in addition to the user data of a node. A node is similar to an object as known from object-oriented programming. For example, a node can have various attributes that can be read and written and provide historical data access. It is possible to define and execute methods with arguments and return values. OPC UA also supports events that can be sent to exchange specific information between devices. By combining nodes and hierarchical data, this protocol can represent different real objects - from small sensors to large industrial plants.
The test adapter in use
The following section provides an overview of two demo test suites in SOAtest. These are based on generated test case procedures and use the test adapter. Both test suites follow the general test case structure, but two different SUTs - one for each protocol - were tested.
Modbus tests in SOAtest
In the first case, an industrial quarter-turn actuator with integrated control unit communicating via Modbus is tested. The test case implemented in SOAtest consists of two steps: In the first step, the actuator is moved to a specific target position, while the second step checks whether the actuator has actually reached the target position.
First, an MQTT message is sent to the test adapter. It contains the specification of the desired Modbus register and the value to be written to the register, i.e. the target position. The test adapter translates this information into a corresponding Modbus operation, whereupon the value is written to the register. This step triggers the control unit of the SUT and causes the drive to move to the desired position.
In the next step, the actual position of the SUT is compared with the desired position. As the SUT is a mechanical system, the target position is only reached with a certain delay, whereby the time required depends both on the position itself and on the configuration of the SUT. In this example, it is only checked whether the SUT reaches the target position at all. How much time is required for this, however, is not the subject of the test.
In order to obtain confirmation that the SUT has reached the target position, a polling mechanism has been implemented in SOAtest. SOAtest sends MQTT read requests at a defined polling rate to the test adapter, which converts these requests into corresponding Modbus operations and reads the current position from the register. The read value is transferred to SOAtest and compared with the desired value. This polling continues until the expected value is reached or a specified period of time has elapsed. The following section contains a more detailed description of the MQTT write and read requests in SOAtest.
OPC UA tests in SOAtest
The second example shows the test execution via OPC UA. For this purpose, a simple OPC UA server was set up for demonstration purposes, which could have an interface to an industrial component or application in a real use case. The demo server consists of several nodes, including one node of type 'boolean' and one of type 'double'.
The test case is similar to the previous Modbus example. Since there is no real system behind the OPC UA server, it is not necessary to request SUT data as in the Modbus example here, but an example explains how data is read and written via MQTT and the test adapter.
Figure 6: Left: Test cases for the tests via OPC UA; right: the individual steps of a test case.
© ParasoftAs can be seen on the left in Figure 6, four separate test cases were implemented in two different SOAtest test suites. Just as in the Modbus example, test steps for sending (writing) data to the SUT and for receiving (reading) data from the SUT were also defined here. The four test cases write values of the types boolean and double to the corresponding nodes in the OPC UA server. The respective values are then read from the OPC UA nodes in order to check with an assert whether the value was written correctly in the previous step.
An SOAtest test element was used to send test data to the OPC UA server, which sends the data via MQTT. The message itself is defined as 'JSON' and contains additional information required by the test adapter. The various parameters required include the command type ('Read' or 'Write') and a further parameter containing an array with test data. This array contains the value to be written as well as a timeout, the polling rate and the accuracy. In the OPC UA use case, only the value to be written is relevant.
The second test step consists of an MQTT subscriber element in SOAtest and an assert step. In addition, the test adapter is triggered so that it can read data from the OPC UA server via another MQTT send element in SOAtest. A topic is defined for the subscriber to which the test adapter sends. When the test is executed, the subscriber is started first. An MQTT message is then sent to the test adapter, which triggers a request to read the desired value from a specific OPC UA node. The MQTT message for reading is created in a similar way to the write request, but the command is now defined as 'Read' and no parameter is required.
Once the test adapter has received the read request, the desired value is read from the OPC UA node specified in the 'mbt_signal' message property. An MQTT message with the relevant value is then sent back to SOAtest.
The MQTT subscriber receives the message, converts it into a format that can be parsed by SOAtest and compares the received value with a specified target value using an assert.
The next steps
The project demonstrated that SOAtest's MQTT capabilities are sufficient to communicate with the ifak test adapter and execute test steps for systems based on Modbus and OPC UA. The test cases generated by the test generator can be manually converted into a corresponding test suite and executed. The current approach demonstrates the feasibility of the concept of executing generated tests in SOAtest for different communication protocols supported by the test adapter.
Future work will develop the conversion of generated, abstract test cases into a fully functional SOAtest test suite. This concept could significantly enhance the seamless integration of an existing model-based testing tool into SOAtest and the current testing capabilities of both tools.
This project was carried out as part of the 'Eureka ITEA3 Testomat Project - The next level of test automation' (www.testomatproject.eu).
The authors
Karsten Meinecke is Head of Test Systems at Ifak.
Rix Groenboom is Strategic Innovations Manager at Parasoft.
You might also be interested in
An edge app store for the industry
At the end of April, the OWL Mechanical Engineering Initiative presented the Industrial APP Marketplace: an online platform for Docker containers. What is behind the Marketplace and what opportunities does it offer users?
read more...The dilemma of IoT security vulnerabilities
It should be a matter of course that identified and published security vulnerabilities in IoT products are immediately closed with updates. Due to a lack of technical possibilities, this is rarely the case. This option is unacceptable for industrial...
read more...IIoT communication from the sensor to the...
Data exchange from IO-Link via JSON and MQTT becomes possible
The IO-Link community has published a specification that describes how data from IO-Link can be exchanged via JSON REST and MQTT - in the sense of transparent communication in the IIoT.
read more...OPC UA, MQTT and co.
The Industrial Internet of Things (IIoT) covers very different areas of application. The available connectivity technologies and standards are just as diverse as the application areas. How can the right technology be filtered out in each case?
read more...Handy wiring and PoE++ tester
Softing IT Networks launches the 'CableMaster PoE', a handy wiring tester with integrated length measurement and tone generator. It also performs PoE (Power over Ethernet) tests up to class 8 and 90W (PoE++).
read more...Company management handed over
Olaf Römer, founder of ATEcare, has handed over the management of the company to Paulo Soares.
read more...A resource-based approach
The added value of TSN results primarily from IT/OT convergence in the end points. Exploiting the potential is still complex and requires in-depth knowledge of network technology. An application-centric, resource-based approach is presented as an...
read more...Energy self-sufficient IoT communication in 5G networks
In the Internet of Things, sensors act as core components for data acquisition and communication. How can they be connected as wirelessly as possible and supplied autonomously in order to make installations cost-effective?
read more...Alexander Graham Bell died 100 years ago
With the very first telephone, Alexander Graham Bell, who died 100 years ago, ushered in a completely new era of communication - which has since led to the age of smartphones.
read more...