A great deal has been written, posted or spoken about the potential of augmented reality (AR), especially in recent months and years. Almost every company now knows that AR applications can not only be used to boost marketing and product sales, but can also bring real added value to the design, production or service areas. In conjunction with the Internet of Things (IoT), new business models are also possible, or at least the expansion of a company's own range of products and activities.

However, what has so far remained hidden from companies is a concrete approach to implementation - a kind of guide with the first steps and the tools and resources required. What data do I need and what sources does it come from? Which tools are used to process it until I can see the finished result on my smartphone, tablet or via data glasses?

Numerous conversations with interested companies have shown that the financial and time costs of potentially complex and difficult-to-use software tools are also estimated to be very high.

But in practice, things are different. With the right tools - such as Creo Illustrate and Thingworx Studio from PTC - it doesn't take much effort to create your own AR experience. What's more, the only data source required is the CAD program or a complete 3D CAD model of the desired object.

Example: Dismantling a hand-held circular saw

For example, the disassembly of a hand-held circular saw can be supported by augmented reality by displaying all the necessary steps - using data glasses, for example. Initially, an application is considered for all models in a series and therefore without individual device recognition. Assembling the appliance is also possible in return and could form a second part of the application. Trainees or prospective service technicians at a company can use this application to learn how to correctly disassemble the device by having the steps displayed on the device in front of them via a tablet, for example. The application can also be used directly during service work: After all, thanks to this technology, all service technicians no longer need to know every model down to the last detail. Last but not least, customers benefit: For example, when the saw blade needs to be changed or the appliance needs regular care and maintenance. Nobody wants to wade through complicated, multi-page operating instructions. Visual instructions on a smartphone or tablet simplify the procedure many times over.

Defining the sequences

Creo Illustrate is used first to animate the individual disassembly steps; the AR application is then created in Thingworx Studio. As not every company works with CAD software from PTC and Creo Illustrate is absolutely essential for this, the software is an integral part of the Thingworx Studio starter package and also allows 3D CAD models to be implemented from other CAD programs.

Users can animate individual elements in Creo Illustrate step by step, such as the red lever shown here.

© PTC

First, the complete 3D CAD data of the circular handsaw is loaded into Creo Illustrate. The user then sees a digital twin of the circular handsaw in front of them. They can now begin to define the individual steps and sequences of disassembly and have them displayed graphically step by step - for example, which screws are to be loosened and how, or how the saw blade or the housing are to be disassembled.

For visualization purposes, the user can choose from various design options in an editor. For example, individual parts can flash, move in certain directions, rotate or fly away from the device. Even screwdrivers and other tools can be displayed. The duration for which each graphic element is displayed and the order of these sequences can also be set. All settings can be checked continuously in the preview until the device has been completely disassembled.

Once the entire animation for the disassembly of the device has been created and a name assigned, both the 3D CAD data and the animation itself are transferred to the Creo Viewing format PVZ and loaded into Thingworx Studio. For the creation of basic AR applications, it is also possible to pull PVZ files directly from Creo or Windchill. However, it is not possible to create animations in this way. For other CAD programs, the solution contains corresponding PVZ file adapters, for example for Catia.

Determine basic parameters

The first steps in Thingworx Studio involve creating a new project, including the target server from which the application is to be loaded. Another important step is the selection of the medium via which the AR experience is to run - smartphone, tablet or data glasses.

Once all these basic parameters have been assigned, the model of the circular saw is placed in the 3D space in front of the user. This is done using the widget and information bars on the right and left of the user interface. 3D and 2D mode each have their own set of widgets, which are necessary for further processing. In 3D mode, the exact position in the room and the size scaling of the object can be determined and the animation loaded, among other things.

In the next step, a thingmark - similar to a QR code - is placed on the device and uniquely defined for this application. By defining this 'marker' for the application 'Dismantling the hand-held circular saw' and later attaching it to the physical device, this exact application can be loaded simply by scanning this Thingmark using mobile devices. The starter package contains a basic pool of 1000 Thingmarks. Finally, in 3D mode, information can be displayed in the space above, next to or below the saw, for example a name for the application itself.

Designing the user interface

The user then switches to 2D mode to design the subsequent user interface. Depending on whether smartphone, tablet or smart glasses output was initially specified, a 2D interface now appears in exactly the selected format. Here, the user has the option of creating and graphically editing buttons for starting and stopping the AR animation or for further information, adding labels and creating all other elements that are necessary or helpful for handling and running the AR application.

Finally, a password can be assigned for the respective application. Once all configurations have been saved, nothing stands in the way of the AR experience. With a little practice, AR applications can be created in just a few minutes - assuming little animation effort. The starter package also includes a wide range of sample projects. Companies that are developing their first own applications can learn many design variants and one or two tricks here.

If something changes on the object and, for example, a new handle is installed, the program understands these changes and embeds them accordingly. For more extensive changes, only the animation sequences should be checked and adjusted if necessary.

IoT - the next level

Caterpillar uses AR applications in the maintenance area: engine temperature, oil level or mileage to date are displayed on the corresponding end devices.

© PTC

In addition to general applications such as operating instructions, it is also possible to customize the AR experience and view a single, specific product. For example, Caterpillar's service technicians can activate an AR application on their tablet by scanning the corresponding Thingmark and have real-time data such as engine temperature, oil level or previous mileage virtually displayed directly on the construction machine.

The whole thing works by integrating corresponding sensor data into the 3D space in Thingworx Studio. Similar to the assignment of labels and the widgets available for the further design of the virtual space in the first example, the user is shown a selection of all sensors available for a product or machine after logging in. These sensors or a display for the sensor data can be positioned and scaled as required in the virtual space. All further steps before and after remain the same; in the end, only the range of information that the designer can access is expanded.

In the case of more general instructions, it is generally possible to operate several models in a product range with one application and the corresponding Thingmark, for example if the steps for changing the saw blade from the original example should be the same for all models. In this case, only a selection menu appears after activating the app to select the exact device model with all its design elements or a model in a corresponding configuration. However, at the latest with the individual recognition of objects in combination with real-time data, the pool of available Thingmarks must be expanded accordingly in order to be able to include entire product series. After all, in the Caterpillar example, every excavator and every bulldozer requires a Thingmark.

In practical use

In practice, there are already numerous other examples of the use of AR for monitoring and maintaining technical equipment. The racing car manufacturer Griiip from Italy monitors its racing cars using a digital twin that is displayed on a tablet, including all telemetry data, after the engineer has scanned the Thingmark on the car.

Schneider Electric enables its service technicians to virtually superimpose various statistical values such as location, temperature, battery status, possible further service life and much more on a tablet over the UPS system in front of them, which is installed in a microcomputer center, for example. If the status indicator changes to orange, maintenance is required. The service technician receives clear instructions on how to proceed. This allows them to complete their task without errors.

Creating your own AR applications can actually be quick and easy if complete 3D CAD data is available. PTC, for example, offers companies the Thingworx Studio starter package free of charge for 90 days for testing purposes. The first costs for use are only incurred once commercial offers have been developed and implemented.

Author: Andrea Hallscheidt is an application specialist at PTC.