For a long time, the computationally intensive tasks of image processing could only be handled by PCs. However, with the constant increase in performance of embedded hardware in recent years, smaller and cost-effective processor boards from the embedded sector are also coming into focus for handling even demanding image processing applications. Such boards in combination with a camera module are on everyone's lips under the term embedded vision, as they solve many vision tasks cost-effectively and efficiently.

The interface for the connection between the camera module and processor board plays a key role in embedded vision systems, as it must ensure fast and reliable data transmission. Proprietary and lean interfaces can generate high development costs, which is why standards are increasingly gaining acceptance in the embedded sector. Examples include Plug&Play-capable USB 3.0 interfaces or - especially for FPGA-based systems - LVDS interfaces. In various cases, however, one interface standard has proven to be particularly advantageous, which has so far established itself primarily in the field of mobile devices: the MIPI CSI-2 interface.

Standard for mobile devices

The MIPI Alliance (Mobile Industry Processor Interface) is an association of almost all manufacturers of applications or hardware in the mobile communications and consumer electronics industry. It has set itself the goal of standardizing all important interfaces between mobile processors and peripherals - such as position sensors, cameras, input interfaces and displays. This makes it easier for manufacturers of mobile peripherals to adapt their hardware to a wide variety of processor types. Conversely, this creates a larger range of potentially compatible peripherals for processor manufacturers. Both sides benefit and can develop and produce more cost-effectively.

An embedded vision system consisting of a 'Snapdragon 820' and a 'dart' camera module from Basler with BCON for MIPI interface.

© Basler

Two well-known and widely used MIPI standards are the DSI (Display Serial Interface) specification and the CSI (Camera Serial Interface). The latter currently plays a decisive role in embedded vision and describes three elements of an interface between the camera and other hardware in the MIPI CSI-2 specification:

• the physical layer of signal transmission (D-PHY or C-PHY),
• the CSI protocol based on this for image data transmission and
• an interface for camera configuration via I²C, the so-called CCI (Camera Control Interface).

The image data is transmitted serially via individual lanes. The maximum bandwidth can be scaled via the number of these lanes. In practice, two or four lanes are typically used.

The terms C-PHY and D-PHY denote the physical realization and the maximum transmission rate of each lane. D-PHY with differential transmission via two lines plus a clock line common to all lanes is currently widespread. This concept allows a maximum data rate of 2.5 Gbit/s per lane.

The protocol for configuration, the Camera Control Interface, is physically based on the I²C standard, but varies greatly in detail depending on the manufacturer.

In general, the MIPI-CSI-2 specification is designed to be very close to the hardware. In contrast to the standards from the GenICam family commonly used in the image processing market, however, MIPI CSI-2 lacks a standardized software stack, a standardized programming interface (application programming interface, API, such as GenAPI) and a standardized image data interface (such as GenTL).

MIPI advantages

As the mobile market is huge, the mobile processors used are produced in extremely high quantities. The size of the market and competitive pressure are also leading to ever more powerful processors in short innovation cycles. Even very cost-effective low-end processors today generally have two MIPI-CSI-2 interfaces with two or four lanes on-chip.

Even if a ribbon cable offers advantages, a connector is not defined in the MIPI specification.

© Basler

Many manufacturers of formerly pure mobile processors such as Qualcomm, Rockchip or Samsung have now discovered the image processing industry as an interesting market and are therefore now also offering their products - partly via module partners - in smaller quantities and with the long-term availability required in this sector for industrial applications. On the other hand, more typical industrial embedded processors such as the 'i.MX' family from NXP, some 'Sitarra' SoCs from Texas Instruments, products from Nvidia, the 'Atom' SoCs from Intel et cetera are now also equipped with MIPI-CSI-2 interfaces, making this new camera interface an obvious choice for the image processing market.

A major advantage of MIPI CSI-2 is that it enables extremely slim and cost-effective machine vision designs to be realized in which sensors or camera modules with a very high bandwidth can be used. However, it is not only the fact that processors are inexpensive and becoming ever cheaper that can reduce unit costs. Thanks to the large number of processors available, developers also have a great deal of freedom of choice depending on the application in order to optimize the design of a system. Among other things, this also makes it possible to minimize the power consumption of an embedded system.

For many products based on embedded systems, available installation space is an important constraint. MIPI CSI-2 enables very small designs that can be realized through the board-to-board connection via a ribbon cable. This is otherwise only possible with LVDS-based connections, but not with a connection via the USB 3.0 interface, which requires significantly more space due to its connectors.

Another advantage of the MIPI-CSI-2 interface is that the image data can be transferred directly from the camera module or sensor to the processor. This eliminates the need for intermediate processing or conversion of the data, which is necessary with USB 3.0 systems, for example. The data is therefore processed directly, which reduces the processor load. There is also no need for corresponding hardware such as microcontrollers. Due to these circumstances, very lean embedded systems can be realized with MIPI CSI-2.

Limitations of MIPI CSI-2

However, there are also some hurdles with MIPI CSI-2 that play a role in the development of embedded vision systems:

• Cable length: only short cable lengths of usually no more than 20 to 30 cm are possible. This is not a problem for mobile applications such as smartphones, but in many industrial applications this maximum length can be an exclusion criterion.
• Connector: The MIPI Alliance has not standardized a connector. Connecting a sensor or camera module must therefore always be done individually and often requires the hardware to be adapted, for example using an adapter board.
• Driver support: Neither driver nor software stacks are defined in the MIPI CSI-2 standard. The customization of a sensor or camera module to the CSI-2 driver of a given system-on-chip (SoC) must therefore be carried out by the developer. When selecting a sensor or camera module intended for an SoC, attention must therefore be paid to suitable drivers, which results in restrictions in the selection of sensors or SoCs.
• Pixel formats: Most CSI-2 drivers only support a few pixel formats.
• Camera API: Anyone familiar with image processing will have become accustomed to the standards of the GenICam family, which guarantees full access to all the features of a camera through the standardized camera API. With MIPI CSI-2, the situation is different: the lack of standardized feature designations and a standardized API makes it difficult to reuse existing code when migrating from one camera module to another or when switching to a different SoC.

Not without effort

All of these disadvantages can be overcome, albeit with possible effects on development and manufacturing costs. In principle, the MIPI-CSI-2 interface is suitable for embedded vision systems where a lean and cost-optimized design is desired. However, the integration of a corresponding camera module involves a certain amount of effort, particularly on the software side. Sometimes drivers and the image data interfaces have to be adapted to the applications. In addition, other restrictions, for example on the hardware side, such as the limited cable length, must be taken into account.

The system developer must take into account the scope and compatibility of the drivers as well as any existing camera software and its software interfaces. It is advantageous if suitable drivers exist that offer other features in addition to the pure image data flow, for example for camera control and suitable software interfaces. If these points are taken into account, efficient embedded vision systems can be built on the basis of MIPI CSI-2, enabling high-performance end products to be manufactured.

Author:
Frank Karstens is a Field Application Engineer at Basler in Ahrensburg.