AI chip developed for decentralized use without the cloud

The first prototypes of the chip designed by Prof. Hussam Amrouch - Professor of AI Processor Design at TUM - have already been produced by semiconductor manufacturer Global Foundries in Dresden. Unlike conventional chips, the computing and memory units of 'AI Pro' are located together. This is possible because the chip works according to the principle of 'hyperdimensional computing': This means that it recognizes similarities and patterns, but does not require millions of data sets to learn. Instead of being shown countless images of cars, as is the case with deep learning (which is used in conventional AI chips), the new chip combines various pieces of information - such as the fact that a car has four wheels, usually drives on the road and can have different shapes. "Humans also abstract and learn through similarities," explains Amrouch, just like the new chip.

An important advantage of brain-like thinking: it saves energy. For a defined training of a task, a 'sample', the new chip consumed 24 microjoules, while comparable chips required ten to a hundred times more energy - "a record value", comments Amrouch. "This mix of modern processor architecture, algorithm specialization and innovative data processing makes the AI chip something special."

Prof. Hussam Amrouch in his Garching laboratory at the Siemens Technology Center. © Andreas Heddergott / TU Munich

This also sets it apart from all-rounders such as the chips from industry giant Nvidia. "While Nvidia has built a platform that relies on cloud data and promises to solve every problem, we have developed an AI chip that enables customer-specific solutions. There is a huge market here."

Built on the model of the human brain

The one square millimeter chip, which currently still costs 30,000 euros, has around 10 million transistors and is therefore not quite as densely packed or as powerful as Nvidia chips with 200 billion transistors. But that is not Amrouch's primary concern. His team specializes in AI chips that process data directly on site. This means they do not have to be sent to the cloud, processed there together with millions of other data sets and then sent back again. This saves time, server computing capacity and reduces the carbon footprint of AI.

The chips are also tailor-made for specific applications. "That makes them very efficient," chip expert Amrouch is convinced. For example, they focus on processing a person's vital data via a smartwatch or on supporting the navigation of a drone. The fact that this personal and sometimes sensitive data remains on board the device means that the question of a stable internet connection and cyber security does not even arise.