Smart chips for greater energy efficiency

Deep neural networks (DNNs) are an elementary component of AI and are trained in a complex process with very large amounts of data. This is why they are responsible for an increasing proportion of the computing load and therefore for energy consumption and CO2 emissions in data centers. Prof. Dr. Marco Platzner from the Institute of Computer Science at Paderborn University led the "eki" project and explains: "Deep neural networks are a type of AI that works on the principle of the human brain. The 'deep' part refers to the fact that the networks have many layers that process data and recognize patterns, analyze images and process language." After the DNNs have been trained with huge amounts of data, the models resulting from the process are used. As a rule, GPUs or CPUs are used for this, but their energy efficiency is low. The project team has therefore developed a solution: With the help of freely programmable chips, so-called field-programmable gate arrays (FPGAs), the energy efficiency of AI systems can be optimized for DNN computation.

By comparison, conventional processors execute fixed instruction sets, while the circuitry in FGPAs can be customized. This creates a kind of customized hardware. The advantage: depending on the application, the chips consume less energy and calculate faster than graphics processors. The disadvantage: they are more complex to program. But even this hurdle has been overcome. Researchers from the Department of Computer Engineering have been working on energy-efficient computing using FPGAs together with the Paderborn Center for Parallel Computing (PC2) at Paderborn University for a long time. "The company AMD/Xilinx had already developed the open source program FINN for neural networks on FPGAs. In close collaboration, we were able to contribute our experience to make FINN even better and focus on energy efficiency," explains Prof. Platzner.

To reduce energy requirements, the scientists have simplified the AI models by removing unnecessary connections within the AI and ensuring that complex functions run efficiently. DNNs were also distributed across several FGPAs. Another focus was on developing reliable methods for predicting the energy requirements of individual components. The researchers succeeded in doing this by extending FINN. They were also able to measure the consumption of complete inference runs and compare them with other technologies. An inference run is the moment when an AI model applies its knowledge to react to new data. "It is particularly pleasing that we were able to achieve an increase in energy efficiency of up to ten times compared to the use of graphics processors. This not only reduces power consumption, but also - depending on the electricity mix - CO2 emissions. As the use of AI is constantly growing, the energy requirements of DNNs will become an important environmental factor in the future," concludes Prof. Platzner.

The code that the scientists have developed is openly available in FINN. In addition, the PC2 at Paderborn University offers workshops to introduce interested parties to the use of the methods for DNN mapping on FGPA systems and for energy analysis.

In addition to Paderborn University, Hamm-Lippstadt University of Applied Sciences, South Westphalia University of Applied Sciences, the HPC company Megware (Chemnitz) and AMD Research Labs in Ireland were also involved in the project. The Federal Ministry for the Environment, Climate Protection, Nature Conservation and Nuclear Safety funded the project with around 1.5 million euros over a period of three years.