TU Darmstadt
Study shows limitations of LLMs
A new study led by TU Darmstadt has revealed the limitations of AI models such as ChatGPT. The researchers came to the conclusion that it is a fallacy that LLMs can perform complex tasks correctly without human support.
A new study led by TU Darmstadt has revealed the limitations of AI models such as ChatGPT. The study, which will be presented at the annual conference of the Association for Computational Linguistics (ACL) in Bangkok in August, concludes that these models are less capable of learning independently than previously assumed. There is no evidence that large language models (LLMs) develop a general "intelligent" behavior that enables complex thinking or planned action.
The study focuses on so-called "emergent capabilities" - unexpected leaps in the performance of language models that were observed as they scaled up. Although these models can handle more and more language-based tasks, such as recognizing fake news or drawing logical conclusions, due to larger amounts of data and more complex structures, there is no evidence that they develop sophisticated thinking abilities, according to the researchers.
The scientists, including TU Professor Iryna Gurevych and Dr. Harish Tayyar Madabushi from the University of Bath, found that the models only acquired the ability to perform relatively simple "However, our results do not mean that AI poses no threat at all," Gurevych emphasized. "Rather, we show that the alleged emergence of complex reasoning abilities associated with certain threats is not supported by evidence and that we can control the learning process of LLMs well after all. Therefore, future research should focus on other risks posed by the models, such as their potential to be used to generate fake news."
For users of AI systems such as ChatGPT, this means that these models should not be relied upon to perform complex tasks correctly without human assistance. It is recommended to give clear instructions and provide examples. The tendency of models to produce plausible-sounding but incorrect results - known as confabulation - persists, according to the study, even though the quality of the models has improved considerably in recent times.
You might also be interested in
ChatGPT and code analysis
LLMs open up new possibilities for analyzing and improving code. This article is the first of a three-part series on experiments that show how already available general language models influence safety-relevant processes and activities.
read more...Series of articles on Large Language Models
ChatGPT in the industry
ChatGPT was the hype topic last year. This technology is expected to enter the industry in 2024. A new series of articles explores the opportunities and challenges of ChatGPT for the industry.
read more...Combining robotics and ChatGPT
Prof. Schöllig (TU Munich) uses ChatGPT to develop choreographies for swarms of drones to match the music. A safety filter prevents the flying robots from colliding. LLMs such as ChatGPT can therefore be used in robotics in principle.
read more...AI, ChatGPT and co. are changing our brains
Digital work is a real challenge for the brain. The use of AI tools such as ChatGPT can also change processes in the brain. What does this do to our control center?
read more...LLM as the basis for voice control
Sabo Mobile IT relies on 'Large Language Models' - LLM for short - for its Sabot voice control system. Thomas Sykora talks about the technology behind it and data protection in an industrial environment.
read more...Paderborn University becomes part of 'ELLIS NRW'
Paderborn University is one of the founding members of the new research unit 'ELLIS Unit NRW'. The European network bundles AI research in North Rhine-Westphalia with a focus on machine learning and explainable AI.
read more...AI 'CheckMate' optimizes industrial processes
With 'CheckMate', the University of Klagenfurt has developed an AI system that independently creates algorithms for complex combinatorics and optimization problems. The technology is designed to make industrial planning and logistics processes more...
read more...Real-world laboratory for AI compliance
New project phase launched with three focal points
The new real-world laboratory for legally compliant AI and robotics has started its work at the AI Progress Center in Stuttgart. It primarily supports small and medium-sized companies with technology development, regulatory issues and the...
read more...Self-learning sensor systems monitor industrial manufacturing processes
In the 'GenSATIOn-Edge' project, the Fraunhofer Institute for Microelectronic Circuits and Systems IMS is working with partners to develop intelligent sensor systems for the continuous monitoring of industrial processes. AI models run directly on...
read more...