The generative text AI 'FhGenie' offers employees the opportunity to securely edit, change or generate texts with non-public data. The GDPR-compliant service is being introduced in line with the recommendations of the "Chatbot" task force of the Fraunhofer ICT Group for the Fraunhofer-Gesellschaft.

The disruptive potential of generative AI systems has been clear since the publication of ChatGPT at the latest. The expected economic and social impact is enormous - which is why fast and secure access to existing systems is essential for large research organizations in order to gain further first-hand knowledge, drive technological developments and maintain innovative strength, according to the Fraunhofer-Gesellschaft. It has now activated the AI chatbot 'FhGenie' for its employees. It is based on Microsoft's 'Azure OpenAI Service', which enables organizations to use the latest technologies for generative AI from OpenAI such as GPT-3.5 or GPT-4, Codex, ChatGPT and DALL-E 2 from the Microsoft Azure Cloud.

"I am very pleased that we have succeeded in developing an AI chat system with Microsoft that is specially adapted to the requirements of the Fraunhofer-Gesellschaft in a very short space of time," says Prof. Ingo Weber, Director of Digitalization and ICT Infrastructure at the Fraunhofer-Gesellschaft. "We have noticed that many colleagues already want to use chat-based AI in their work and for research purposes. However, the current public offerings are problematic for business use, for reasons including data protection, confidentiality and information security."

With FhGenie, Fraunhofer employees from research to management to administration have the opportunity to gain experience: "The Azure OpenAI service from Microsoft, on which FhGenie is based, enables Fraunhofer employees to use the potential of generative AI systems for their own work. Given the rapid pace of technological development, it is important to familiarize yourself with the existing solutions and take advantage of the benefits," summarizes Alexander Britz, Head of the Public Sector division at Microsoft Germany.

GDPR-compliant internal chatbot

The generative text AI FhGenie is based on the OpenAI language model GPT-3.5-turbo and is to be expanded as quickly as possible to include the more powerful and multimodal GPT-4.0, which can also process images. FhGenie currently consists of a web application with a framework based on the Python programming language and the JavaScript runtime environment Node.js and can be easily accessed by employees via the browser after SSO authentication using a smartcard. To ensure data security, the instance was configured in a European, Fraunhofer-owned Azure subscription - a closed area within Microsoft's cloud platform.
Requests via the Azure OpenAI service, which is also implemented specifically in the Fraunhofer subscription, are made via API Prompt - a customized interface that forwards the commands to the AI system. Users receive their results via completion (auto-completion of answers). All data entered remains with the Fraunhofer-Gesellschaft and is not used to train the AI. Apart from an automated filter program that checks for misuse, the content entered is not checked or stored.

The GDPR-compliant Azure OpenAI service application can currently be used Fraunhofer-wide with the consent of the works council, with access authorizations being assigned decentrally by the IT managers of the individual Fraunhofer Institutes and facilities. Current usage figures show the openness of the Fraunhofer workforce to the new service: as of today, over 19,200 of the more than 30,000 Fraunhofer employees are already registered for the service, of which around 6,500 are active users.

There are numerous potential applications for the AI chatbot - the full range of which will probably only become apparent in the future. FhGenie already offers Fraunhofer employees support not only in creating and improving texts and presentations, for example for internal reports, but can also contribute to the faster development of software, prototypes or data analysis scripts. There is also potential in the further development of internal knowledge management, for example in the form of a "dialog-capable intranet". Researchers at the Fraunhofer Institutes can use the experimental space created to quickly expand existing expertise and test AI models together with customers.

The respective individual use is left to the users themselves; for legal reasons, only personal data may not be processed with FhGenie. "We are currently in a transitional and experimental phase in which it was important for us to quickly offer a usable solution in order to be at the forefront of rapid technological developments," explains Prof. Weber. "Of course, there are still many unanswered questions, such as copyright or liability for incorrect results. We have therefore asked those responsible to raise awareness in their respective institutes and departments that all users are responsible for the quality of the AI-generated results and must check them themselves."

AI language models and Fraunhofer

Numerous Fraunhofer Institutes are already active in initiatives and projects related to the research and application of generative AI and foundation models - for example, the OpenGPT-X consortium for the development of large European AI language models. This is another reason why, following the publication of ChatGPT, a task force of the Fraunhofer ICT Group developed recommendations for the Fraunhofer-Gesellschaft on how to deal with AI language models. The activation of FhGenie is part of the first stage of a three-step process comprising access to (Access), adaptation (Adapt) and further development (Advance) of large AI models, whereby further manufacturers are to be acquired as cooperation partners for the Fraunhofer-Gesellschaft.

"Large language models and generative AI enable new products and make our working world significantly more productive. As an application-oriented research institution with an interdisciplinary approach and strong links to industry, the Fraunhofer-Gesellschaft is in an excellent position to effectively put these innovations into practice. The relevant technology is already in daily use at companies - for example in the financial and healthcare sectors - and is achieving efficiency gains of up to 50 percent. With FhGenie, we are also ensuring that everyone at Fraunhofer can immediately and securely exploit this potential and develop it for our customers and partners," says Prof. Dr. Stefan Wrobel, Vice Chairman of the Fraunhofer ICT Group and Director of the Fraunhofer Institute for Intelligent Analysis and Information Systems IAIS.