Trend Micro has published a study on the risks associated with the company-wide use of large language models (LLMs). In the study "Risks of Unmanaged AI Reliance: Evaluating Regional Biases, Geofencing, Data Sovereignty, and Censorship in LLM Models", the researchers analyzed more than 100 AI models with over 800 specially developed prompts. The aim was to examine regional biases, geofencing, indications of data sovereignty, political and cultural contextual understandings and content restrictions.

According to the company, thousands of repetitions were carried out to measure differences over time and across locations. In total, more than 60 million input tokens and over 500 million output tokens were included in the analysis.

Inconsistent or outdated results

The results show that identical inputs can generate different responses depending on the region, language and model, and that outputs change even with repeated interactions with the same system. In politically sensitive scenarios, such as the description of disputed territories or national identities, the researchers found clear regional biases. In addition, the tested systems delivered inconsistent or outdated results in individual cases for tasks that require high accuracy, including financial calculations or time-critical information.

Robert McArdle, Director of Cybersecurity Research at Trend Micro © Trend Micro

"Many organizations assume that AI works like traditional software, where the same input reliably produces the same output," explains Robert McArdle, Director of Cybersecurity Research at Trend Micro. "Our research shows that this assumption is not true. LLMs change their responses depending on region, language and protection mechanisms, and these responses can differ from one interaction to the next. If AI spending is directly integrated into customer journeys or business decisions, companies risk losing control over brand communication, compliance positioning and cultural connectivity."

Risks increase, especially for globally active organizations

According to the study, these risks are particularly heightened for globally active organizations that use AI systems across borders and have to take different regulatory requirements and social expectations into account. The authors also see particular challenges for the public sector, as AI-generated content could be perceived there as official guidance and non-localized models raise questions of sovereignty and accessibility.

The full study "Risks of Unmanaged AI Reliance: Evaluating Regional Biases, Geofencing, Data Sovereignty, and Censorship in LLM Models" can be found here.