DIGITAL LIFE

Study shows more misinformation in Russian, Chinese and Spanish chatbots

The world's leading chatbots generate more false information in Russian, Chinese and Spanish than in English, a study developed by NewsGuard has revealed.

The study "The multilingual failure of AI (artificial intelligence)" monitored the 10 main language models in AI, such as ChatGPT, Copilot or Meta AI.

"A total of 30 warnings based on 10 false claims spread online were tested on the 10 chatbots (virtual assistants that use artificial intelligence and programming to communicate via text with users). These 30 questions were then translated and tested in seven languages, which resulted in a total of 2,100 responses from the chatbot", says the study.

In the study, conducted in January, of the 2,100 responses in all languages ​​analyzed, 555 (26.43%) contained false information, while 402 (19.14%) provided a non-response and 1,143 (54.43) contained a devaluation.

The analysis was conducted in seven languages: Russian, Chinese, Spanish, English, German, Italian and French.

In Russian, a failure rate of 55% was found, and of the 300 responses, 105 (35%) contained false information, 59 (20%) provided a non-response, while 136 responses (45%) contained misinformation. In Chinese, the failure rate corresponds to 51.33%. Of the 300 responses, 100 (33.33%) contained false information, 71 (23.67%) provided a non-response and 129 (43%) contained misinformation.

The study indicates that, in the analysis of Spanish, a 48% failure rate was found, so that of the total responses, 81 (27%) contained false information, 63 provided a non-response (21%) and 156 responses contained misinformation (52%). In English, the failure rate is 43%.

In this language, 69 responses contained false information (23%), 60 (20%) provided a non-response and 171 (57%) contained misinformation. In German, the failure rate is 43.33%, with 65 responses containing false information (21.66%), the same percentage being considered a non-response and 170 (56.66%) containing misinformation.

In the case of Italian, the failure rate is 38.67%. Of the 300 responses, 75 (25%) contained false information, 41 (13.675) provided a non-response and 184 (61.33%) contained misinformation.

French has a failure rate of 34.33%. Of the 300 responses, 60 (20%) contained false information, 43 (14.33%) provided a non-response and 197 (65.67%) contained misinformation. In this sense, the results indicate that AI chatbots are systematically weaker in providing accurate information in languages ​​where fact-checking ecosystems are less robust.

In addition, the authors also warn that news ecosystems may contain a high volume of state-affiliated and unreliable sources, which may influence the responses generated by AI bots.

mundophone