financetom
Business
financetom
/
Business
/
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
News World Market Environment Technology Personal Finance Politics Retail Business Economy Cryptocurrency Forex Stocks Market Commodities
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Jan 29, 2025 7:14 AM

(Reuters) - Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.

The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday.

That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.

Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.

The Chinese startup did not immediately respond to a request for comment.

NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.

Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.

NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.

On prompts related to the Azerbaijan Airlines crash - questions unrelated to China - DeepSeek responded with Beijing's position on the topic, NewsGuard said.

"The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models," D.A. Davidson analyst Gil Luria said.

Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.

Comments
Welcome to financetom comments! Please keep conversations courteous and on-topic. To fosterproductive and respectful conversations, you may see comments from our Community Managers.
Sign up to post
Sort by
Show More Comments
Related Articles >
New Zealand to press ahead with media content pay law
New Zealand to press ahead with media content pay law
Jul 1, 2024
SYDNEY, July 2 (Reuters) - New Zealand's conservative coalition government will proceed with a bill that would make it compulsory for digital technology platforms to pay media companies for news, it said on Tuesday. The bill is being introduced as New Zealand media companies struggle against technology firms for advertising dollars, leading them to find new ways to provide news...
Samsung Elec union in South Korea to strike between July 8 and 10, union official says
Samsung Elec union in South Korea to strike between July 8 and 10, union official says
Jul 1, 2024
SEOUL (Reuters) - A workers' union at Samsung Electronics will strike between July 8 and 10, an union official said on Tuesday, as it steps up industrial action against the country's most powerful conglomerate. The union is figuring out how many unionised workers will join the strike, the official told Reuters by telephone. Son Woo-mok, leader of the union, said...
Australia tells internet firms to say how they will stop children from seeing porn
Australia tells internet firms to say how they will stop children from seeing porn
Jul 1, 2024
SYDNEY, July 2 (Reuters) - Australia is giving the internet industry six months to come up with an enforceable code detailing how it will stop children seeing pornography and other inappropriate material online or face having a code imposed on it, a regulator said on Tuesday. The eSafety Commissioner said it wrote to members of the online industry demanding a...
Northern Data considers AI unit US IPO at up to $16 billion, Bloomberg News reports
Northern Data considers AI unit US IPO at up to $16 billion, Bloomberg News reports
Jul 1, 2024
July 1 (Reuters) - Germany-based Northern Data AG ( NDTAF ) is exploring a U.S. initial public offering for its artificial intelligence (AI) cloud computing and data center units at a valuation of as much as $16 billion, Bloomberg News reported on Monday, citing people familiar with the matter. The company, which provides infrastructure for high-performance computing, plans to enlist...
Copyright 2023-2026 - www.financetom.com All Rights Reserved