financetom
Business
financetom
/
Business
/
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
News World Market Environment Technology Personal Finance Politics Retail Business Economy Cryptocurrency Forex Stocks Market Commodities
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Jan 29, 2025 7:14 AM

(Reuters) - Chinese AI startup DeepSeek's chatbot achieved only 17% accuracy in delivering news and information in a NewsGuard audit that ranked it tenth out of eleven in a comparison with its Western competitors including OpenAI's ChatGPT and Google Gemini.

The chatbot repeated false claims 30% of the time and gave vague or not useful answers 53% of the time in response to news-related prompts, resulting in an 83% fail rate, according to a report published by trustworthiness rating service NewsGuard on Wednesday.

That was worse than an average fail rate of 62% for its Western rivals and raises doubts about AI technology that DeepSeek has claimed performs on par or better than Microsoft-backed OpenAI at a fraction of the cost.

Within days of its roll-out, DeepSeek's chatbot became the most downloaded app in Apple's App Store, stirring concerns about United States' lead in AI and sparking a market rout that wiped around $1 trillion off U.S. technology stocks.

The Chinese startup did not immediately respond to a request for comment.

NewsGuard said it applied the same 300 prompts to DeepSeek that it had used to evaluate its Western counterparts, which included 30 prompts based on 10 false claims spreading online.

Topics for the claims included last month's killing of UnitedHealthcare executive Brian Thompson and the downing of Azerbaijan Airlines flight 8243.

NewsGuard's audit also showed that in three of the ten prompts, DeepSeek reiterated the Chinese government's position on the topic without being asked anything relating to China.

On prompts related to the Azerbaijan Airlines crash - questions unrelated to China - DeepSeek responded with Beijing's position on the topic, NewsGuard said.

"The importance of the DeepSeek breakthrough is not in answering Chinese news-related question accurately, it is in the fact that it can answer any question at 1/30th of the cost of comparable AI models," D.A. Davidson analyst Gil Luria said.

Like other AI models, DeepSeek was most vulnerable to repeating false claims when responding to prompts used by people seeking to use AI models to create and spread false claims, NewsGuard added.

Comments
Welcome to financetom comments! Please keep conversations courteous and on-topic. To fosterproductive and respectful conversations, you may see comments from our Community Managers.
Sign up to post
Sort by
Show More Comments
Related Articles >
ASML Receives Approval for Netherlands Headquarters Expansion
ASML Receives Approval for Netherlands Headquarters Expansion
Jul 5, 2024
07:47 AM EDT, 07/05/2024 (MT Newswires) -- ASML Holding ( ASML ) may expand its headquarters and manufacturing facility in the Dutch town of Veldhoven, the Netherlands Council of State ruled Wednesday, rejecting complaints by two residents opposed to the semiconductor company's expansion. The ruling allows ASML ( ASML ) to expand the facility where it produces chip-making machines and...
BMO on The Day Ahead in Canada
BMO on The Day Ahead in Canada
Jul 5, 2024
07:45 AM EDT, 07/05/2024 (MT Newswires) -- Canada will publish its Labour Force Survey (LFS) for June at 8:30 a.m. ET on Friday, pointed out Bank of Montreal (BMO). The bank noted that, amid the gyrations in financial markets and economic data, employment trends have been surprisingly uniform, with the three-, six- and 12-month averages not far from 35,000. BMO...
Europe's top cognac makers will attend China meeting on anti-dumping probe
Europe's top cognac makers will attend China meeting on anti-dumping probe
Jul 5, 2024
LONDON, July 5 (Reuters) - Europe's top cognac makers will attend a hearing on July 18 on China's anti-dumping probe of the industry for their first chance to defend themselves in person, an industry source said. Main brands Martell, which is owned by Pernod Ricard , Remy which is part of Remy Cointreau, and Hennessey, owned by LVMH, received a...
Walmart Must Face Lawsuit Over 'Bait-and-Switch' Pricing, Court Rules
Walmart Must Face Lawsuit Over 'Bait-and-Switch' Pricing, Court Rules
Jul 5, 2024
07:45 AM EDT, 07/05/2024 (MT Newswires) -- Walmart ( WMT ) must face a proposed class action lawsuit claiming the retailer charges higher prices at the checkout counter than it posts on shelves, a federal appeals court ruled Wednesday. The 7th Circuit Court of Appeals in Chicago reversed a lower court ruling and decided that a class action representing consumers...
Copyright 2023-2026 - www.financetom.com All Rights Reserved