financetom
Business
financetom
/
Business
/
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
News World Market Environment Technology Personal Finance Politics Retail Business Economy Cryptocurrency Forex Stocks Market Commodities
DeepSeek's chatbot achieves 17% accuracy, trails Western rivals in NewsGuard audit
Jan 29, 2025 6:56 AM

Jan 29 (Reuters) - Chinese AI startup DeepSeek's chatbot

achieved only 17% accuracy in delivering news and information in

a NewsGuard audit that ranked it tenth out of eleven in a

comparison with its Western competitors including OpenAI's

ChatGPT and Google Gemini.

The chatbot repeated false claims 30% of the time and gave

vague or not useful answers 53% of the time in response to

news-related prompts, resulting in an 83% fail rate, according

to a report published by trustworthiness rating service

NewsGuard on Wednesday.

That was worse than an average fail rate of 62% for its

Western rivals and raises doubts about AI technology that

DeepSeek has claimed performs on par or better than

Microsoft ( MSFT )-backed OpenAI at a fraction of the cost.

Within days of its roll-out, DeepSeek's chatbot became the

most downloaded app in Apple's ( AAPL ) App Store, stirring

concerns about United States' lead in AI and sparking a market

rout that wiped around $1 trillion off U.S. technology stocks.

The Chinese startup did not immediately respond to a request

for comment.

NewsGuard said it applied the same 300 prompts to DeepSeek

that it had used to evaluate its Western counterparts, which

included 30 prompts based on 10 false claims spreading online.

Topics for the claims included last month's killing of

UnitedHealthcare executive Brian Thompson and the downing of

Azerbaijan Airlines flight 8243.

NewsGuard's audit also showed that in three of the ten

prompts, DeepSeek reiterated the Chinese government's position

on the topic without being asked anything relating to China.

On prompts related to the Azerbaijan Airlines crash -

questions unrelated to China - DeepSeek responded with Beijing's

position on the topic, NewsGuard said.

"The importance of the DeepSeek breakthrough is not in

answering Chinese news-related question accurately, it is in the

fact that it can answer any question at 1/30th of the cost of

comparable AI models," D.A. Davidson analyst Gil Luria said.

Like other AI models, DeepSeek was most vulnerable to

repeating false claims when responding to prompts used by people

seeking to use AI models to create and spread false claims,

NewsGuard added.

Comments
Welcome to financetom comments! Please keep conversations courteous and on-topic. To fosterproductive and respectful conversations, you may see comments from our Community Managers.
Sign up to post
Sort by
Show More Comments
Related Articles >
Zillow Group Insider Sold Shares Worth $4,343,507, According to a Recent SEC Filing
Zillow Group Insider Sold Shares Worth $4,343,507, According to a Recent SEC Filing
Nov 15, 2024
05:28 PM EST, 11/15/2024 (MT Newswires) -- David A. Beitel, Chief Technology Officer, on November 13, 2024, sold 59,929 shares in Zillow Group ( ZG ) (Z, ZG) for $4,343,507. Following the Form 4 filing with the SEC, Beitel has control over a total of 165,015 shares of the company, with 165,015 shares held directly. SEC Filing: https://www.sec.gov/Archives/edgar/data/1617640/000161764024000131/xslF345X05/wk-form4_1731709314.xml ...
Live Oak Bancshares Insider Sold Shares Worth $616,837, According to a Recent SEC Filing
Live Oak Bancshares Insider Sold Shares Worth $616,837, According to a Recent SEC Filing
Nov 15, 2024
05:25 PM EST, 11/15/2024 (MT Newswires) -- John W. Sutherland, Chief Accounting Officer, on November 13, 2024, sold 12,588 shares in Live Oak Bancshares ( LOB ) for $616,837. Following the Form 4 filing with the SEC, Sutherland has control over a total of 14,015 shares of the company, with 14,015 shares held directly. SEC Filing: https://www.sec.gov/Archives/edgar/data/1462120/000157856324000447/xslF345X05/wk-form4_1731709059.xml ...
Zillow Group Insider Sold Shares Worth $347,778, According to a Recent SEC Filing
Zillow Group Insider Sold Shares Worth $347,778, According to a Recent SEC Filing
Nov 15, 2024
05:27 PM EST, 11/15/2024 (MT Newswires) -- Jeremy Hofmann, Chief Financial Officer, on November 13, 2024, sold 4,702 shares in Zillow Group ( ZG ) (Z, ZG) for $347,778. Following the Form 4 filing with the SEC, Hofmann has control over a total of 106,537 shares of the company, with 106,537 shares held directly. SEC Filing: https://www.sec.gov/Archives/edgar/data/1617640/000161764024000132/xslF345X05/wk-form4_1731709392.xml ...
Zillow Group Insider Sold Shares Worth $836,680, According to a Recent SEC Filing
Zillow Group Insider Sold Shares Worth $836,680, According to a Recent SEC Filing
Nov 15, 2024
05:26 PM EST, 11/15/2024 (MT Newswires) -- Jennifer Rock, Chief Accounting Officer, on November 13, 2024, sold 11,412 shares in Zillow Group ( ZG ) (Z, ZG) for $836,680. Following the Form 4 filing with the SEC, Rock has control over a total of 61,424 shares of the company, with 61,424 shares held directly. SEC Filing: https://www.sec.gov/Archives/edgar/data/1617640/000161764024000133/xslF345X05/wk-form4_1731709453.xml ...
Copyright 2023-2026 - www.financetom.com All Rights Reserved