Jan 29 (Reuters) - Chinese AI startup DeepSeek's chatbot
achieved only 17% accuracy in delivering news and information in
a NewsGuard audit that ranked it tenth out of eleven in a
comparison with its Western competitors including OpenAI's
ChatGPT and Google Gemini.
The chatbot repeated false claims 30% of the time and gave
vague or not useful answers 53% of the time in response to
news-related prompts, resulting in an 83% fail rate, according
to a report published by trustworthiness rating service
NewsGuard on Wednesday.
That was worse than an average fail rate of 62% for its
Western rivals and raises doubts about AI technology that
DeepSeek has claimed performs on par or better than
Microsoft ( MSFT )-backed OpenAI at a fraction of the cost.
Within days of its roll-out, DeepSeek's chatbot became the
most downloaded app in Apple's ( AAPL ) App Store, stirring
concerns about United States' lead in AI and sparking a market
rout that wiped around $1 trillion off U.S. technology stocks.
The Chinese startup did not immediately respond to a request
for comment.
NewsGuard said it applied the same 300 prompts to DeepSeek
that it had used to evaluate its Western counterparts, which
included 30 prompts based on 10 false claims spreading online.
Topics for the claims included last month's killing of
UnitedHealthcare executive Brian Thompson and the downing of
Azerbaijan Airlines flight 8243.
NewsGuard's audit also showed that in three of the ten
prompts, DeepSeek reiterated the Chinese government's position
on the topic without being asked anything relating to China.
On prompts related to the Azerbaijan Airlines crash -
questions unrelated to China - DeepSeek responded with Beijing's
position on the topic, NewsGuard said.
"The importance of the DeepSeek breakthrough is not in
answering Chinese news-related question accurately, it is in the
fact that it can answer any question at 1/30th of the cost of
comparable AI models," D.A. Davidson analyst Gil Luria said.
Like other AI models, DeepSeek was most vulnerable to
repeating false claims when responding to prompts used by people
seeking to use AI models to create and spread false claims,
NewsGuard added.