June 25 (Reuters) - Social media platform Reddit ( RDDT )

said on Tuesday it will update a web standard used by

the platform to block automated data scraping from its website,

following reports that AI startups were bypassing the rule to

gather content for their systems.

The move comes at a time when artificial intelligence firms

have been accused of plagiarizing content from publishers to

create AI-generated summaries without giving credit or asking

for permission.

Reddit ( RDDT ) said that it would update the Robots Exclusion

Protocol, or "robots.txt," a widely accepted standard meant to

determine which parts of a site are allowed to be crawled.

The company also said it will maintain rate-limiting, a

technique used to control the number of requests from one

particular entity, and will block unknown bots and crawlers from

data scraping - collecting and saving raw information - on its

website.

More recently, robots.txt has become a key tool that

publishers employ to prevent tech companies from using their

content free-of-charge to train AI algorithms and create

summaries in response to some search queries.

Last week, a letter to publishers by the content licensing

startup TollBit said that several AI firms were circumventing

the web standard to scrape publisher sites.

This follows a Wired investigation which found that AI

search startup Perplexity likely bypassed efforts to block its

web crawler via robots.txt.

Earlier in June, business media publisher Forbes accused

Perplexity of plagiarizing its investigative stories for use in

generative AI systems without giving credit.

Reddit ( RDDT ) said on Tuesday that researchers and organizations

such as the Internet Archive will continue to have access to its

content for non-commercial use.