SAN FRANCISCO, July 19 (Reuters) - Security experts said
CrowdStrike's ( CRWD ) routine update of its widely used
cybersecurity software, which caused clients' computer systems
to crash globally on Friday, apparently did not undergo adequate
quality checks before it was deployed.
The latest version of its Falcon Sensor software was meant
make CrowdStrike ( CRWD ) clients' systems more secure against hacking by
updating the threats it defends against. But faulty code in the
update files resulted in one of the most widespread tech outages
in recent years for companies using Microsoft's ( MSFT )
Windows operating system.
Global banks, airlines, hospitals and government offices
were disrupted. CrowdStrike ( CRWD ) released information to fix affected
systems, but experts said getting them back online would take
time as it required manually weeding out the flawed code.
"What it looks like is, potentially, the vetting or the
sandboxing they do when they look at code, maybe somehow this
file was not included in that or slipped through," said Steve
Cobb, chief security officer at Security Scorecard, which also
had some systems impacted by the issue.
Problems came to light quickly after the update was rolled
out on Friday, and users posted pictures on social media of
computers with blue screens displaying error messages. These are
known in the industry as "blue screens of death."
Patrick Wardle, a security researcher who specialises in
studying threats against operating systems, said his analysis
identified the code responsible for the outage.
The update's problem was "in a file that contains either
configuration information or signatures," he said. Such
signatures are code that detects specific types of malicious
code or malware.
"It's very common that security products update their
signatures, like once a day... because they're continually
monitoring for new malware and because they want to make sure
that their customers are protected from the latest threats," he
said.
The frequency of updates "is probably the reason why
(CrowdStrike ( CRWD )) didn't test it as much," he said.
It's unclear how that faulty code got into the update and
why it wasn't detected before being released to customers.
"Ideally, this would have been rolled out to a limited pool
first," said John Hammond, principal security researcher at
Huntress Labs. "That is a safer approach to avoid a big mess
like this."
Other security companies have had similar episodes in the
past. McAfee's buggy antivirus update in 2010 stalled hundreds
of thousands of computers.
But the global impact of this outage reflects CrowdStrike's ( CRWD )
dominance. Over half of Fortune 500 companies and many
government bodies such as the top U.S. cybersecurity agency
itself, the Cybersecurity and Infrastructure Security Agency,
use the company's software.