financetom
Technology
financetom
/
Technology
/
What the History of Linux Says About the Long Road to Decentralized Storage Adoption
News World Market Environment Technology Personal Finance Politics Retail Business Economy Cryptocurrency Forex Stocks Market Commodities
What the History of Linux Says About the Long Road to Decentralized Storage Adoption
Apr 16, 2024 11:37 AM

The evolution of computing has been marked by a series of paradigm shifts, from mainframes to personal computers, and now to the cloud. Currently, the common path to deploying web infrastructure is to be a paying customer of a large company like Amazon or Google ( GOOG ), and write infrastructure as code to spin up a constellation of interoperable services on their machines.

Claudia Richoux is the founder and CEO of Banyan, a decentralized file storage system built on top of the Filecoin network.

However, the current dominance of cloud hyperscalers, with their lion’s share of the world's computation, is not a natural monopoly.

Instead, it's a temporary state, as the open-source community works tirelessly to cook up a decentralized solution that will revolutionize the industry once again. A decentralized solution will allow users to freely move between service providers instead of being locked into one entity, which means more price competition and freedom for businesses to negotiate on terms with their cloud providers instead of the current state where they’re deeply entangled with and dependent on them.

The lessons of Linux

To understand the potential of a decentralized cloud, we need only look at the history of the Linux operating system. It took a decade for the Bells Labs developed software Unix to evolve into Minix, and another decade for Minix to become Linux. Finally, after yet another decade, Linux saw widespread enterprise adoption, becoming the obvious choice for developers.

Today, Linux open source software dominates the market, running on 80% of public servers and 100% of supercomputers. It's what all devs inevitably choose when they spin up an EC2 (Elastic Compute Cloud), a service by Amazon that allows users to run applications in AWS’s public cloud.

Linux's success was built on the power of compounding, shared, open development gains. Operating systems are monumental and intricate, requiring millions of hours of specialized labor, research and testing to create competitive ones from scratch. However, after years of government-funded and hobbyist-fueled incubation, Linux reached a usable state, and the benefits began to multiply.

As companies started using and depending on open-source operating systems, these massive operating system kernel codebases, or the the core of a computer's operating system, became “the commons,” and people pitch in to improve it. Everyone who uses Linux kernel searches for bugs. Some even contribute patches.

Open-source codebases are hard to kill

In other words, open source software support can be cheaper than closed-source software support plans, just because the cost to create the product itself is so much smaller. The cost of Linux to a business is denominated in fractional developers lending a hand, or in support plans (for companies without in-house kernel expertise).

See also: Is the Computing Sector Surfing the AI Surge?

In contrast, the cost of building and maintaining a custom operating system is astronomical, requiring the controlling entity to cover the entire lifetime cost. This necessitates a high cost of consumption to make the endeavor economically viable. If multiple closed operating systems compete, each must fund its own development and find sufficient market share to recoup their substantial investments. This explains the state of the market today to a substantial extent.

The cloud is another operating system

The "cloud" as we use it today on hyperscalers is simply another operating system, a tool for abstracting away when and where jobs run, their resources and how they interact. Whether it's a managed database, an open-source Kubernetes cluster or running a program on your computer, both clouds and operating systems are highly sophisticated and complex resource managers, schedulers and security providers.

The cloud operates at a much larger scale, making AWS, GCP, Azure and DigitalOcean ( DOCN ) the mainframe computers of our time, independently maintained within each company.

So, where is our open-source contender? OpenStack was developed by Rackspace ( RXT ) and NASA in the 2010s as an open-source cloud stack intended to be deployed to any datacenter to turn it into your own AWS. It has been said to be dead (for reasons I’ll discuss later), but is still under active development by companies (mostly Chinese) that use it instead of AWS due to government infrastructure restrictions.

One thing we notice first is that open-source codebases are hard to kill because their knowledge and power are encoded in git repositories, not in fallible human processes. OpenStack will never actually be dead, even if the original business models based on it are long gone. This means that open-source contenders have a lot of chances to win, because they can just keep resurrecting.

Righting the wrongs from our last try

OpenStack "failed" because every single company wanted to have their own distribution of the software to control the development roadmap. This led to fragmentation and competition rather than collaboration. Most major vendors released their own OpenStack distributions, fracturing the ecosystem, and then fought for market supremacy (which prevented development and progress).

Lack of standardization across OpenStack distributions made it difficult for users to deploy and manage clouds consistently. Additionally, the complexity of OpenStack made it challenging to install and operate, especially compared to more polished public cloud offerings from Amazon, Microsoft ( MSFT ) and Google ( GOOG ).

See also: Internet 2030: The Future and How We Get There

Without a strong central governing body, it was hard to drive a cohesive vision and rally the community around shared priorities. Many companies that initially supported OpenStack later reduced their investments or pulled out entirely.

How do we avoid failing again? This time, we have a secret weapon: nodes, consensus, block rewards, public goods and ecosystem alignment research. Node distribution is a mostly solved problem after 15 years of running blockchains, so the lack of standardization is something we can defeat.

By figuring out the right public goods mechanisms and how to align users, we can unite people around protocols instead of encouraging endless forking, competition and defection. Governance and shared roadmaps are a problem we’ve been continuously hacking on, and we're getting there because the shared state and shared value of blockchains bind communities together and force them to collaborate (in ways that easily-forkable open-source repositories simply don’t).

We have the financial primitives and the collaboration tooling nearly ready to work. We already have promising starts, but we must keep working.

The road to the winning end-state

We’re closer than we realize to a decentralized cloud go-to-market, which could bring hundreds of billions of dollars to the ecosystem if we really capture the market the way Linux did. Cloud markets are also growing at 11% annually — and what can AI adoption do but accelerate that? It makes bitcoin ETFs look like lunch money.

There are some gaps holding us back from winning right now, though, and the place we’re the most sorely lacking is sensible product decisions. The current state of Web3 attempts at decentralized cloud services are questionable, with limited product-market fit. Most people use these protocols only to game their token incentives, which is unfortunate.

On the bright side, speculation can drive product improvement and growth cheaply, as long as we listen to user feedback and use reward mechanisms with care.

Look closely at projects like Banyan, Estuary and ArDrive, and ask yourself if you'd choose them over Dropbox ( DBX ), especially if your job depended on it

We may not need to reuse OpenStack directly, but we need to build something similarly usable (and similarly reliable, which is the difficult part). We will have to blend our home-grown incentive structures with Web2 and cloud hyperscaler monitoring, compliance and security technologies in order to succeed. We need to design hybrid centralized/decentralized SLAs (service level agreements — the contracts between you and a cloud storage provider). We need to secure it with a buffet of reliability incentives, security attestations, zero-knowledge proofs, fully homomorphic encryption, computation fraud proofs, governance protocols and more.

See also: What Is Decentralized File Storage and How Does it Work?

Most of us in this segment have prototypes and MVPs, shallowly cargo-culting Web2 cloud products’ successes, instead of thoroughly attempting to rival the incumbents by working with users and adapting to their needs. However, technically or theoretically doing storage or computing or CDN-speed delivery is not enough, which is where most of us are. This will not attract paying users, token inflation will run out, and you will die.

The thing that attracts paying users will be doing the product iteration cycle, evolving our offerings to meet their needs and going to market like a Web2 cloud company.

Go watch a YCombinator video or two. Choose to do the hard thing. Fortunately, the Web3 community is almost infinitely patient to test and give feedback, because we all want (need?) this to work. We're competing with a mature and intimidatingly well-funded industry, and it's going to be uphill for a few more years.

Look closely at projects like Banyan, Estuary and ArDrive, and ask yourself if you'd choose them over Dropbox ( DBX ), especially if your job depended on it. Now think about how much work needs to be done to get there. Still, decentralized cloud infrastructure isn’t a question of if, but when.

Comments
Welcome to financetom comments! Please keep conversations courteous and on-topic. To fosterproductive and respectful conversations, you may see comments from our Community Managers.
Sign up to post
Sort by
Show More Comments
Related Articles >
Wheat price gathers the positive momentum – Forecast today - 05-04-2024
Wheat price gathers the positive momentum – Forecast today - 05-04-2024
Apr 4, 2024
Wheat Price Analysis Expected Scenario Wheat price tested the 569.80$ level and encountered solid resistance. Stochastic indicates a positive overlap, supporting the possibility of surpassing this level and initiating further bullish correction. The next positive target is awaited at 583.50$. The bullish trend scenario remains valid and active, conditioned by price stability above 552.80$. Expected Trading Range Between 555.00$ support...
Sugar price is stable – Forecast today - 05-04-2024
Sugar price is stable – Forecast today - 05-04-2024
Apr 4, 2024
Sugar Price Analysis Expected Scenario Sugar price didnt show any strong move yesterday, maintaining stability below the 22.20$ level. As a result, the bearish outlook remains unchanged, targeting 21.35$ as the next main station. Breaking 22.00$ will facilitate reaching the awaited target. The continuation of the bearish wave depends on price stability below 22.20$ and, most importantly, below 22.60$. Expected...
Factbox-Who are the candidates running in the 2024 US presidential election?
Factbox-Who are the candidates running in the 2024 US presidential election?
Apr 4, 2024
(Reuters) -President Joe Biden and former President Donald Trump will face each other in the U.S. presidential election on Nov. 5 in what looks set to be a divisive, closely fought contest. Several third-party hopefuls are also running. Here is a list of the candidates: DONALD TRUMP Trump, in office from 2017-2021, has secured enough delegates to clinch the Republican...
Brent oil price continues to achieve the positive targets – Forecast today - 05-04-2024
Brent oil price continues to achieve the positive targets – Forecast today - 05-04-2024
Apr 4, 2024
Brent Oil Price Analysis Expected Scenario Brent oil price rallied strongly upwards yesterday, achieving the awaited extended target at 91.00$. The price surpassed this level and settled above it, supporting the continuation of the bullish wave on the short-term and medium-term basis. Additional gains are expected, with the next target reaching 92.75$. To reinforce positive expectations, the price needs to...
Copyright 2023-2025 - www.financetom.com All Rights Reserved