Hacker News new | past | comments | ask | show | jobs | submit login
Centralization is inevitable, but decentralization is right behind it (subconscious.substack.com)
170 points by agreeahmed on Nov 1, 2022 | hide | past | favorite | 154 comments



The point of open protocols is not that there will actually be many providers, but that one could switch providers easily if they do something fishy. It's about the threat, and what can happen on margin.

Keeping a sword dangling over the monopolist's head is reasonable.


You are correct. The greatest threat to someone providing a service based on an open protocol is that users might pick up and leave.

Consequently, a service provider will plow enormous resources into differentiating their service, and generating other lock-in effects, so the users find it hard to leave.

A popular open source protocol is actually the perfect playing field for a would-be monopolist.

The monopolist can make their service better and better with cash and execution, but meanwhile the protocol is stuck at a baseline since a varied ecosystem doesn't upgrade itself as quickly. So it is less and less attractive.


> A popular open source protocol is actually the perfect playing field for a would-be monopolist.

Whereas when there's no popular open source protocol, the OSS ecosystem upgrades itself fast, like a cheetah? :P Whatever mental model you're using here might just be going over my head and I'd appreciate you being more explicit about it


Microsoft: Embrace, Extend, Extinguish.


Riiigght, that makes some sense, thank you! Need to consider to understand if open source is really advantageous, but you've pointed out how it might be


What's an example of an open protocol (what this thread is about) where MSFT successfully followed this strategy?


HTML before IE6. At one point Microsoft owned the web browser market to the same degree that Google owns it today.


Anything from this century?


IE6 was the most popular web browser until 2008. It wasn't until 2010 that we finally could somewhat ignore it when it dropped below 20% market share. It was a happy day when I could completely ignore it in corporate in 2018 when it was uninstalled from all computers.


> A popular open source protocol is actually the perfect playing field for a would-be monopolist.

At first I thought "email seems a good example of this: most people don't want to leave Google, I don't want to leave FastMail"... but then I remembered that I could just leave FastMail and take my email to Google. Which proves the upthread point about the open protocol being a safety net for users.

Can you talk about some examples of monopolised open protocols?


Email is the paradigmatic example. You can export your mailbox and use `pine` if that's your jam. But in the real world, most users of Gmail will depend on at least one feature of Gmail that isn't easily replicable elsewhere.


> Email is the paradigmatic example

But there's no monopoly. There's feature-based lock-in built around the open protocols, and it's very effective, but it's not a monopoly.


I wonder what gmail's churn rate even is? It is surely staggeringly close to zero.


I wouldn’t be surprised if they have more churn via death (people, companies, abandoned projects) than any other single source.


Not if the switching cost is high and/or slow. The problems of a monarchy will not be solved by setting up a competing neighboring kingdom.

To bring the topic back to networking, it has been said that decentralization is the worst form of networking except all those other forms that have been tried from time to time.

(With apologies to Winston Churchill.)


> Not if the switching cost is high and/or slow. The problems of a monarchy will not be solved by setting up a competing neighboring kingdom.

That was how the US grew between 1700 and 1914. Everyone in Europe sick of monarchies just moved out to the new world.

When there isn't a finite amount of land to squat it becomes a viable long term solution.


The U.S. isn’t (and wasn’t) a kingdom.


> Not if the switching cost is high and/or slow.

This seems to be one of the key goals of Jack Dorsey's bluesky over Mastadon. Mastadon apparently has a lot of friction when trying to switch servers, even though it's a federated protocol. Bluesky claims to want make it much more trivial to take your identity to any provider on the federated network.


I bring up open protocols because I think they is necessary, if not sufficient, to keep the switching costs low.


I agree. A few years ago, I wrote an article on this subject :

hello@joe — greeting, editorial rule and name on the Internet

http://apichat.fr/doku.php?id=hello_joe_greeting_editorial_r...


Plus, if these protocols are built on top of cryptographic methods, it might not be so easy for the providers to do something fishy in the first place.


Since somebody chose to downvote the above comment, I think it bears elaboration for those who might not understand. A contemporary example is domain names. The current domain system is a monopoly, and it sucks. You pay companies a never-ending and ever-increasing rent to do literally less than nothing. A domain isn't much more than a key/value pair in a small database. You're paying never-ending rent to them for them to not go out of their way to delete your entry.

Relying on a centralized competitor to offer something better would be dubious. If they gain substantial marketshare there's nothing stopping them from deciding to simply swap back to our current system. It could even be done retroactively since everybody has those lines in their terms and conditions that amount to 'We can do anything, and you can't do anything about it.'

But with a decentralized system you have the possibility of a new type of domain where while one operator may be able to be the only one that can create them, once they transfer that domain to you it becomes literally impossible for them to access, change, or affect it in any way, shape, or form. Decentralization is able to turn various digital goods into something much closer to their physical counterpart where you are buying something that is no longer connected to the seller after checkout. And this can really help ensure the integrity of services that otherwise might be susceptible to bait and switch business models.


I'm not defending registrars/registries per se, but...

> A domain isn't much more than a key/value pair in a small database.

No, it's often a huge investment and important part of security approaches (certificates, validation, email, etc.)

> You're paying never-ending rent to them for them to not go out of their way to delete your entry.

Yes, you're paying them to not update that value to point to someone else's server. Just like you pay your landlord to not only keep your apartment in working order, but to not let someone else live there.

> becomes literally impossible for them to access, change, or affect it in any way, shape, or form

How is this a good thing? Someone takes over your domain and nobody is able to get it back? You just spent millions of dollars on marketing to get people to visit your somenameforme.tld and now it points to another site because your key leaked. ¯\_(ツ)_/¯?


Keys don't "leak" but of course they can be stolen - like any real product. If you spend $40 million on your Van Gogh, and it's stolen - that's not going to be a happy moment. Fortunately keeping a key safe is much easier than keeping a physical asset safe. And you'd only need to access your archive the times when you need to change the IP for your domain or are otherwise transferring/selling it.


Keys can easily leak; there was a high-profile example not so long ago where a bunch of GPG public keys shared factors due to a bug in generation.


SIM swap attacks happening at every level of the mobile industry would probably invoke cool rebuttals as well.

https://consumer.ftc.gov/consumer-alerts/2019/10/sim-swap-sc...

You are not going to be able to pay extra to make rotten to the core pretty.


> A contemporary example is domain names. The current domain system is a monopoly, and it sucks. You pay companies a never-ending and ever-increasing rent to do literally less than nothing. A domain isn't much more than a key/value pair in a small database.

Is there any legal or technical reason why someone can't set up a DNS server that resolves domain names to different IP addresses than those traditionally used?



Beyond network effect, nope. This is one of the more convenient ways to block in-app ads on mobile. Set your DNS to something like AdGuard's servers, and they'll resolve ad delivering domains to a no-op. And suddenly all major third party ads in apps or the web alike are gone.


> Beyond network effect, nope.

So why does no-one do this/ Obviously people do do it for ad-blocking, so why not for other purposes? Too difficult to get traction, perhaps?


You can do it on your own systems all you like, but the thing people love about domain names is that anyone who types in a domain can get to that particular site. The fact that we all generally agree where a domain points to is part of what makes them useful.

If I decide I want to have my personal website be called google.com I can edit my hosts file to point google.com to my webserver and everything works for me, but anybody else going to google.com is going to get Google's page instead unless I somehow convince them to edit their files or use my weird DNS server.

Even if I come up with some great reason that makes people want to accept that google.com is better when it goes to my website instead, what happens when someone else decides they should be able to use google.com for something else and they convince a bunch of people to use their weird DNS server? We could fragment the DNS all over the place and have several conflicting entries for every popular domain name depending on what DNS server you're using but is that really better?


People do, it's just niche.

OpenNIC has a lot of non-ICANN domains but runs in a fashion similar to ICANN (central organization deciding rules for TLDs). Iirc there are some small non-US communities that use this a lot.

There are also a decent amounts of crypto TLDs (.bit for namecoin, Ethereum has a couple, etc).

There are of course a lot of corporations that use custom DNS domains for internal purposes, like .corp or .internal or the like.

It is difficult to get people to change their DNS serves en masse to something non-standard. The lack of being able to get real TLS certificates is also an issue though there is work in this direction: OpenNIC is developing a traditional CA afaik and namecoin has some experimental stuff to support DNSSEC and TLSA to have TLS certs enforced by the blockchain.


There are DNS blocklists for mail servers. It's a slightly different thing, but basically there's if there's an entry it means there's a reputation for the mail sender. Some of them are present/not present and some have a score. Some are against the IP of the sender only, while others are against the domain name or full hostname of the sender. You just set your mail server to do the right lookup against the right DNS server, and incoming mail can be filtered based on the results of the DB someone else is maintaining.


There are a very few alternate roots that have actually gotten traction - the most famous of which is .onion


.onion is weird in that it isn't actually DNS; it doesn't support DNS records (like A, SRV, TXT, etc) and applications that use it don't use DNS to connect to things behind onion services.


There are no legal nor technical reason why someone can't do that. There's also no reason why the TLD can't be a number[1]

1: https://news.ycombinator.com/item?id=2179216


Just to clarify, are you thinking of something like E2EE?


Sure, Binance could run off with your money tomorrow if they wanted you, but if you keep most of your money in your actual Bitcoin wallet and only deposit what you need to, your money is 100% safe.


Big lesson learned from Mtgox.

Unfortunately wallets have their cons too, and there are a lot of people who have lost their passwords or thumbdrives and thus access to their crypto.

No silver bullet.


Web3auth/Torus and Argent trying to solve that problem.

But yeah, the moment you have key recovery, even if it's decentralised and requires multiple parties, you aren't 100% in charge anymore.

But, it's probably a good alternative to being 0% in charge with custodial wallets.


Didn't the Mtgox people finally get their BTC back but only after a few years and the price 100x'd ironically? Either way, diversify ones portfolio including where the moneys are kept.


Don’t know if distribution has happened yet. But there were at least a few investors buying claims, so plenty of people were able to cash out.

You could say that’s actually a better outcome than losing your private key.


100% safe from some forms of loss but you still have various risks associated with passwords, key phrases, wallet phrases, software bugs, backups, etc.


Centralization and decentralization happen in waves as a natural result of ossification and complexity growth.

As an example, IBM was once the dominant market force in computation. As the business grew and became increasingly byzantine, it was unable to respond to market changes or market demands in any meaningful way, and therefore the market became decentralized. This happened to Microsoft. If Apple continues on its current path it will happen to Apple. If Google continues on its current path, this will happen to Google. The entire internet was once significantly decentralized both in structure and in content providers. The content got silo'd by a few major players, and it is now swinging back the opposite direction while the structure is becoming increasingly AWS/Azure/Google. This too will eventually start going the opposite direction as those infrastructure providers ossify.

Nothing is forever.


"Most things are not forever"

https://www.webmd.com/a-to-z-guides/what-is-pfas

There.. Fixed that for you


"PFAS chemicals don't break down _easily_ over time"


I admit, I didn't reseach the thousands of PFAS chems for the joke :p


I'm beginning to suspect that we will not see a federated protocol as successful as e-mail in my lifetime. I would, in fact, be only mildly surprised to see e-mail become much less federated than it is today (which is much less federated than it was 20 years ago).

Spam (and other bad actors) is one huge, and obvious, reason why; it's led to a significant consolidation in e-mail as well; SMTP delivery is hit-or-miss these days, and you can get ahold of a person to fix it only if you are "big enough"

The web plays a big part in this too. You can continuously deliver a client that updates its protocol in lock-step with the backend. This makes alternative clients something hard to do, and if you can't separate the client from the service, then it significantly increases the friction to change. Eudora didn't care which e-mail server it connected to, and you could take Eudora with you when you got a new ISP and lost your old e-mail address.

How much of my monthly fees to fastmail is spent on delivering e-mail, and how much of it is "keeping up with the gmail webclient"? I suspect it's more on the latter than the former.


> I'm beginning to suspect that we will not see a federated protocol as successful as e-mail in my lifetime.

This is one of the smartest things I've read in a long, long time. It seems deceptively simple or shallow, but it's not.

At least once or twice a year, for the past ~15 years or so, I reflect on how the email protocol has survived and thrived despite numerous attempts at "reinventing email" and such.


.. and it's funny how a system where there are no guarantees of delivery (people email to reach their inbox in seconds, but the delivery time is not guaranteed at all, nor is delivery!), no privacy (IMHO most emails are transported in plain text and one only needs to sniff the traffic..), and no guarantee of authenticity (you can send an email with any "FROM" address, pretty much), well how a system like that can in fact be incredibly useful. Of course, the layman thinks all these things are in fact guaranteed...


In one sense it's simple and shallow, but I feel like it needs to be said.

In a way it's like saying "we won't have a baseball team as good as the 1939 Yankees in my lifetime" in the sense that e-mail is the '39 Yankees of federated protocols.

On the other hand a lot of boosters for various new federated protocols want to compare their pet project to the '39 Yankees when they it's more like the 2021 Orioles.


It is true that power laws are seen everywhere, and that there are reason for those.

I like the article, because it exposes very quickly a very powerful idea, but I disagree with the conclusion.

Information networks are built by humans, they are designed, they are not necessarily bound by the rules of evolution. Which means we may yet be able to find a system that is not afflicted by the curse of perpetual destruction and reconstruction.


It's very easy to design a system not afflicted by the curse of perpetual destruction and reconstruction.

The key challenge is that that system, by design, will be less efficient than other competing systems. So if there is a surrounding meta-system that allows it to compete with others, the other systems that tend towards consolidation will drive out your system.

It's essentially the same problem we see in economics today with supply chains. Companies that optimized their supply chains to have more margin made more money and won... as long as the supply chains were all running perfectly. But once COVID happened, it became clear that those businesses had no resilience.


> Companies that optimized their supply chains to have more margin made more money and won... as long as the supply chains were all running perfectly. But once COVID happened, it became clear that those businesses had no resilience.

Good analogy thank you.


"Information networks are built by humans, they are designed, they are not necessarily bound by the rules of evolution. Which means we may yet be able to find a system that is not afflicted by the curse of perpetual destruction and reconstruction."

Given humanity's track record, destruction and reconstruction seem like a solid bet.

Not to mention, systems will require change anywhere nature interacts with it.


When framed as extremes in a false dichotomy, centralization seems the better option. In the reality of decentralized social media servers, thousands of community servers of various sizes rather than individuals make up most of the nodes and homes for people. In the current #Twittermigration, the biggest general servers are buckling from the mass inflows of members. More servers are easily springing up to provide new homes. The architects and maintainers of the fediverse anticipated the need to make setup and administration easy, but we were maybe not prepared for such a big pulse. Thanks for the boost Elon.


That power-law comparison between major freeways and air travel routes is quite misleading, IMO. The highly dense road networks surrounding the major cities (i.e. around Los Angeles, Bay Area, Seattle, Houston, New York-to-Florida, etc.) are left out, but for comparison to air travel they need to be included. Doing so would generate a very different map (with a very different power law).

The freeway map shown looks like the major truck-freight corridor for transporting goods (many collected at coast ports from overseas shipping) across the country to warehouses and distribution centers, certainly not what most people use for travel by car. Hence, comparison to air travel doesn't make much sense.

As far as centralization and decentralization, for most things this comes down to technological limitations (ignoring the issue of economic monopolies blocking local regions from adopting the latest technologies). Local agriculture and local energy production are a lot easier to maintain (and benefit from no transport costs), than say, local chip foundries. Maybe someday cheap 3D printers capable of spitting out nanoscale RISC-V products in someone's garage will be available, but certainly not anytime soon.


Nit: Dunbar’s number is casually used in a way that doesn’t seem relevant here. It’s not true that we can only remember about 150 entities. People know many thousands of words.

(It is true that there are limits on what people will remember.)


Dunbar's number is that ON AVERAGE people can only maintain 150 relationships.

"maintaining relationship" is not the same as remembering or knowing the person's name. It is spending enough time with the person that you have an ongoing relationship, which is eased if you remember a number of details about them (job history, hobbies, birthdays, etc).

On average matters because the number is about group size, not any one individual's ability. For the group to cohere, most of the people in the group need to know most of the people in the group.

You will find people who can only manage 100 or fewer relationships, likewise, people who can maintain many more, even orders of magnitude more.

It isn't about the individual. It is about the group.


Another way to put it is that for a group of 100-150 or so, you'll be able know everyone decently well. A small college graduating class, or a tiny high school perhaps.

At about 300 or so, you'll "recognize" most of the group, but you won't know much about many of them, or really have a relationship.

And once the group is above that, you either need to have it hierarchical setup (you don't know all the members of the families at your school, but you recognize the parents, etc) or it just becomes a mash of smaller groups (no matter how you try to pretend it isn't).


Dunbars number relates to remembering relationships between people which is 150 * ~149 things not 150.

You might not remember what everyone in the group thinks of everyone else, but keeping track of grudges is often relevant.


It is nowhere near 150 * 149 because people in human networks have different amounts of connectivity, “influencers” with many connections, and others with few connections, as well as cliques with high internal connectivity but low external connectivity. And relationships between influential individuals matter way more. https://en.m.wikipedia.org/wiki/Social_network


Influencers are more a function of very large social networks, they don’t really represent small tightly knit communities very well.

Consider something like a thanksgiving dinner you have what looks like central figures say a grandparent that everyone is related too, but the grand kids have tighter bonds to their parents than the grandparent.

Extend that to a village and you get more social people with more connections across the community, but those aren’t even close to the strongest bonds.


the internet is even less constrained than the airport network. With practically zero friction keeping it in check, it can centralize completely. So, we can't put our eggs on decentralization, but instead we should put them on competition. Seemingly however, there is no competitor to silicon valley. And the valley itself is extremely anticompetitive and monopolistic, with different parts of the net being consolidated to a few monopolistic giants. our hope for competing ideas in digital networking are going to come from siloed out ecosystems, like china and russia. Indeed, the cold war was a time of great progress in almost everything


Is it possible to host a site on a Chinese or Russian internet such that it is impossible to be taken down by Western forces?


Yes but no, as the Chinese or Russians may be much more likely to take down your site.

If the site is supported by the country, it is VERY hard to take down.


I assume so, in russia

Of course western governments can block it


Plug: I'm working with the author on a decentralized protocol for networked thought and a companion browser called Subconscious. We're a very young project and just getting off the ground.

If you're interested in topics like the one in the article, all our work is open source and we love getting into it with folks on Github or our Discord (link in the README): https://github.com/subconsciousnetwork/noosphere


I've been exploring Noosphere as a protocol-level solution to the dilemma posed here about centralization. Some really interesting tech! Definitely worth checking out if this stuff interests you.


yet you choose two closed source centralized solutions for your project: Github and Discord


"We should improve society somewhat..."

Seriously though, I would be happy to have a polite discussion about the reason for our choices in our Discord server!


So make the change. If your idea is strong and has gravity then people will follow (clearly the guy who is working hard with no reward is a FOSS developer ;)


Preach yo! the medium is the message.


Have you considered not asking people to agree to rules before joining your Discord?


Yes, we have considered the likely social dynamics of a discussion space with and without a code of conduct.

The discussion space is for our project which we are inviting other humans to contribute to. We are open and welcoming to all, but it's not the town square. We strongly prefer to work in a social context where every participant has explicitly agreed to be kind and respectful of each other, to recognize each other's humanity and to abstain from violence of any kind towards others.

Folks who won't agree to such terms may still fork our work or contribute on Github if they are so inclined.


Isn't such a code of conduct generally implied? And isn't it more generous to wait until it is broken to bring it up?

Also, the rules are not so general or unambiguous as you've described them here. They mention speech patterns and topics that are highly loaded on contemporary politics.


To head off any uncertainty, the code of conduct we ask participants to agree to is nearly identical to the Rust Code of Conduct: https://www.rust-lang.org/policies/code-of-conduct

I have found that all it takes for one remain on the right side of such a code is to have good will in one's heart towards other humans. We have hundreds of participants, a lot of lively and thoughtful discussion on a range of topics, and not a single user has been found in violation of the code of conduct to date.


You say you're welcoming to all. But that isn't true, because you refuse access to anyone who doesn't agree to these rules. And a subtantial fraction of people would not agree to them, as you well know, regardless of whether they would ever break them.

That you got them from somewhere else doesn't head off uncertainty in their interpretation. It does raise the possibility that they're a bit of groupthink spreading through the open source software scene, versus something your group believes in enough to have indpendently developed. Historically such things tend to function more as shibboleths than meaningful exercises and I suspect that's the case here.


Whenever people use those words by themselves, it sounds like they don't have much information but are trying to form some conversation anyway.

Centralization and its opposite are extremely general concepts. You can't have a conversation about them because they mean nothing without specific context. Adding "the internet " or "the web" doesn't help because they're very vague too. It's like talking about "drinks" versus "not-drinks".


Centralization means easier driver for those who centralize and SPOFs. We centralize anytime we want an easy path even if it's fragile, and always resulting consequences are bad after a short "honeymoon" timeframe.

My conclusion then is different: we need common ideas, common tools not centralization. What's wrong with the classic web? Well that information access without search engines was a nightmare, but there is no need of "central" search engines as network hubs. YaCy offer an example FLOSS implementation of a distributed search network whose effectiveness much depend on numbers of indexing peers. In a fully "common web" all who host html pages also participate in the search distributed network and search quality became excellent, than some argue we can also distribute webpages like the ZeroNet idea and a new VERY FAST and VERY EFFECTIVE web is here. Oh, it's utopia just because of big players interests against the most and ignorance of the most, witch is not a technical thing. And that's just an example.

Logistic networks are another example where the centralized free scale network is easier than a hub-less network, but fragile, while a hub-less network is hard to keep, especially if not all nodes cooperate, but far more resilient.

The same happen in politics with historical absolute monarchy vs democracies and pretty anywhere else.

My final reminder: in winter is cold, in summer is hot, sometimes is too dry, sometimes too wet, we human as essentially any living being have tried our best to "mitigate" such natural phenomenon to live better. If certain free-scale networks are a natural thing that does not means we have to accept them without trying to made something better for us.


Centralization is a prevailing force. There are also many countervailing forces.


We centralize because we created/use a economical system where that such system behaviour is promoted.

We could also use a system which demotes or outright forbids too much centralization.

So it's not inevitable at all.

Or at least it isn't if you just look at how people are likely to interpret the title, reading "centralization" as in "central to the whole system".

Local increases in "centralization" are normal, e.g. people centralize in cities, but calling that centralization is misleading IMHO. They are local aggregates not central to the whole system.

More important if we e.g. look at (biological) nature centralization is nearly always the prelude of collapse and anything more then very localized centralization is often an indication for potential problems.

So we probably should change our system to demote too much centralization.


Decentralization is sort of a bad term in that it can have very different contextual meanings and the exact meaning is hard to pin down.

I like looking at it from a perspective of privacy, ownership/sovereignty, and choice.

A system that has a few dominant centralized players that are interchangeable is decentralized as far as I'm concerned. An example would be static web hosting. A system with hosted centralized options but where I can also self-host is also decentralized. An example would be something like GitLab.

I want systems where I own and control my own data and where it's not being data mined without my approval. I want systems where someone isn't always looking over my shoulder. I want systems that treat me as the customer not the product.


Hierarchy and centralization are ways to manage complexity and stay efficient as the article correctly points out but one additional point, topological pressures aside is also division of labour and specialization.

Sufficiently large systems benefit more and more from specialization, the emergence of content delivery networks on top of the more generic internet is an example. As the types of content we consume are getting more and more sophisticated, bandwidth heavy and capital intensive, specialized infrastructure and distribution pays off more and more.

For that reason peertube and other alternatives are pretty much structurally doomed. They'll never compete or be as economical as services and infrastructure optimized for that task.


This criticism of the Ethereum ecosystem also does an excellent job of identifying the forces that undermine decentralization in the long run:

https://moxie.org/2022/01/07/web3-first-impressions.html

One reason the blockchain has persisted longer than other decentralized systems is that it introduced economic incentives for maintaining its system. I think this is the best strategy for making decentralized systems that resist the tendency toward centralization while remaining competitive with centralized ones.


The author doesn’t understand bitcoin; The pie chart is misleading. All you have to do to see that it is wrong is see how the chart has changed over time. Why isn’t one big mining pool taking all the others? Why did the miners lose the block-size wars?


"Inevitable" is the last desperate cry of those who have given up thinking.


It can also be caused by people who think in an imperfect manner. Most people think according to how they have been taught to think, not what is perfect.

A critical component of a well designed trap: consider all suggestions that you are trapped in an imperfect methodology to be conspiratorial/etc, and therefore false. Checkmate.


I think I follow you. It's a way to shut down conversation before it's even begun.


Let's just hope that nothing like that ever happens on this planet!


Oh hey super cool, I think these are what you call small-world networks which show up in a lot of different places.

I’m actually reading a chapter right now on the architecture of the brain that talks a lot about this in the context of our neural connections: Chapter 2 or Cycle 2 “Structure Defines Function”[0].

[0]: https://neurophysics.ucsd.edu/courses/physics_171/Buzsaki%20...


tl;dr Large, centralised networks are better for data collection and advertising than smaller, decentralised ones.

Centralisation versus decentralisation should not be an either/or style debate.

It is possible to have both types of networks. Each can be valuable in different contexts.

For example, we need to have networks for noncommercial purposes as well as commercial ones. Small networks, e.g., L2/3 overlays, managed by their users can be useful for noncommercial purposes. Large networks managed by central third party authorities can be useful for commercial purposes.

With respect to noncommercial internetworks, one lesson of The Internet is that large scales ones are inherently disadvantaged. Besides the problem of proprietary, non-compatible protocols mentioned in the top comment, centralised "infinite scale" networks also become infested with "tech" companies, commercial intermediaries, who usurp them as honeypots for commercial and political advertisers.^1

On "The Internet", noncommercial network use, e.g., communicating with friends and family, is being compromised for commercial purposes by a relatively small number of single, central authority websites. Hundreds of millions of people with small networks of friends and family, e.g., less than 100 nodes, are being unecessarily intermediated by central "hubs" that collect data and serve targeted advertising and/or sell advertising services.

If those smaller networks were not connected nor managed by a third parties, then the attraction of injecting advertising into peoples' noncommercial internet use falls dramatically. The incentive for, and feasibiity of, data collection is similarly reduced. Those are features, not bugs.

Whereas for commercial uses, e.g., sales of goods and services, inter-business communications, etc., small networks are far less valuable. For these applications, we need networks that scale to large sizes. Arguably that utility has already been proven as successful. Advertising and data collection may be appropriate in the commercial context.

1. The irony is that The Internet was supposed to be a means of disintermediation. In some contexts, that is true. In others, it has allowed for the greatest level of intermediation in history.


It is much simpler to explain that in this article: centralization is inevitable because the UI/UX you use for entering any protocol (decentralized or not) is centralized. Hence the UI/UX could decide what to show and the order.

It could be call "the listing problem", you can decentralize Amazon (e.g. OpenBazaar) but at the end the app you use to list the product could censor and filter what they want without caring about the protocol behind.


"The highway network is not scale-free" - no shit, highways refer to roads of a specific size. You're taking a slice out of the road distribution.


Centralization is better than decentralization in 99% of cases, but the 1% is really important: when the centralized system fails, or is controlled by an adversary (e.g. malicious government).

IMO decentralized internet (and social media and currency) should be possible, routinely tested, and readily-available when the centralized internet fails. But a neutral, trusted, central server will is basically always preferred when it is available.


> Centralization is better than decentralization in 99% of cases

This is way overgeneralizing, and not remotely true for a large swathe of things. Centralized systems are actually less efficient unless the central agent is a party to every interaction due to the nature of the system. If the central agent is only a party by virtue of the imposed structure, you've created overhead, made the system difficult to change, and made it brittle to variance from design assumptions.

Decentralized systems can evolve much more rapidly, can survive catastrophic variance, and are more efficient in a large number of cases.

Centralizing mail delivery works well, centralizing economies or ecosystems does not.


> Centralizing mail delivery works well, centralizing economies or ecosystems does not.

Take the set of all systems conceivably used by human beings. Email providers, social networks, parent-teacher associations, dungeons and dragons campaign groups, neighborhood watches, everything.

Now, classify each of those things into one of two groups: either "more like the USPS" or "more like national economies". What is the ratio of "USPS-alike" groups, versus "national economy-ish" groups?

What does the obvious answer to this question -- 100% vs. 0% -- tell you about the relevance of centralization vs. decentralization?


If you want to define centralization as humans gathering in groups, I guess.

A large portion of human social interaction is peer to peer though.


Centralization is a property of systems. And yeah, as participation in a system increases, centralization becomes increasingly necessary.


Tim Wu makes a similar argument in The Master Switch, a history of information networks and monopolies going back to telegraph networks in the 19th century.


Why stop at the 19th century? It's not like the cost of transmitting information was less expensive before then, nor was it less centralized (people who had the resources/knowledge to propagate such information).


The book already was pretty long...


Centralization is just the impact of digitization. Digitization is inevitable, because efficiencies are tremendous.

Then you will have a battle for control of centralization centered on verifiable identity, kyc and centered ultimately around financial system control, which is the center use case of centralized identity platforms.

All incentives during this phase are centralization, consolidation and scale. Decentralization efforts will be carefully controlled and excluded by very powerful interests. Financial incentives will prevent meaningful adoption.

The carrot will initially be digital money centered on banking directly in the Federal Reserve. Those licenses have been issues for several years and the direction is clear.

There is a lot of carrot left to fuel the consolidation phase, and as economic conditions change and global conditions degrade people will embrace centralized solutions in the short term.

Then you will have an interruption to the central control systems which will lead to decentralized systems.

It will take much longer than people realize for this cycle to play out. We’re still in the consolidation phase. I’m a huge believer in decentralized systems, but like self-driving cars, it’s less of a technology problem and more of an autocratic and powerful interest story.


Sorry but you are full of shit. Centralization is not digitization. Centralization is relying on the use of coercion. It's a /central/ source of truth.

The identity system we've been stuck with since WW1 will finally be replaced with something decentralized and approaching fair thanks to digitization. What we're lacking is the software that gives individuals self sovereignty.


That’s not a very kind response.

I’d love to have a constructive conversation rather than resorting to base name calling.

I mean to say that centralization is caused by digitization because of the financial incentives, not that centralization IS digitization. And all of our current systems are systems of legacy control.

In fact I generally agree with you. Generally the identity system will be interrupted, and at the core digitization of the legacy system will happen. I’m simply saying that there will be further consolidation and centralization first and it will take longer than most people think due to the powerful financial interests involved.

To put it in your own words, I generally believe we will have to go through a economics “with the tragedy” before we get to the part where the tragedy gets removed.


I believe it was kind but the tone does not get across the textification.

Glad to hear your justifications. They are more persuasive than the initial representation.

Thank you for engaging the constructive conversation.. now w.r.t. the power of financial interests, I think it is overestimated and easily shattered but we will just have to see what happens.


> I believe it was kind but the tone does not get across the textification.

I don't know that it is kind to say to someone that they're "full of shit".


Sorry I just read this exchange again and I totally misread your first comment. Tbh I'm a bit tilted at the moment. I'll take a break.


I absolutely love you for this comment. I totally understand and have been there lately myself. Hang in there my friend.


Centralisation and decentralization goes in cycles. When we centralized too much, it breaks down, and the cycle repeats.


The pace at which this happens is determined by the barriers and limits of the context. This is why centralists want to have big barriers such as proprietary technology and government backing by laws and other force that protects their monopoly. To create a more dynamic environment in which we can swing back to more decentralized when a central hub becomes too powerful, we need to support communities and open protocols and resist the barriers, laws, and policies that prevent.


I can't help but compare to US elections, the higher the bar to vote and less registered voters, the more centralization Republicans can pull off. The only way Democrats can win is if voting and registration are easier and lots of people show up to vote, at least nationally.



You can centralize things... on yourself or your family.


It's also worth mentioning that in order to increase your status in any of these networks, you're going to have to serve the central nodes in some way. So if you're trying to grow your social media following, eventually you're going to have to get the attention of an influencer who will in turn rub off some of their influence on you. If you're trying to increase your capital, you're going to need to provide value to some influential capitalist. If you want to increase your political power, you're going to have to be appointed/elected to a role in some powerful political institution.


And yet our economic and social systems have never been more decentralised...


Yea, not really. The wild west was wild. People used to make up their own banknotes, there were fraud and schemes everywhere. We take for granted that our bank won't steal our deposits, that's a new concept. Most of that is derived from some degree of 'centralization' - that said - it's definitely not this 1-to-many scheme that the language implies. The Fed has immense power, but is not really in your business. You can exchange money with whomever you want (almost) and it's none of the Feds' business.

Much like 'AI' I think this word 'decentralization' triggers something within.

It's worth considering that impulse (i.e. freedom is really important, and we should think about it), but also needs constant contextualizing.


What your describing is just a different, more local form of centralisation. If your bank manager gets to refuse to give you your money, he has all the power. That's very very centralised, even if it's centralised in him, not in a CEO or a president.

I have 4 accounts at 4 different banks. If one of them refused to hand the cash over, even without any redress, I have 3 other accounts to fall back on. So my life is much less centralised than Wild West Bill.

The same applies to almost all elements of life. Back in the wild west, no one would date you if you didn't go to church. The priest literally had centralised power to refuse you a partner (and God help you if you were gay or the wrong colour or from a "bad" family). Today no one knows or cares. Very very decentralized. Ditto employment or housing or crime and punishment.


The bank manager will go to jail if he doesn't give you your money. And then the bank will give you your money. There is a 'system' of laws and regulations.

This is unlike true decentralization i.e. 'trustless', which, paradoxically has to be based entirely on trust.


But also, the wild west was founded upon the US wars with the native americans (centralized) and railroad monopolies (centralized).


Good point, except neither of those were as centralized as you're implying.


Meta and Visa suddenly go permanently offline. What does the social and economic landscape look like now?


Meta goes offline, nothing of any importance changes.

Visa goes offline, people switch to mastercard or use cash.

Now compare that to living in the wild west, and the only town in 100miles bans you because you're slight different to them.


India has its own payment processing stack, UPI so I expect Visa going down shouldn’t lead to a major disruption. It’s yet to be tested however so we won’t know until Visa actually goes down.

WhatsApp going down on the other hand will be a big pain as lots of businesses and people rely on it. Though I expect them to figure out alternatives within a few weeks.


It's a little rough for about a week and by a month out everything's basically completely fine.


I go to the bank and get cash. Businesses accept that cash for goods and services. I observe no difference in my social media habits because I do not use Meta products.


Good for you, Mr. Exception-to-the-rule. (Btw, the same is true for me too).


And me. So your "exception" hypothesis is getting weaker :)

You've really got me thinking.

Is there a word for that thing when "everyone knows" what they think "everyone is doing", but once they start to dig deeper it turns out nobody really is? Everyone is pretending to, because they think that's what everybody should say and do?

Mass delusion? Group fallacy? Fashion? Groupthink? None quite capture it.


I think what's happening here is selection bias. The people who proudly have no Meta accounts are replying to each other saying "me too". The hundreds of others who were reading did not reply.


Sample bias is a thing, sure.

I'm alluding to something a bit more interesting, I hope.

We don't have a way to test those hundreds of voices comprising the supposed silent majority. I'm saying that when/(BIG if) one does - via some hitherto undevised ingenious experiment which always yields truth - we find that no... in fact everyone is at best ambivalent, but mostly going along with what they think the majority view is.

I'm sure there's a name/concept for this in group dynamics.

Consensus mythology? (I'm just making that up)

A hallmark is that interrogating the group yields one answer, but each individual considered privately will give you a very different answer.

But it's not "group-think" I'm talking about, because that implies a more overt pressure. Rather it emerges in the absence of coordination amongst an implied majority when there exists a loud propaganda message designating some other group a minority.

I think this effect has important implications in the kind of 50:50 + swing elections we've seen over the past few years. For example, in Brexit, a mass of lethargic voters assumed "everyone will vote against" and didn't verify the reality by asking lots of diverse friends. It's not that people are trapped in actual bubbles so much as bubbles of their own mind based on assumptions about those around them.

Sorry, that's probably a long and clumsy way of saying something obvious.


I see what you're getting at. I do know that many people, when asked about Whatsapp (especially parents complaining about the "obligatory" school Whatsapp groups), wish they didn't have to use it. I'm not sure how serious that sentiment is, or if they really would wish that if they understood the consequences, but as an experiment I have been attempting various community organisation projects that explicitly do not use Whatsapp. They've been quite successful, so far! We have just started co-ordinating a monthly kid's play session in our street, and I have a regular biking group.


"Consensus mythology"

in philosophy it is known as heterophenomenolgy hetero = same, phenomenology = experience


Thanks gg, that's an interesting lead.


Sample bias. HN crowd != people I know in real life.


Just go and get cash during a bank run. How hard can that be?


That wasn't provided as a condition. In a bank run, Visa's existence or non-existence is irrelevant. Credit is frozen.


Do you really think Visa and Meta could go under without causing a bank run?


By "Visa" do you mean "all payment clearing houses" and by Meta do you mean "all social media"?

Because if Visa shits itself (it has) I use Amex, Mastercard, cash, debit, or even checks.

And if Meta shits itself I don't even notice except to make fun of it.


Think about the situation for a second. You'll be switching to other methods, and so will everyone else, all at once. Do Amex and Mastercard even have the capacity to take up the slack with no notice? Do you really think that every critical component of a cost-optimised system will have been over-provisioned to be able to take on hundreds of millions of additional customers in a single day? And if one fails, the demand switches to the remainder and crashes them too. It's a classic cascade failure.


I suspect it would take up the slack - depending on how the failure happened and when it happened.

I don't believe payment processors have the same number of payments every second, so Amex and MC have to be sized for the largest spike (+ some) that they can endure, and so unless Visa fell right when they were already maxed out, they'd likely continue.

And Visa has fallen before; if it was anything like a long-term failure, you know all the other processors would be spinning up as much extra capacity as they could. And many stores still have the paper machines for credit slips.

https://www.wired.com/story/visa-outage-shows-the-fragility-...


That seems about as far from true as a statement can be.

Can you give an example?


Can you elaborate?


In my modern, urban, life I have 1000 different options for my food supply or friendships or housing or employment. Aside from a very unengaged, far away, government there is no centralised entity in my life. And even that I can replace by moving country.

100 years ago, there was one major employer in my area (east London), a few grocers and bakers, people's activities were tightly controlled and linked to family status. Participation in all elements of economic and social life was tightly controlled by a small clique of economic, clerical and political elites.

People think about centralisation as a corporate or government thing and a unitary national authority. But it's not. It's part of every later of society. When we lived in tribes, the chief had totally centralised power over almost everything. Villages were the same with an overlord somewhere you'd never see. Towns were lighter but still quite centralised. Modern cities and urban life is about as decentralized as it is possible to be.

That's my understanding of centralisation. Sorry it's not better explained.


I bet there's a true hive-mind species out there that would disagree.


Even bees centralize around the queen bee.


I think this is now widely considered to be a false anthropomorphization of what's really going on in a hive.


That is, until the ease, quality and utility of fully decentralized systems exceeds the pain of the fragility and abusiveness of the current centralized offerings.

The power of unenclosable carriers offered by breakthroughs such as Holochain https://holo.host is breathtaking.


> That is, until the ease, quality and utility of fully decentralized systems exceeds the pain of the fragility and abusiveness of the current centralized offerings.

I agree. A centralized system will seek to expand to further consolidate control & create cartels which often act against the interests of those outside of the cartel. A system of decentralized components that becomes centralized can be forked to create alternative systems that address the interests of those who maintain the alternative systems.

It's like forest garden management technique...using fire to create new growth & biodiversity. The mature forest has less biodiversity & will eventually die out on it's own, but a fire will clear the less "fit" organisms leaving space for new organisms to grow.

> The power of unenclosable carriers offered by breakthroughs such as Holochain https://holo.host is breathtaking.

Is the Holochain api/architecture stable or is there still churn?


This seemed like a cool project that actually had a use for a cryptocoin that wasn't pure scamming. Shame it was a scam ponzi anyway


Hmm? SDK reaching 1.0.0, almost ready for public beta, and capable of deploying global-scale full-stack Apps without server infrastructure?

Perhaps you have it confused with another project.


My issues with the project aren't related to the technology, but the rapacious insiders and victims who got ripped off. I know, i know, they're greedy and stupid for getting ripped off but I still feel bad for them and don't like people that do this stuff


You aren’t addressing the points the article made about efficiency and other selection pressures.


Classic peer-to-peer networks have low connectivity. Scale-free networks are resilient (most nodes don't matter), but are vulnerable to attack (kill a few "hubs").

DHTs (such as underlying Holochain) are scale-free, but also automatically heal from attacks.

Most importantly, they operate essentially unimpeded even in the face of successful network partitioning attacks.

All while each "agent" (You) retain complete cryptographic control of your data and identity.

And, if/when you want to migrate your data (exfiltrate), re-package the functionality (create new clients, or integrate multiple networks), or even completely fork the entire application -- all of this is at your fingertips, and no entity can stop you.


+1 - author argues that some degree of centralization is a natural property of interconnected systems to due the aggregated effects of things like preference attachment and economies of scale.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: