Hacker News Reader

Comment by commandlinefan | Hacker News Reader

Well, the author is talking about the "mainstream internet" that most people think of: we used to have to point out that the internet wasn't the same thing as the world wide web, now we have to point out that it isn't the same thing as facebook, twitter and amazon. There's nothing inevitable about a centralized internet technologically: packet-switched networking is by it's nature decentralized. However, my pipe dream of someday seeing a truly decentralized mesh network that nobody can ever control or censor gets less hopeful every day - not because it's technically infeasible, but because it would require more people to get on board than I believe people have the will for.

View on HN

Essentially, the tradeoffs don't make it worthwhile for the average user: having to do more work to interoperate with fewer people? "Sign me up," says almost nobody.

Actually, I am working on a technology to do exactly this. With the increased trust that comes from extreme privacy, simplified permissions management, and a new security model users can comfortably do things that aren’t available on the WWW.

When you continue to think in terms of social media where a centralized service extracts user data as a blunt financial weapon from a massive user count then decentralization doesn’t make any sense. When you turn this around to solving more practical problems the applications and revenue model change.

Nobody perceives existing service in such terms except for HN geek crowd. An average person doesn't even have a concept of online privacy internalized, for him worrying about online privacy is basically the same as worrying about TSA agent examining his vacation backpack: he has nothing to hide. But it's even worse, since airport examinations happen in physical world and they cause at least some inconvenience, whilst FAANGs harvesting your data is invisible and only brings more comfort.

Advertising something with better privacy, permission, etc. will only bring you people that already care about it, i.e. the same 0.01% of geeks. I can see only two ways to achieve the goal of decentralized and privacy-respecting "Internet":

- Convince people that privacy is important. A very hard task in our age for several reasons. Also, it needs to be supported by legislation.

- Make the services more convenient to use than existing ones. Arguably impossible due to manpower imbalance and technical issues (privacy and security almost always come at a price of convenience).

> An average person doesn't even have a...

An average person has whatever perception the market and product communicate. If you are banking on perception alone the current approach is good enough. The only way to push through that is to provide new capabilities or solutions to old problems other approaches refuse to provide.

You aren't selling privacy. You are selling a product with privacy included.

Yes, this is what I'm saying. So in this this case you're trying to make a product that is both superior in terms of convenience, features and price AND has privacy included. If you manage to do that, then yes, there is a chance. It's just that it's extremely hard to outperform current incumbents given that you're setting yourself very serious constraints, both technical and ethical, compared to them. Not to mention the budget.

At various points in time 0.01% could be said about computers, Internet, Web, Ad blocking, 3rd party cookies. Pioneers pave the way.

And specifically, few people want to join services that are mostly populated by people who got kicked off of other services.

Deplatforming from the big services helps create a moat around them by creating a huge population of obnoxious trolls, crackpots, and hate mongers who will descend upon any newer or smaller challenger like those clouds of moose killing black flies up North.

Look at BitChute and other small video sites now that YouTube has been deplatforming trolls and crazies. They are full of toxic waste. BitChute now gets called “poopchute.”

> They are full of toxic waste.

4chan is considered toxic waste. Old Youtube was full of toxic. Old Tumblr was toxic waste.

Those are / were home to the most creative communities.

Now? New centralized media is starting to look a lot like old media: standardized, no spice, boring. Perfect for a protestant society.

Old 4chan, YouTube, and Tumblr were not full of neo-Nazi propaganda unless you count ironic jokes that were obviously jokes, so no. Then the great Basement Blitzkreig of the 20-teens happened.

Were they obviously jokes? Because in hindsight, a lot of that behavior looks like it was really clandestine signaling allowing neo-Nazis to find each other

>in hindsight, a lot of that behavior looks like it was really clandestine signaling allowing neo-Nazis to find each other.

Of course it was. The premise that racists wouldn't take advantage of communities which let them communicate openly and gave them the plausible deniability of "irony" while doing so is absurd. Why wouldn't they, when the community "mocks" them by acting just like them, reinforcing their beliefs, spreading their gospel and making them feel as at home as possible?

I mean, it's not exactly a scathing rebuke, is it?

>Deplatforming from the big services helps create a moat around them by creating a huge population of obnoxious trolls, crackpots, and hate mongers who will descend upon any newer or smaller challenger like those clouds of moose killing black flies up North.

In many cases these smaller sites were created specifically to cater to those "obnoxious trolls, crackpots, and hate mongers" as a "free speech/censorship resistant" alternative to the mainstream. BitChute's selling point is serving content which would be banned on YouTube - obviously that's what they get. If they or similar sites are victims of anything, it's their own success.

That doesn't invalidate the point though. You can be entirely ethical in the goals of your platform, as well as have a legitimately ethical audience you hope to cater to and still run into the problem pointed out.

The only argument you need to make for a competitor to make sense is "<Platform X> isn't perfect." That's accurate for all of them. A good competitor will have all the same challenges a 'bad' competitor would.

If those people end up being less like the average person on social media, that could be a good proposition

I don't use social media (facebook, twitter, linkeding) because of the kind of people on it - but I do follow a series of website of interesting people.

Hacker news is somewhere in between. The content is half twitter / half interesting.

> interoperate with fewer people

Quality, not quantity... ;)

interoperating with fewer people means that the quality of the people (for those operations at least) can be increased.

As a schoolboy, I had friends whose parents consciously didn't bring broadcast TV into their house. They did have a vcr. They didn't want TV.

You won't reach Carol Couchpotato, but there will always be a core of people willing to living outside the FAANG coral.

If you care about the decentral internet, set yourself realistic expectations. Work with the coalition of the willing. Make it as welcoming as you can. It'll be fun times!

I really like this point. What really matters is your experience, your connections, and your contributions. What everyone else has done has usually been a herd mentality, now is no different, but being able to carve out a little space of your own with those who want that as well--- that is more possible today than it ever has been. In a way, we are in a golden age of being able to do it. Be the change you want to see in the internet.

What it would take is for us all to set up nodes. I'd do it, if anyone else would: tragedy of the commons. At least ham radio guys are doing it.

Set this up: https://spectrum.ieee.org/geek-life/hands-on/build-a-longdis...

I'm game.

You are a hero. Thanks for the link.

This domain always returns HTTP status code 418 in my locale...

>nobody can ever control or censor gets less hopeful every day

Any communication medium that isn't controlled or censored will break under a deluge of spam.

Weird, that hasn't happened to the ones I've been on for years.

If someone tried to build an app on one of those systems where you were forced to accept messages by default until you blacklisted them then it probably would. It would be terrible.

Thankfully no users actually want apps like that, we've already got email and web social media sites for having a public directory of strangers to contact. That is a use-case that is only sometimes needed, and when it is you only need to use it once.

Not true. I've been thinking about this problem for a while and there is a solution: https://adecentralizedworld.com/2020/06/a-trust-and-moderati...

This is still centralized. You have 1 place that stores the trust ratings. The fact that it crowd sources the ratings doesn't change that it's centralized.

It also doesn't really work. You have to choose between not gettimg messages from those outside your circle of trust or getting spammed.

Thank you for this fascinating write-up. It's given me a lot to think about. I'm quite curious as to why no existing social networks have tried to implement your ideas in a meaningful way, and to a lesser extent why no new start-ups have.

Perhaps when I have a few hundred million to invest in a better platform I'll reach out! (One can dream).

You can ban obvious spam and not censor the rest. So yes, you'd get some of the spam, but at least you wouldn't be censoring the rest. But you'd still need _some_ form of control, of course

We see that a lot in practice, but it's not clear that there aren't solutions.

For example, if it costs someone $0.00001 to send you a packet, that could still be decentralized but the economics change a lot. Maybe for the better.

But then spammers just send like they usually do, by exploiting somebody's insecure setup and using their resources.

If being exploited in that way was more expensive, it might happen less often.

There is no solution which isn't equivalent to censorship, because "spam" is just a category of speech.

It isn't clear to me that we've got a good technical, never mind social solution, to the problem of anonymous actors.

I suspect that "be careful what you wish for" is applicable to the idea of a communication medium that "nobody can ever control or censor". We already have numerous problems caused by anonymity with our existing infrastructure (DDOS, ransomware, threats, defamation, etc.).

I think neither extreme, a completely anonymous infrastructure or a completely non-anonymous infrastructure, is desirable.

I think Urbit's approach here is actually pretty good.

Pseudonymous IDs that have a small, but non-zero cost.

Your reputation is tied to the ID and the economics don't allow spinning up millions of them to spam people.

They're building a from the feet up first principles approach to really solving the centralization incentive issues of the modern web and operating systems.

It's super ambitious, but I think it's pretty cool (and it's not vaporware, the current stuff actually works - it's just early on).

A lot of the other attempts at decentralized systems can't fix the server/client model or the spam problem - all which push users back to centralized services.

https://urbit.org/understanding-urbit/

Urbit is interesting, but this troubles me:

> there are only 2^32 (~4B) Urbit IDs, so they cost something.

> Ultimately, we want your Urbit ID to feel like a civilizational key. If your Urbit ID were a piece of hardware, you could tap it to unlock a door, swipe it to buy a coffee, and plug it into any computer to log in. Your Urbit ID should be a unique, beautiful object that’s both an address and a wallet. It’s a key to a secret club and the ticket to your digital life.

So I guess almost half the current global population is excluded from civilization?

Apparently it's taken us ~30 years to come close[1] to this number of users on the current internet. Seems like not an issue for a very long time, and if it becomes one, it can be amended.

That said, anyone can get on Urbit for free as a comet. The only real limitations on comets are social in nature—not technological.

[1]https://en.wikipedia.org/wiki/List_of_countries_by_number_of...

They talk about that - if they're approaching the limit they'd make changes to produce more IDs.

The network has a built in governing body that can do things like that (and they would since restricting access isn't the goal).

But if the number of available Urbit IDs can be scaled up and down on an as-needed basis, why did they start with 4B and not, for example, 1M?

32-bits is just a convenient round number to wrap everything around.

I like the idea but it does seem like you're setting up a poll tax to participate fully in the internet, simultaneously having it high enough to deter or at least raise the bar on scams and low enough to be affordable to the whole world is a hard ask.

They have free IDs too, but groups can block them to reduce spam.

The free IDs have some other negatives too I think (long names, harder to recover, etc.).

The price is quite low ~$10 for a 'planet' or standard ID. Maybe in the developing world that price is too high, but I'd suspect if it succeeded to the point where that was an issue it could be solved without too much trouble.

The ID solution is cool because it fixes the spam issue which is one of the core incentives behind centralization.

The other core reason to centralize is server management and finding other users. If you design an OS that has this bit baked into it from the start you can make this complexity invisible to the user. You can have apps be p2p by default without the user having to know about it, you can make it easy for users to send photos or messages directly to each other rather than always having to interoperate with a middleman company (FB, Google etc.).

Their approach of starting as a VM that runs in linux is a good idea, it allows iteration on something that's immediately usable. Their initial focus on chat too I think is smart - it gets people using it while they improve it and immediately seeing changes.

Long term goal would be getting urbit running on its own hardware without the virtual layer, but I think if they started trying to do that they wouldn't succeed. With the current approach they might.

> I like the idea but it does seem like you're setting up a poll tax to participate fully in the internet, simultaneously having it high enough to deter or at least raise the bar on scams and low enough to be affordable to the whole world is a hard ask.

It's not actually that hard, because ordinary users can generate one ID and use it for many years, but spammers have to generate a new ID every five minutes because they immediately get blocked and have all content associated with the blocked IDs get retroactively dumped in the spam folder. So the cost to spammers is literally a million times higher than it is for ordinary users, which is exactly the sort of thing that actually works.

It's still important to think about how much $10, the current Urbit ID price, represents in some parts of the world. The World Bank has a poverty line of ~$3-4 per day and over half of India lives (or did in 2017) on less than that and the official rural poverty line is around $6-7 a month. If we're setting up the next internet we really shouldn't be locking out anyone due to price.

Even for someone living on $3/day, paying $10 once every ten years represents 0.09% of their income. You could also plausibly impose alternative costs, like making someone solve captchas for an hour. Then people in rich countries pay $10, people in poor countries spend an hour (which to them is significantly less than $10 of their time).

that's like 500-1000 USD for the average middle class american - a steep price to pay for a membership with no obvious benefits

I wonder why can't score IDs by page-rank them - one's identity gets credibility based on the social score - hacker news manages to be quite effective at preventing spam using this technique

> that's like 500-1000 USD for the average middle class american

Which would be $50-$100/year, i.e. still not very much.

Recall that it costs significantly more than that to have internet access.

> a steep price to pay for a membership with no obvious benefits

The obvious benefit being that you get to be in the same network as the people in rich countries, and that network is not overrun by spam or controlled by a multinational conglomerate that uses its ownership of the means of communication to cost you significantly more than $10.

> I wonder why can't score IDs by page-rank them - one's identity gets credibility based on the social score - hacker news manages to be quite effective at preventing spam using this technique

What do you do with new IDs that have no reputation? If they can't post they can't earn a reputation, if they can post they can spam.

> So the cost to spammers is literally a million times higher than it is for ordinary users, which is exactly the sort of thing that actually works.

The law-abiding user is the one that incurs the real cost. Spammers pump and dump millions of IDs. This drives up the cost of getting an ID. The normal user that just needs a new ID for some reason is the one that suffers the most here.

I don't care if it's expensive for a scammer. If it's cost-prohibitive for grandma to get a new ID, then it defeats the purpose.

Email spam could be almost totally eliminated if you had to send someone a 1-24c micro transaction for them to receive your mail.

Over the course of a year the costs to send would be balanced by the profit from receiving for your average person but for spammers it would be untenable.

Is email SPAM really a serious problem today? (Unless by SPAM you refer to marketing emails of various sorts that you agreed to in some manner and that provide an unsubscribe link.) Sure. Charging for emails would cut down on the volume some--and would eliminate the email that's occasionally interesting but I don't routinely read--but would also marginalize the use of non-commercial email for those with little money.

It's only not a problem because centralized megacorps solved it and everyone has to use them because of the spam problem.

If you're trying to come up with a model that isn't using centralized companies to solve the problem for you then it's still a problem.

Part of the incentive towards centralization is because the spam problem is hard to solve in a decentralized way with the modern web.

> It's only not a problem because centralized megacorps solved it and everyone has to use them because of the spam problem.

Is this a perception or do you have data on this?

See my response to parent. Email spam from the point of view of self-hosted infrastructure is no longer a problem in my experience. No megacorps needed (or wanted, IMO).

The problem I had, when I tried to de-megacorp my e-mail, was with knowing whether the e-mails I sent got reliably delivered.

I set up SPF, DKIM, DMARC, used an IP not on any blacklists, and so on - everything guides told me I ought to do. Still, not only would my test messages often go to spam, I'm pretty sure I saw some vanish entirely, not even making it to the spam folder.

And this is a very insidious problem - did I e-mail X and they just didn't deign to reply? Or did my message never reach them?

I wish there was a 1-step process for white-listing known good emails, for a single address, or a single domain, or a list of either.

I also wish there was a process for exporting/importing these lists.

With SPF/DKIM/DMARC (whichever of those verifies that the email header was actually set by someone who has access to that email sending address) on and open-relay on STMP turned off, I envision a web-of-trust-lite world (which is probably the wheel I'm reinventing) where I and everyone I exchange email-address-trust-cards like business cards help to crowdsource what is and is not spam based on shared whitelists and degree-of-separation weighting. If I think something is spam, my 1st degree of separation (who has handshaked with me to verify that yes we've met) also probably does, but his colleague, maybe not.

MUAs and online email service providers would check the compiled whitelist for each account before running their own spam filtering.

We have good online adblock lists, can we do the same for email spamlists without 27 clicks to whitelist or blacklist emails?

Servers would have to not blackhole spam-tripping emails and reply with 'you spam = bad', that is another problem entirely.

Personal anecdote: I wrote my own email server; it has no spam blocking of any kind other than checking the Spamhaus blacklist. That gets me down to a spam message every few days at most.

I've wondered how much spam filters actually cost in terms of compute resources and electricity.

I bet it's a lot. I bet there has to be a better way.

One tool I use is called greylog for postfix. Effective for me.

Totally fair. Though the only way to solve spam control in a decentralized way is probably to put in a system that no one will actually use.

> Is email SPAM really a serious problem today?

Spam in email is basically not a problem anymore, in my experience.

~15 years I would get literally thousands of spam emails per day in my spam folder. But this was the days when email servers were configured to accept anything no matter how malformed. And many people had misconfigured smtp clients, thus attempting to enforce constraints would drop a lot of legitimate mail. So spam was a huge problem.

Today it's very different. I no longer see any smtp clients from legitimate senders with any protocol misconfiguration. So for years now I enforce every detail at the smtp connection level (with postfix), before the email is accepted.

I still see failed attempts in the postfix logs but they get blocked right there. Actual spam getting through to the spam filter is down to a few a week.

Then checking DKIM drops all of these in the spam folder.

Spam that gets through these steps to my actual inbox is now so rare that it's always a surprise. Less than ten a year, tops.

So from my perspective (running my own email infrastructure for a handful of domains, personal and business, friends and family) email spam is no longer a concern.

> DDOS, ransomware, threats, defamation, etc

These problems are not caused by anonymity. DDoS and ransomware are problems that have their roots in poor security of consumer operating systems. Very few DDoS participants or ransomware victims are willing to be such. Threats and defamation are often done by people literally posting on social media accounts with their names and personal details.

It's actually really easy: force people to pay per packet. The problem is that we don't have any global system that has solved fast micro-transactions at a global scale without the concept of a permanent identity.

At the dumbest level forums would look much different to what they do today if every post cost you $1 and every downvote $0.25.

>my pipe dream of someday seeing a truly decentralized mesh network that nobody can ever control or censor

Sounds like i2p: https://geti2p.net, except it's done on top of the Internet (but would also work on a mesh network).

That, or Freenet. Or Tor. None of which ever really took off - mostly due to user apathy rather than technical feasibility. Sadly.

Apathy is a symptom of lack of immediate value. Torrents and p2p went mainstream when dinosaur content distributors failed to provide service that people wanted. That turned into streaming services which made bank because they addressed the need for on demand entertainment. Centralized media publishers like Facebook and Twitter are now the dinosaurs. Create a news service that fulfills the need for unfiltered on demand reporting and you can be it. Look at the streamers that are beginning to do this. Give them a platform that can’t be censored and allows people to subscribe and donate to them. Figure out the video storage and delivery problem. The world needs you.

Have you heard of Arthur Brock and Eric Harris-Braun's project Ceptr/Holographic-chain? They have developed a distributed p2p data integrity engine that enables agents to bootstrap new distributed apps as fast as the Ruby on Rails framework:

"Holochain is an open source framework for building fully distributed, peer-to-peer applications.

Holochain is BitTorrent + Git + Cryptographic Signatures + Peer Validation + Gossip."

Scuttlebutt [1] is the Decentralized Mesh you're looking for. Still early stages but will only improve over time. Has a few thousand users (approx, it's a mesh after all), and is developed completely in the open with clients written in multiple languages.

It can't be censored or shut down because all communication spreads through peers, with few central points that can be blocked.

1. https://scuttlebutt.nz

Torrents seem to match your definition pretty well. And it's a quite successfull technology.

Even if you had a mesh network, global low-latency communication requires undersea cables, and regional high-bandwidth hubs. Besides the undersea cables, local hub-and-spoke networks are needed to provide low-latency regional communication.

Centralization of network infrastructure is shaped by the physics and capital costs of communication technology.

Indeed. There are dozens if not hundreds of people consistently working on decentralized tools that fit the current laws/social climate. Not even China can fully centralize the actual internet, so what hope does anyone else?

China doesn't have to control/centralize all of the internet - just enough that the vast majority of people cannot access content that cannot be policed. VPNs still very much work and are used by tech savvy there - but they are effectively useless for most people, thereby just making them not attractive.

VPNs are the dumb persons idea of a secure internet. You've just sold all your secrets to a third party.

Tor is a slightly better solution, but again, not by much. You're hoping that a randomly chosen VPN will not have an interest in what you're looking at and that the two VPNs behind it are owned by someone else so they can't correlate the metadata.

I know. I've never advocated for using VPN providers as a valid way to get Internet privacy.

All good, it's just that I've heard too many people suggest VPN as this magic bullet that solve all our problems.

In my experience VPNs are used by everyone I’ve interacted with, including very tech unsavvy users.

i used to be part of a decentralized mesh network. I see that part as less likely each year.

Things centralize, then when the central node gets corrupted, everything is perturbed for a while and somewhat decentralized, until the next centralization begins. Same as disruption theory.

Network effects apply to centralized networks moreso than to decentralized effects. Cost to add a new node to a decentralized network generally remains higher than onboarding a new customer to a centralized network. And that kills it over time.

(Consider for example that the "new node" may be malicious... a decentralized network needs a lot of duplication of effort to fight malicious actors, a central network can do this cheaper over time.)

Even git, built on the very idea of decentralization, centralizes onto github and gets a lot of value out of that.

Your pipe dream is thought of and echoed by many, but the problem I see on an all-too regular basis is that many/most worthy projects never seem to reach critical mass. It's not as if we don't have enough people to make it grow and achieve liftoff - we just seem to lack the impetus to all push at the same time.

One possible parallel is that we have many roads/highways all essentially built and controlled by centralised authorities and this is used not only by law-abiding citizens but also by criminals. By criminals you could also include people that don't have the same ideology as the State.

Depending on the country laws you can or can't do something like FreiFunk

https://en.wikipedia.org/wiki/Freifunk

Even if countries laws permit it, your Landlord can say no. Because gathering of youth, disturbing residents, and/or businesses. Happened to me.

edit: Funny thing is, landlord/housing society is rolling out their own public wifi access points across their properties now in cooperation with the regional ISP.

You can't just replace. You need to make a better game than what is provided.

The most difficult aspect of this is it is hard to compete against free.

If you read Tim Woo's master switch you will get everything is designed to be centralised. Internet is designed to be centralised.

The bottom line is that the infrastructure (the cable and the cell towers are centralised). Until we can legally encrypt over Ham radio and quality peer reviewed open sourced post-quantum encryption algorithm Internet is centralised!

Almost all Internet users have an organization between them and the people they would communicate with. Hell, your registrar can punt you for your content, which means that it's a vanishingly small number of people, those on the dark web and a few other places, who have access to a truly decentralized internet.

The phase transition takes place when open infrastructure, bandwidth and compute capacity together reach a critical threshold; new ICs will emerge through virtual infrastructure and resource sharing.

Given that the internet relies on physical medium to travel you will sadly require actual anarchism before you can achieve digital anarchism.

Or, more likely, cooperatives. Apply the same anarchist logic to power and it breeds lethality, but I know local power coops work pretty well IME.

What if there was a simpler way?

Is the Internet itself leading to more freedom or less? Or is it human nature spilling into bits and functions that is?

>Is the Internet itself leading to more freedom or less? Or is it human nature spilling into bits and functions that is?

The funny thing is, the internet is big enough and complex enough that all of these are true.

Even the relatively restricted freedom of social media, itself just a part of the greater internet, is greater than the freedom people had to broadcast their views and communicate with a worldwide audience prior to the internet, which is almost zero.

Reply on news.ycombinator.com