Hacker News Reader

Comment by aahortwwy | Hacker News Reader

I can think of plenty of SV darlings that would take far longer than a month to recover from such an event.

That said, given the content Parler was hosting they really should've planned for something like this, so it makes sense to terminate the CEO.

View on HN

They had a backup plan for getting kicked off AWS, supposedly, but the backup plan terminated their relationship with Parler before AWS did.

I can imagine Parler also might have some problems attracting the kind of talent needed to run a site like that.

Their linkedin employees page didn’t have that many infra people which is not surprising for a stack so heavily leaning on managed infra. Just hiring/contracting that expertise in such a short time would be monumental task. Guess they’re one of the unlucky ones who learned the value of infrastructure team (the hard way)

Huh, it's almost like no provider in America is cool with your platform being used to plan an insurrection. Or open calls for violence against and murder of elected officials.

except, Twitter?

Suspect there are hosting providers in other countries.

Like Facebook, Instagram, and Twitter? Parler was a scapegoat to deflect attention from the rest of this category of software.

Facebook, Instagram, and Twitter all had moderation (whether its effective is another debate).

It's hardly an apples-to-apples co.parison with Parler.

So did Parler. And whether it was effective was another debate as you said. But upon my causal visits to Parler I did not encounter a single call for violence. It seemed like it was as exceedingly rare as anywhere else.

>whether its effective is another debate

Why?

For one thing, they have agreements between whoever provides them service, and if they have not been taken down, it means their business partners feel that they uphold their end of the bargain. Or they run their own infra so they only answer to themselves.

Parler had moderation too. The difference is that Facebook, Instagram, and Twitter's moderation lean left, and Parler's leaned right.

I feel like you may have a right-skewed view of this issue. If you look at it globally it may be different, and I don't know where you're from. Here in the United States (where all four sites are headquartered) sites like Facebook/Twitter are moderated somewhat right leaning for their role in aiding right-leaning former president Donald Trump in 2016. He lost their support later due to his abuse of power but that's a bit out of scope of a HN comment.

Parler was well outside the norm and was given amole opportunity to change.

https://arstechnica.com/tech-policy/2021/01/filing-amazon-wa...

Hard to see why it matters if they are elected officials.

Same reason it matters if they're cops.

Search for you will be more aggressive.

Your prosecution will be more aggressive.

Your punishment will be more harsh.

Because it was an attack against democracy itself. They weren’t attacking individuals, but the role those individuals hold.

Parler didn’t even have an events section, it was also fairly difficult to discover content and from what I saw very corporate-like (ie moderated well, mundane posts, etc).

Twitter or Facebook on the other hand, ive seen people call out to riot and burn down cities. Literally, there was a Facebook event to riot in my town (person who planned it was arrested). It led to millions in damages, and I feared for my life.

I was surprised parler got de-platforms because it was the most mundane option TBH.

Apart from the lobbying power of Facebook Twitter size company, it is pretty hard to deplatform them.

On a purely technical level Facebook runs their own infra. They would be hard to deplatform. The only people who are probably capable of doing that is the l3 type service providers. Even there Facebook , google own investments in undersea cables.

Also the size of their deal value to the vendor makes a huge difference. Amazon's revenue impact from parler would not even be a rounding number in their annual report. The impact of loosing fb for many vendors could kill them .

This is true.

Apple can't get rid of Facebook or Google Maps, no matter how much they hate them, but they can easily get rid of pesky startups while claiming to be standing on principles.

Not much of a moral victory there, though! (even if you believe they were right to do so, which I do not.)

Pre-Fortnite-sage I wouldn’t have been so sure, but they had 116m users at that point and were still banned. Obviously that’s still an order of magnitude smaller, but…

Also Fortnite is not utility application it lot easier to ban a game. It is much harder for say Google Maps, while we can discuss the "utility" of the Facebook app, even with facebook main, plenty of people use it as critical communication tool.

On the one hand, Google deplatformed Parler by blocking their app from the Play Store, and on the other hand it continues to be "cool with" 3rd-party content on Youtube which is "openly calling for violence against and murder of elected officials".

https://youtu.be/jRLH8E_CpP0?t=205

For the record, I'm absolutely fine with Google deciding what content stays and what content goes when it comes to their hosting infrastructure. But in so doing, they need to retain legal culpability for the content that remains.

That is not what "openly calling" means.

Fair enough. But can we at least agree that the line between the 2 examples is so fine that the monologue comedian on a satirical TV program had to run his joke by his lawyer's lawyer in order to make sure he was standing on the right side of it?

> They had a backup plan for getting kicked off AWS, supposedly.

As the saying goes, "one backup is zero backups."

is this induction, but in opposite direction?

N+1 backup is N backup

1 backup is 0 backup

"Two is one and one is none."

We have a similar saying in my country: One backup is no backup, two backups are a half backup, and three backups are a full backup!

> I can imagine Parler also might have some problems attracting the kind of talent needed to run a site like that

Naaah, plenty of fash-curious engineers out there.

Most Silicon Valley darlings run disaster recovery exercises simulating setting up a datacenter from scratch in less than 24 hours (not that Parler should be compared to any of these companies, clearly)

This. Absolutely this. We used a second location and provider.

Yeah but that usually means switching from AWS to GCP not, build a data center from scratch oh and by the way nobody from electricians to truckers will work with you because your platform was used to plan a violent attack on America.

Right, so step #1, don't plan a violent attack on America?

Broad accusations aside, don't shift the goalposts.

No one can setup basic infrastructure beyond the datacenter in 24h.

Yeah, all those liberal electricians and truckers.

They don't have to build a data center from scratch. Plenty of hosting providers (yes even "cloud" ones) in the country or outside it would take their business in an instant.

They could probably set up hosting in Russia or China though. Just at a slower latency

I think you're overestimating the ability of many "Silicon Valley darlings" to recover from what happened to Parler. First, at least these days, most Silicon Valley darlings are pretty heavily tied to whatever cloud service that hosts them, so disaster recovery exercises mean spinning up another project in a different AWS availability zone, not moving from AWS to GCP or whatever.

Granted, given Parler's mantra they should have known they would be particularly vulnerable to getting kicked off AWS, but I'm highly skeptical that many Silicon Valley darlings (and I've worked at some) would have been able to spin up something quickly on a completely new hosting platform.

Just curious... in your experience, are most "Silicon Valley darlings" also more apolitical and pragmatic than this CEO's statement seemed to indicate he is? Or do they run the gamut?

Just curious as this was my overall sense when I lived in the Bay Area and it would be consistent with a lot of the corporate "censorship" of various diverse viewpoints that's going on now.

> are most "Silicon Valley darlings" also more apolitical and pragmatic than this CEO's statement seemed to indicate he is?

Yes, but with an important caveat. Most all good business leaders are pragmatic, because if you're not there's a very good chance you won't be a good business leader.

However, Parler's entire raison d'être was as a right wing alternative to Twitter et al (you can say "free speech" alternative all you want, but it's beyond ridiculous to claim Parler's community wasn't extremely hostile to liberals/progressives on the site, to the point that I rarely if ever saw any on there), so it would be expected for him to be quite political as it aligns with the business culture of his company. His problem was that he wanted to make his stand without an adequate technological plan if he had to build everything off the major (and even not-so-major) cloud providers.

Yes, this is what you're meant to do, but there are plenty of companies that don't (including some multi-billion dollar ones).

Not entirely sure if its a comparable case, but first thing that comes to my mind is 2011 PSN outage[1]. I guess any change of that magnitude should take an unprecedent effort to resolve.

[1] https://en.wikipedia.org/wiki/2011_PlayStation_Network_outag...

They had months of warning from AWS too. "Hey guys, content calling for violence is violation of our TOS, you should make a plan to keep that off your platform", yeah yeah whatever man.

Are the permissionless decentralized tools really not prepared for any level of throughput yet?

Genuine question

Are there any tools for hosting large volumes of user generated content in a decentralized permissionless way?

Even if you have to pare down the front end and reduce reliance on some infrastructure concepts, is there any way to do it right now?

Not if you want anyone to actually use it or if you want to be able to monetize it.

Honestly they seem to have a captive audiance, who feel they are being censored. If anyone remotely mainstream could succesful transition to being hosted on tor, it could probably be them (not that tor is decentralized.)

Social media like Parler and Twitter are largely fueled by notoriety and attention though, so I don't think decentralized anonymity is a good fit for the audience.

Gab is a fork of Mastodon and it’s still running

Gab doesn't federate with the rest of the activitypub ecosystem.

Are the permissionless decentralized tools really not prepared for any level of throughput yet?

Nope. Since Web browsers don't speak any fancy decentralized protocols you either need a centralized/censored gateway like ipfs.io or everyone has to install client software which kills adoption.

Isn't parler's primary userbase a phone app (i.e. client software)?

Not any more, considering that their app was banned from iOS and Google Play.

Given that he initially represented that they had no real hard dependencies on AWS, I assume that the Mercers quickly lost patience when it became clear that they were locked into AWS, with no contingency.

Maybe the problem is that no other capable provider was willing to take them in?

In which case you'd need to moderate your content to match your providers' T&Cs, or you'd need to moderate your ambitions to match your choice of providers.

But telling your owner that you can meet their desire to allow people to breach T&Cs, and that you can scale to the size of their ambitions, means you're either lying or don't understand the problem that you're solving.

It sounds like they had prepared to move from AWS to another mega-provider like GCP or Azure. The site is too large (and probably poorly optimized) to just spin up a few DO droplets.

Going with another major scale cloud provider was definitely poor planning there

I get that everyone just wants to shit on Parler, but AFAIK they didn't host any content that wasn't present in far greater quantity on Facebook and Twitter. It would have been reasonable to expect to have access to the same platforms as those companies.

The difference here is both your examples have their own DCs ;)

However, given the predictable circumstances they might be in , they should at least expect every cloud provider to pull the plug at them, and should have gone the route of self-hosting. Instead, they have created a social media website out of Wordpress (out of all things), that performs ridiculously slow and eats up all of their maintenance budget, on AWS. It’s baffling how the right populists cannot find or hire any competent engineers; heck, on the leftist side people do far better with much less money (for example, look at Extinction Rebellion, where they manage to self-host their entire communications system: https://media.ccc.de/v/36c3-11008-server_infrastructure_for_...)

I'm baffled by their choice of WordPress. No wonder they need so many servers.

I don't understand why they should have expected everyone to deplatform them. Facebook hosts the same content in far greater quantity and has clearly demonstrated that they can't moderate effectively, but there's no bulls-eye on them over that. What's different about Parler besides the ownership?

> What's different about Parler besides the ownership?

Facebook is 233x larger, profitable, and self-hosting - they don't have to keep their cloud service provider happy.

Parler was dependent on AWS for services but didn't comply with their terms.

Depends on who you ask. I would be delighted to see a bulls-eye on Facebook for their complicity in what's been going on. I don't see them as significantly better in moral terms than Parler, and Facebook's body count for willingly collaborating with seditionists and genocide instigators is WAY WAY higher than Parler's.

Facebook is bigger, and has power.

That means that (a) they're harder to attack, and (b) they have purposes beyond ONLY fomenting insurrection and genocide. That means they'll operate in more of a gray area by intention, and will in fact try to self-moderate when the cost of fomenting insurrection and genocide risks becoming more expensive than the profit (ideological OR financial) for fomenting insurrection and genocide.

The difference between Parler and Facebook is, Facebook will take your money but then throw you under the bus if you're malicious and endangering their platform. Parler went down with the ship because it was more important to them than just taking the money.

As a proportion of the total amount of content, Parler is/was vastly more violent and hateful than either of those platforms.

And in any case, they own their own DC's.

> It would have been reasonable to expect to have access to the same platforms as those companies.

Wait, which of those companies is using the publicly offered terms of any public cloud vendor?

You're right, they don't. We don't know what Twitter's contract with AWS says, but it's likely that Twitter doesn't have to play by the same rules as their much smaller competitors.

You mean building out their own datacentres?

Obviously not. They contract with cloud providers as well.

Proper planning would have been what, coloing hundreds of servers worth millions of dollars as a DR plan? And having no guarantee that the colo provider wouldn't also deplatform them?

How much do you think their aws bill was per month for hundreds of servers? Their traffic bill alone was probably worth it already. Don’t contract this work to ibm or deloitte or something and you’ll be fine

I'm not talking about cost. I'm saying that surviving deplatforming is much more difficult than armchair orange website architects seem to acknowledge.

Yeah except you did talk about cost. Cost is not the issue here.

> surviving deplatforming is much more difficult than armchair orange website architects seem to acknowledge.

Somehow the internet is full of websites (some of which probably had much higher traffic) that did just fine though

I do agree with parent that this is an interesting political hypotheical though. There was so much media focus on Parler for a while and ultimately AWS was exonerated as the good guy in the mainstream narrative.

If Parler had self-hosted, would the potential wrath of the media just have turned to their upstream telco or whoever instead of AWS?

Colo facility or your transit provider doesn’t have a brand name it has to protect. You can look at gab, 4chan, tpb etc for counter examples. I’m actually fairly certain they would be fine on oracle, tencent or alibaba too

The Pirate Bay have managed.

lasagnaphil's comment provides some insight that proper planning would have involved not using WordPress and thus needing a fraction of the servers.

Oh please. Colo those servers and don’t publicly expose your relationship with the colo provider, spin up frontend proxies on other providers to hide your backend.

This is basic stuff, the problem with Parler is that their team seems to have no experience hosting controversial content.

I wonder if employees left. Working for Parler past January 6th is going to be radioactive on a resume.

Folks will have pretty long sabbaticals that just happened to end on January, 2021 :D

This is why you should build for bare metal.

It's fine to use k8s and containers, but don't lock yourself into Lambda, SQS, etc.

You should be able to spin up anywhere.

If Parler did that, how does that affect their time to market? Their ability to scale with growth? Maybe if they did that, they would still be operating, because they would have never hit critical mass to the point where anyone cared about Parler.

You say that like it would've been a bad thing.

No, I just think "always build for bare metal" is bad advice.

They may even have built effective moderation tools, instead of having a backlog of 26,000 requests!

That works until your colo provider tells you to take a walk. Your landlord evicts you from warehouse where you host your backup datacenter space, your ISP cuts you off. It's almost like everybody drew the line at openly planning insurrection. Maybe they could have tried not doing that?

4chan, Stormfront, Gab, and plenty of other sites are alive and well, so I don't even buy the "what if datacenters and ISPs ban us?" argument.

I think the difference is that the Parler CEO touted their unique (i.e. lack of) moderation as a feature of their website.

The sites you listed are notorious bad actors but they at least have a degree of moderation.

Plenty of sites stay up that are not liked by powers to be. White supremacist forums. Or Pirate Bay

White supremacists are well liked by the powers that be though?

Some of 'em. Depends on who you ask. It's bad for business in a global economy.

You're making a value judgment when my argument is simply about keeping your website alive in the face of powers that wish to control or censor you.

Imagine hypothetically that you have website about the plight of the Uyghurs and Amazon was a Chinese business.

Amazon's behavior revealed they are not above terminating service for businesses. I can imagine a world where they become emboldened and attempt to deny service to parties they don't like.

Perhaps you're hosted on AwS and they make an offer to acquire your business. Can they shut you off or raise your prices if you reject their offer? Can they escape antitrust scrutiny and use this as a strategy to low ball you?

I view the response to Parler as an operational test of the death star.

We have less and less power the larger these giants become.

I don't buy this argument. Saying "because they banned parler they might lowball you on an aquisition offer and deny you service if you don't accept" is a complete non-sequitur.

As a society we make value judgements all the time. I think most people were in favor of shutting down parler. Most people are not in favor of letting AWS deny you service if you don't accept their aquisition offer and the response will be different.

Alternatively, pretend you're Isis, and want to start up a website on AWS so you can plan terrorist attacks on US soil, maybe blow up Jeff's ball's in Seattle?

I don't think we should force amazon to host ISIS' terrorist planning

And what if it happens with 0.01% of users on a popular Muslim social network?

What percentage of Parler users were advocating or planning for violent protest?

Also note that this stuff happens on Facebook, Twitter, and other social networks all the time, and yet they're still online.

These are really messy gray areas, and the choices we're making will set precedents for our future.

There are so many potential responses by both the host and the service/platform. You can ask the network to deal with it (as they did with Parler), you can involve the FBI, invoke lawsuits... I don't think deplatforming is the second course of action to take when politely asking doesn't work out.

Was it even Amazon's prerogative to ask for content removal? It might be in their TOS, but what would the DOJ say?

Unless the FBI or CIA says it needs to go, I don't think it's Amazon's call. That should be our bar.

Social networks should be able to set their own rules and operate within the laws of the United States. Web hosts shouldn't meddle.

Reply on news.ycombinator.com