Hacker News Reader

Comment by stupidcar | Hacker News Reader

The competitive advantages of owning both the application and the data are so high that don't see widespread BYOC ever happening without government intervention.

I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

This wouldn't be a panacea, but it at least makes it theoretically possible to write a replacement client for Google Docs that could be a drop-in replacement.

View on HN

I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

I think that is a bit much, I would much rather make reverse engineering/screen scraping/whatever for interoperability be 100% legal with zero grey area. Including a bit that says TOS/Eula/NDA/NonCompete/any contract can not give away this right.

Edit: basically let asshole companies use technical means to try to stop us, but give them no legal recourse if we manage to get the cheese out of their trap.

We can split the difference by mandating that whatever storage services are provided via "webapps" must also be provided via a plain API. Users shouldn't have their data locked behind proprietary javascript. This would create zero additional burden, as every webapp needs such an API for the front end to talk to anyway.

Maintaining a stable API surface is a burden.

That said, I would agree with such a mandate, as the costs are probably worth it.

It wouldn't have to be any more stable than what's already required to coordinate with the proprietary front end. A company would actually have an incentive to stabilize the API, to encourage cooperative use rather than migration.

(And if a company starts churning their API in bad faith, that's exactly the kind of thing courts are meant to figure out)

Stable isn't required. Just let the users use the same API the client software (JS,mobile) uses, with password or key authentication. Free market will build clients.

For security against social engineering, add some rigamarole around opting into API access.

I'd like to see cold storage providers split from app providers.

Strict enforcement policy on cold storage data warehouses. And sensible policy for the app providers, like not caching PII.

Implement and enforce at the fed level using this division in a manner that is something akin to Glass-Steagall.

Owning data should not be something providers can ever exploit.

[edit]: I also think this could be used as a means to break apart Big Tech

That seems like a stronger mandate, but I think what I proposed would actually reign in their power better than your proposed decoupling.

First, different services would still have to agree on data formats to interoperate. The ecosystem would be quite similar to windows file formats, and we've got plenty of experience seeing how that works out. It's better than what we have now, but not a panacea.

The bigger issue is that in addition to wielding data lock in, these companies wield network lock in. With storage decoupling, if I load up my own profile in a hypothetical "Facebook competitor" app and make a post, Facebook has no desire to display that post to users of its app that are my friends. However with my API proposal, competitor apps have the ability to publish directly on Facebook's site and be treated exactly the same as every other post.

Also, my proposal has a longstanding philosophical grounding that the abilities of computing should be available to all - companies shouldn't be able to insist on specific methods of usage that computationally disenfranchise users.

(post author here)

While I agree that commercial incentives are critical to consider, two caveats I'd add:

1) I think the nature and depth of the incentives problem varies a lot by industry. Social media is a really complex case, for example. But I'm most interested in collaborative productivity tools, where I think many companies are okay being incentivized to build the best product rather than build data moats.

2) Cutthroat commercial incentives aren't the only thing that got us here. There are also tech barriers.

If I was starting a Google Docs competitor today, even if I wanted to make it open, I think it would be hard to pull off. What kind of API would I expose to enable realtime editing with good offline mode and conflict resolution? How would I deal with clients that have slightly different rich text representations than my own?

In an alternate universe where we already had a user-owned "web filesystem" with good answers to these questions, I could just hook up my client to that existing system and not even worry about persistence at all. It needs to be _easier_ for devs to build the right thing for users, not harder. And it needs to be a _better_ experience for end users, not a sacrifice. The convenient thing will win.

Hey Geoffrey! We are building this in https://braid.org: a standard protocol and API for synchronizing state with realtime editing, conflict resolution, and a good offline mode. By building this into the web, we're creating the equivalent of a "web filesystem" as you put it, where every file (aka "resource") has full versioning, realtime updates, offline abilities, and merge resolution.

I know you're aware of Braid, but I'm not sure you see how much our work is actually aligned. In fact my deepest personal motivation in creating Braid is to enable the "BYOC" vision — but I've been calling it a "separation of UI from state", where each user can choose their own interface to interact with the world. In today's web, the data owner controls the interface. But this means that they control how way too many people interact with the world, because we increasingly have to interact with the world through computers.

The Braid abstraction makes it easier for developers to program with distributed state, and in the process also makes user-interfaces and back-ends interoperable, so that we can easily switch out different UIs for the same state.

Well it can happen if a significant enough open source project establishes a protocol early enough and doesn't get co-opted.

That or a consortium of companies agree on some sort of standard. (See the work on pc buses in the early pc days)

But you are right. When companies run under trying to get as many users locked in as possible during a paid for investors period. This model is precisely the opposite outcome expected as the whole point is to build a moat around the sales model.

For something to get traction, it has to have excellent UI and UX. So far open source hasn't been very successful at providing that since there is no economic model to finance the immense amount of work required for good UX.

GitLab? Xfce?

That view seems a little shoehorned because the parent comment specified "protocol", implying that it would be the connection between client and storage, thus not user-facing.

Regardless,

* Blender

* Godot

* Krita

* GitLab

* Olive

* Firefox

* Thunderbird

* NextCloud

* Amarok

* Vital

* and more

The problem isn't an "economic model". The problem is that most open-source projects simply don't attract UI designers, UX researchers, and users because contribution flows are more technical or geared toward developers.

UX research isn't "expensive" when referring to open-source projects because it predominantly requires people: designers, researchers, and users. The whole foundation of open-source is volunteerism.

This view is the one that feels shoehorned. "Normies" (not pejorative) have no idea what a protocol is. They just want something that works and is convenient as possible. That's how we ended up at the $free+$surveillance App Store / modern Web model.

Since when was HN a normie site?

HN builds products for "normies".

What IS the contribution flow for a designer, anyway? If they can't submit a PR, it seems like the best they can do is submit an issue with a mockup. Are volunteer developers likely to jump on that and implement it?

Most developers I've worked with don't exactly love going back and forth with designers on UI stuff.

Volunteerism is much rarer than paid work, and paid work prefers proprietary too get the money to pay.

And volunteerism was much more popular in the 1990s when it was easy to write software by hard to sell it. App stores suffocated open source by offering a path to revenue.

I posited a similar point to a former colleague (mine was more about eventual market dominance due to excellent UI+UX), and he came back with Craigslist as a perfect example of a product that gained a massive user base well into an era where its UI felt very outdated. A few years after this exchange, I feel more like I was right about my point: FB Marketplace has almost completely taken over as far as a classified product marketplace, but the point still stands that Craigslist was able to do a ton with very lackluster UI+UX

Another counter example: amazon. By far the worst userinterface of any webbshop, no filters, hardly any categories a search were very small changes in spelling/wording make huge changes in results. And still they are by far the biggest game in town.

I think the problem with this analysis of Craigslist vs. Facebook is twofold. One, Craigslist had, in fact, a great UI. It was simple, clean and accessible to everyone, no matter whether you were using the newest desktop browser, a screen reader, a cheap phone or a potato. Craigslist is a success story of bullshit-free UI.

It got beaten by FB Marketplace because Facebook has almost three billion captive users, and a marketplace bundled in the service they use many times a month takes less effort to visit than a completely separate site.

From this, and many other cases - like e.g. everyone hating the IM/videoconference system they use and yet using them anyway, I conclude two rules of thumb:

- UI that's good for users has very little to do with what UI/UX specialists peddle today;

- Network effects trump UIs - a sticky service with bad UI will beat a competitor that offers much better UI, but doesn't hold the user captive.

Craigslist also got beaten because they fought becoming a unified marketplace--with lawyers.

Had they not fiercely destroyed all the various "Search all Craiglist" mechanisms, they might have had a chance at fighting FB Marketplace.

UX is a trojan horse, now everything has to be mobile friendly for absolutely no bloody reason. You know what also works on mobile? Zooming, yes you heard me.

Just the other day I had someone, an old person, complain about how a "revamp" of an internet banking site. They rewrote their functional, working UI, with a site that was designed by "UX designers", and you know how he phrased his opinion of the new site? "They made the site for stupid people".

One of the better examples still seeing widespread mainstream use is email, and this even applies to multiple stages throughout the email ecosystem: sending mail, receiving mail, managing mailboxes. Imagine if email wasn't cross provider compatible

The surviving open protocols are like libraries: they were created before the space got commercialized. They got grandfathered in. If libraries didn't exist for centuries already, there is no way they could have been created under current intellectual property laws and IP economy. Similarly, new open protocols don't generally gain widespread adoption - and in the rare case they do, that's only because some companies are using them to gain adoption; once those companies gain the market share they want, they switch to proprietary protocols and the open one dies. See e.g. Google and XMPP or RSS.

What about, tangentially, cellphone standards or HTML standard that have iterated and are now at version X? Your point stands and a simultaneous coevolution of the standard, the userbase, and the (app/thing) functionality does seem possible.

Great points -- w caveat that RSS isn't dead! :)

Technically. Neither is XMPP :). It's just increasingly rare to see in the wild.

Email is only useful because it is cross platform compatible. Otherwise, we’d all be back to writing to some friends in AOL, others in MSN, maybe some back in Compuserve. Some people like to send messages only in Facebook and others via Twitter DMs. None of these are/were interoperable which is why they aren’t ubiquitously used.

Email is only useful because each server can send and receive with other servers.

But we all know this, so what the point here?

I would say that my point is that if interoperability becomes more useful than the walled garden approach, then we will see something like this posts call for BYO client. Email is an example. We have interoperable email because it is was more useful than service specific inboxes.

Until interoperability is necessary for Word, we won’t see an opening for other Word “clients”.

Aside: you could argue that Google built Gmail to make a dent in the ubiquity that was Exchange for corporate messaging. It exploited the openness/interoperability of email in order to do this. If you accept that argument, then maybe they should take a page from that playbook and work to make Google Docs as open as possible. Allow for more interaction with external clients. Let a more open ecosystem develop. That might have the potential to diminish the position of Office for document creation.

Or maybe email is just so old, the owners got established before it could be commercialized... it might happen again, but I’m not hopeful. For example, I don’t expect Twitter to care about federating their system with other messaging applications. There is too much to be gained (selling ads) by keeping people on their site.

Gmail was Embrace, Extend, Extinguish. You can't read your own email using you own email client anymore.

Absolutely, however a major problem occurs, and email is a great example of this. When there is a fundamental need to change the protocol for reasons of privacy, security, authentication, etc. It becomes almost impossible to do when it is as wide spread as email, or it takes a very very long time to spread enough.

When one person controls the whole system, it's easy to pivot rapidly. I guess that's another argument against having widely used protocols (even though I disagree with it.)

I think that's a great argument for keeping widely used protocols. I don't want any one person, or one entity, to rapidly pivot my critical infrastructure (whether it is "broken" or not). The slowness of change is a feature.

What's insecure or frozen about email? We have SPF and all that jazz.

> without government intervention

Well that's the real trick, isn't it? </Han Solo>

Our government is "captured" by campaign financing. Now we have 2 problems: ineffective governmental oversight, AND Citizens United. We have to solve the latter before we can ever solve the former.

We need NPT for software

https://en.m.wikipedia.org/wiki/National_pipe_thread

I don't know if this is a good example :) Anecdote: I recently wanted to extend the hoses for my tabletop dishwasher (the one that hooks up to your tap, not to the wall). It's 2 hoses with 2 connectors each, should be easy. But all 4 are different. After 3 trips to 3 different hardware stores, I found exactly 1 fitting that fits 1 (of the 4) connectors. I ended up slicing the existing hose and extending it in the middle, because that was easier than finding the right fittings. I thought about ordering them online, but that would assume I successfully identified them, which I didn't. Granted, I don't know anything about plumbing, but neither do the average person know anything about technology.

That said, I agree, standards are necessary, I just wanted to rant about NPT.

I genuinely cannot tell you how relieved I am this isn’t a link to the Treaty on the Non-Proliferation of Nuclear Weapons, the thought of FAANG cos “disarming” but snuffing out competitors with government approval is terrifying.

https://datatransferproject.dev/ from yesterday’s iCloud photos thread seems to be a step in this NPT direction though.

I think it's not completely inconceivable that this could be regulated some day with music streaming. There are zero technical hurdles (unlike with complex apps like Google docs), and because there is such potential for decoupling the client the consumer benefits would likely be much greater.

I've been meaning to switch from Apple Music to Spotify for a while now (I'm on Android). I just had a look around the Spotify app today and the interface is terrible, it won't let me play and view my library like I want to. It would be so much easier if I could just use both from a single open source client, and then I wouldn't have to switch off to soundcloud and bandcamp for certain artists.

Wouldn't it be in the interests of players like Google and Apple to create this service/protocol, even before the regulatory net starts closing in?

You should check out Spot[0]. It's a Spotify client written in Rust using GTK. It's quite barebones at the moment and only works with premium accounts, but the interface is much cleaner than the official Spotify client.

[0] https://github.com/xou816/spot

If a terminal is more your style you can even use Spotify-tui[0].

Requires some setup and having an additional client installed for the actual audio playback but it's still interesting using Spotify from a terminal.

[0] https://github.com/Rigellute/spotify-tui

remoteStorage[1] is already a thing. I don't think anyone's made a full Word-style document editor for it yet, though.

[1]: https://remotestorage.io

I generally support the spirit of requiring cloud apps to provide equivalently powerful API access to their service. The (solvable) problem is how do you regulate this without stifling development of new cloud apps?

Another more practical problem is how do you even get something like this on the agenda of a lawmaker? This isn’t a readily apparent problem to most people but it likely would result in less monopolistic outcomes for cloud providers and potentially better and/or more efficient products.

I think the focus here is on the wrong thing, the "app", which makes ideas like legal intervention and other distractions feel relevant.

I think client interchangeability and ultimately user freedom is fundamentally about data driven design, look at all the examples: text is text is text... git is a file based graph of merkle trees. They are both highly abstract, portable, unopinionated data structures that don't describe APIs, UI or implementation - don't like git CLI? you can literally write a completely different one with different porcelain and use the exact same git repos as everyone else, people have done this. Text is the same, it's just more obvious because so many examples exist.

If you start with data and make it successful clients will emerge around it without the choice of making it interchangable.

The problem with things like google docs isn't the app, it's the opaque data format. And then there are formats posing as being open and portable like DOCX which is essentially a proprietary microsoft format forced open due to reverse engineering - all of it's details will no doubt be closely coupled to the history of MS Word features and implementation.

Exactly. But in this case that opaque data format is exactly the thing that enables live editing. Gdocs runs on Operational Transforms. There is basically no way to do OT without a ton of back end support which means tightly coupled APIs.

Martin Kleppmann has a great talk on CRDTs for distributed text editing. It is a hard problem. You also need a dedicated binary format (basically a table) but at least in this case it is a lot easier to use an open format with federated clients.

Separation of application and storage is a great idea!

How about web search and social news feeds? Can there be a similar frontend / back-end separation for Google, FB and Twitter? I am thinking - different search front ends with different UIs, rankings and filters. We could have an EFF-Google and a Conservative-Google. Users could pick their preferred flavor, as I don't think it's fair for Google to decide for us all.

We just have to do what the phone manufacturers did for the mobile phone connectivity. GSM was a huge hit thanks to interoperability.

It was but then deteriorated into a big patent war with every company trying to shove its proprietary patents into the next cellular standard.

So while everyone can implement the standard, very few can do this profitably (In practice, Qualcomm, Mediatek and to some extent Samsung)

OTOH, give it time. Open alternatives exist for every private walled garden. I believe the decentralization/federalization and FLOSS movements have a real chance, even if they move at 0.05 velocity. Private platforms are subject to closure, but the light of an open source project will shine as long as there's someone spending time on it.

In due time we'll have networks of small providers catering for BYOC hosting. Cryptocurrencies can play a part in efficient and fair compensation.

But then again, maybe by the time we catch up to current date, everything happens in walled VR moats.

Maybe I'll be the oddball in the metaphorical cabin in the woods, but given the progress we're making so far I'll refuse to enter a closed VR/AR ecosystem.

Maybe the best solution would be legislated government donations to open source projects, with a mandatory legislative bill each year on who they go to.

> with a mandatory legislative bill each year on who they go to.

Another option would be something similar to farm subsidies, although I don't know if that would be enough.

There's been some of that already, not in the sense of governments forcing the separation, but in the sense of governments making the keeping of data a liability, e.g. with the EU's GDPR.

There's certainly still a whole class of applications when owning the data is a competitive advantage, but there are also lots of organisation whose core competency is not selling data, even though they do have to work with user data. When storing data yourself is a risk, it becomes a lot more attractive to just let the user store that data somewhere themselves and have them give your application access to it.

I'm sure this one of the reasons we're seeing a lot more interest from Europe than elsewhere in Solid [1].

Then of course, there are also lots of organisations struggling to maintain correct and up-to-date data. If multiple organisations access the same data as controlled by the user, the user is more likely to have kept that data up-to-date.

(Disclaimer: views are my own.)

[1] https://solidproject.org/

From a European perspective, it depends. I wouldn't ever even consider something like Auth0 to store our user records for example, due to the potential legal fallout of storing all our PII on American servers. You have to be really careful just what kind of storage provider you pick, where their servers are located and which specific regulation your business falls under.

That being said, I'm a huge fan of splitting concerns - such as connecting anything and everything via OAuth delegations with narrowly scoped permissions!

> The competitive advantages of owning both the application and the data are so high

Honestly this is wrong. Google drive has API for everything, Notion is working on an API, Confluence has API for everything, Trello ... Any serious text editor (and all the ones wrongly cited in the article) software has API for everything. Want to export your Confluence data to Notion ? you can. In the documentation tools space owning the data is not something looked for, because company pay for the service so the data belong to them

BYOC can exists today, the only reason it doesn't. Is that the documents format is super complex and require complex editors. But the format themselves are not secret, the application is (and not even always, for example the Confluence editor is open source).

Those APIs are not meant for building a full blown dedicated clients, they’re for application integrations.

Check that https://developers.google.com/docs/api/reference/rest/v1/doc...

You have the full specification of Google docs documents and API to edit any kind of blocks inside

First, that’s not the full spec for Google docs. It’s missing many aspects of the environment.

Second, is that meant for building a full blown client or is it meant for 3rd party integrations, like plugins?

> It’s missing many aspects of the environment.

I'm curious of what is missing

> Second, is that meant for building a full blown client or is it meant for 3rd party integrations, like plugins?

It is certainly not for plugins, as there libraries to call that API from python, ruby and others which are then not meant to be run in the browser.

My point, is that if you want to build an editor of Google Docs, then you can do it, there is an API to get the doc format and an api to make modification to that document

> I'm curious of what is missing

My initial response was related to missing comments and collaborative real-time editing in the v1 link you provided. However, I searched a bit more and found that their newer API versions include both of those features.[1]

Here's a blog that's announcing their real-time collaborative editing[2] back in 2013, so that's been there for a while.

That blog post also lists a few products that make use of the real-time feature but the only client that's an actual editor ("Neutron Drive") has since died with its domains no longer working.

I guess this is one of the rare cases where you can build a client based on their API and Google actually promoted one of them but I'd still say that it's not really meant for it.

The primary use case for these APIs are bot services like Zapier and integrating a "Save to cloud folder" feature in your own app.

Even if they actively encouraged you to build a client based on their API, who would risk it? We've seen how Twitter handled 3rd party clients after it got big and that's likely going to be the case with every major player.

[1] https://developers.google.com/drive/api/v3/manage-comments

[2] https://developers.googleblog.com/2013/03/build-collaborativ...

You touch on an important point here. Even if you were able to gain full access to the data behind say Confluence, the schema is probably very specialized to their client, contains lots of tech debt, and design is probably very opinionated. You would most likely find it easier to just write your own schema rather than integrate with theirs.

Even better: They have to use the storage provider of your choosing.

> The competitive advantages of owning both the application and the data are so high that don't see widespread BYOC ever happening without government intervention.

This.

Interesting proposal, but I see it greatly reducing development and also further entrenching the cloud providers as Lords Of Everything.

I don't think that's true.

In the areas in which "bring your own client" is the standard ("text editors / IDE, RSS readers, email clients, web browsers" to cite the article), there is a seemingly greater diversity in hosting than when it comes to the backends of the services mentioned.

Yes, many of these _may_ use AWS in the end, but there is a great many of them that don't.

Can I ask why?

As I see it, cloud providers are currently entrenched as Lords of Everything because they can both provide the infrastructure and the products that run on top of it. Making them act more like utility providers would reduce their power, I think.

Think it through:

> enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider.

Now, is the law going to mandate the exact API as well?

The likely implementation of this is that, just as every app developer copies every other app developer, when choosing which User Storage Backend to integrate with, they will pick the most popular one, or a near competitor. AWS, Azure, or Google Cloud. (Apple-focused developers may choose Apple Cloud; I'd expect Facebook to spin up one if this became law too).

Just as you don't get general-purpose OAuth integration so much as "log in with Facebook / Google" buttons. Those are your two choices.

Yes, the original intent would be targeted at Google and Microsoft, which currently own both big web apps and big cloud platforms to run them on. I'm not convinced that splitting them vertically would stick; the convergence effects are very strong. So you end up with (choice of two office suites) x (choice of two backend providers), big deal.

Is it sufficient that Google Cloud Storage would have a separate stock ticker from Google Cloud Apps?

It reminds me of rail privatization and the nonsense of having thin shell companies run the trains while leasing all the rolling stock from a couple of companies and running on tracks owned by exactly one company. It didn't really expand choice and it provided plenty of opportunity for blame deflection.

Your objections seem to be predicated on the assumption that the attempt to separate storage providers from application providers wouldn't be properly enforced. Yes, obviously if Alphabet can just set up Google Cloud Storage and Google Cloud Apps and continue with business as usual, then this won't work.

But what I'm suggesting would be a functioning regulatory regime: An independent authority who classifies cloud providers as utilities, writes regulations specifying standards of interoperability, hears complaints from people and businesses and acts to enforce them. Antitrust regulation that stops collusion between service and application providers, and prevents companies or individuals owning a controlling stake in both.

And the major difference I see in contrast to railways is digital infrastructure is not limited in they same way physical infrastructure is. So long as there's only one set of tracks, you can never have a functioning market running trains on them, and it's impossible to build a second set. But while the capital costs of setting up a new data centre are large, they are still within the realm of possibility for many large companies.

Any such regulatory regime would serve primarily to entrench incumbents by creating vast compliance requirements on newcomers. How's your <pick random> healthcare startup doing?

You imagine a benevolent dictator making optimum choices. I think the F35 design/procurement process is a better metaphor.

With storage we already have "S3-compatible" as a widely implemented and supported standard. Just allow the user to set an arbitrary API endpoint for your regular S3 library and you support a breath of providers

don't see widespread BYOC ever happening without government intervention

This is why people don't like nerds, it's like you actually want to fuck up this industry. The number of users that care about this is a rounding error. What you're talking about is increasing the complexity and flakiness of every product through modularization. Have you ever actually shipped a product for the general public?

>I'd like to see an enforced separation in law between commercial application providers and storage providers. So, if someone writes an app like Google Docs, they can't just store the data opaquely on their own cloud servers. They legally have to integrate with a separate storage provider. And that storage provider has an obligation to make my data accessible and manageable by me.

Same can be said about gmail. In the old days, one can access gmail via imap/pop3. I believe that was removed. All email ought to provide an alternative way of getting the email other than company's web interface.

I'm still using imap and smtp, unless k9 mail for Android is abstracting something else away?

Why do you believe that? https://support.google.com/mail/answer/7126229?hl=en

Probably because as described in the linked article IMAP & SMTP is disabled by default - though if you're willing to ignore the scary warnings they're still possible to use.

> In the old days, one can access gmail via imap/pop3

I believe it's still possible.

Also IIRC it required some adjustments from mail clients to the "imap as gmail sees it" protocol, like "one message being visible in several folders", since "folders" in imap sense were mapped to "labels" in gmail

Gmail still allows imap access, although it requires an app-specific password. (Or it has to be allowed on the enterprise side with G-Suite)

Reply on news.ycombinator.com