Local-First Web Development
localfirstweb.dev
localfirstweb.dev
Native apps have high install friction and a lot of nasty cross-platform tradeoffs. Even though there's always a cohort here (rightly) commenting on how much of a step backwards the web is for GUI app development, one has to admit that we are past the time where a truly cross-platform, install-less native app SDK can be created and cross the chasm to universal adoption. Web tech is what we have, worse is better, yada yada.
Accepting that reality, one of the implications of the web and cloud-storage is that the server component tends to be heavy. While it's convenient for users to have their data centralized in the cloud for consistency and access on multiple devices, this does incur some significant overhead in terms of account managment, authentication, and database hosting.
The thing that I find really cool about local-first web development is that it reopens the door for hobbyist and micro-ISV apps that can served cheaply as static files, where the customer pays for the compute and stores the data. There are lots of potential micro-apps, which don't require network effects or large data storage, that could be built this way and avoid incurring significant ops overhead for the developer. Obviously it runs counter to user expectations today, but I think a lot people might also appreciate useful apps that don't have $5/month subscription as table stakes just to keep the lights on. Also, it doesn't have to be a binary proposition, there could be accounts/sync functionality optional. Most of all, as a developer, it's exciting to have the ability to try out an idea, with optionality to ramp it up into a full SaaS, or scale it down without an abrupt rug-pull on the early adopters.
I think the local first ideology can enable an alternative way to sell B2B, from the bottom up.
Sell to the users directly, at a cheap price and prevent the issues that stems from enterprise sales. This usually only works for small to medium sized businesses, and very rarely large businesses. However, due to the cheap price, the budget for this software that runs locally could be kept on the discretionary account of even large-ish businesses.
Then, if the quality and value of the software is demonstrated, it should grow organically as users recommend it to other people around them.
Over time, the need for enterprise features would arise - such as Single Sign On, and directory integration, sharing of data, etc. Up the charge for these to make up for the loss of revenue early on from early cheap sales.
You mean like web apps partially or completing ignoring platform-specific appearances, conventions, or behaviors?
Let’s not pretend that native apps are the only ones that struggle to work correctly cross-platform. While core logic may be portable in web app, frequently web UIs are happy to make a one-size-fits-all interface. While that may be acceptable in a web page, it is neglectful and insulting in a web app that wants to be treated as a peer of native apps.
So while I was never opposed to your sentiment, I find this to be a strange statement:
> While that may be acceptable in a web page, it is neglectful and insulting in a web app that wants to be treated as a peer of native apps.
This frames native apps as a gold standard, which obviously is a bar that web apps can never reach. There are foundational tradeoffs here that can never be worked around. You might as well demand that all native platforms implement a common API to ease cross platform development.
The reality is that supporting multiple platforms with their native conventions and idioms is a very large undertaking that fewer and fewer developers are willing to do. You can choose to label that neglectful and insulting, but I doubt it will overcome the incentives and market forces that are inexorably pushing in a different direction. I too lament the loss of a rich native app ecosystem. But on the other hand, I'm pretty thankful the web disrupted Microsoft just as their consolidation of desktop computing was nearly complete. You win some, you lose some, but if you want to shape the future, you must clear-eyed and forward-looking.
I don't think it's just about incentives and market forces. There is an inevitable trade-off between implementing the features that people want and working on ever deeper platform integration. This trade-off exists and will always exist irrespective of financial considerations.
Some people (not you) make it sound as if putting more work into platform integration is somehow a more noble goal than building other features, and if it wasn't for those greedy, cost cutting capitalists the natural steady state of all software would be some sort of "native" ideal.
I disagree with that. As a user I don't want developers to put time into every single thing that platform providers invent in order to make their own platform more unique, however useless or even detrimental that differentiation may be for me. Platform providers don't want to be replaceable pipes because that would hurt their margins.
In many cases one size really does fit all my needs when it comes to infrastructure. In a minority of cases I do need really deep platform integration. Both goals have their legitimate place in software engineering even before considering any profit motives and financial constraints.
In many ways the web is the last great open-protocol success, and the only one I can think of push its way all the way up the presentation layer. Kind of miraculous when you think about it.
You can do unique things like offer a trial period without syncing (database wholly in browser), which allows for ridiculously quick onboarding. And then when the user signs up for a paid plan, you just hook up CouchDB and the data syncs right on over.
The sync mechanism works so damn well. It's really cool how easy it is to implement a Google Docs-like sync mechanism with conflict resolution baked in.
I'm really surprised more companies don't pick this stack.
PouchDB does have rather expensive adapters that use memory or LocalStorage. Almost twice the size as the base package (minified).
I really liked it, but the typical pattern of one-db-per-user with internal replication on the server was a bit difficult plan/orchestrate for shared data. I'm patiently waiting for the couchdb PR for document level access control.
These days I use supabase with row level security and a Vue-wrapper that can cache queries locally and update the result as the network request finishes. Works as good as pouch + couch (but naturally comes up short for queries that rely on Date.now())
Finally, in the end what you want to do with your work is to share it, meaning that "doing" your work is for a large part "sharing" your work, which is fundamentally not local-first.
Maybe I am the odd one out here.
I feel like the primary challenge of the field of software development is actually to guard simplicity. Local-first becoming a standard is not unlike SPA becoming the standard: it hurts simplicity and digs the hole we are in deeper still.
To help with this, I think any new technology proposed should advertise when it should NOT be used. In this case, I think, this disclaimer should probably say: "most of the time, for most applications, this should not be used".
I'm a programmer and artist that works with various non local-first tools (ranging from project management/task collaboration, diagramming, programming--iterating and generating code, etc) and the thing I want more than anything is the fast, tightly integrated experience you get from desktop apps. (The web tech stack is all oriented around consumer-first and not producer-first.)
We need a streamlined tech stack for desktop-first applications. Which probably involves a p2p (*ish) connection and sharing mechanism (nostr looks interesting for this), CRDTs or other patterns, and -- why is this so friggin hard -- tools, frameworks and libraries for cross-platform desktop apps (and don't say you can't retain OS-specific touchpoints when doing this).
The complexity has not necessarily something to do with consensus algo, where CRDT are not best in class, but what if your creative app is using some compute shader for some visual effect? This is not possible without WebGPU, and then still someone has to do the work of porting all of this.
Keeping 100% of the application state in exactly one place can eliminate entire universes of problems.
If you are building an app that doesn't actually need to talk to outside things, then sure. Go 100% client-side.
The dragon is having 1 foot in each swimming pool. 50/50 state spread evenly across client & server.
As far as I can tell, CRDTs are the correct solution for syncing state between parties.
In my experience there’s always a logical way to solve conflicts. And CRDTs can have any kind of conflict resolution algorithm, so it’s customizable per application. “Storing conflict resolutions”, eg. “a merge operation”, is not needed and unnecessarily complicates any CRDT.
One local DB/store for each app. Who is the correct sync?
Or is it first in first out?
As an aside, with the future as a (potential) interplanetary species, networks can't beat the speed of light, so we'll have to figure this out then!
As for sharing, that's very much application dependent: personal data could benefit from being shared across a user's device, but not with third parties.
It’s true, and that “almost” is smaller than you think! There’s a concept called temporary disability which shifts the meaning of “disability” from “some permanent crippling condition” to “unable to do X thing in a given moment”.[1]
For example, one place I regularly go without reliable Internet is… the subway! There’s no signal between stations, so if I need access to something I have to make sure it’s downloaded first.
My grocery store also has a basement with no cell reception. Imagine if the notes app with my grocery list required an Internet connection!
That is why I much prefer a web app that lets me download my data in an open and well documented format to a desktop app that locks my data into some obscure proprietary format or even into a particular device.
You are part of a very small minority. The fact is, the majority of the world can't always be online, and so can't be properly served by always-online applications.
Every time I’m in that situation (or get on a plane), I’m reminded of the value of local-first software.
It is more complicated. It works best when the user has a distinct data bucket that is exclusively theirs. A note taking app. Personal finance. But they may have multiple devices so syncing is desirable.
This use case is reasonably common and turns out to be relatively simple to do.
If you have shared data between users. If you have a significant amount of data that just requires being online to interact with the system. Then it has pretty minimal benefits over just centralizing the data and it should be avoided to minimize complexity.
USB cables or thumbdrives make that a snap, and you don't have to rely on some flaky third-party system or a network connection. It's all 100% under your control 100% of the time.
Really, social interactive shared things are the only things where it ever makes sense to be online-first. For everything else, native is way better in every possible way. Speed, security, ability to still access it in the future, etc.
It's good to have online backups in case your house burns down. But that's still second-tier to having local backups.
Even with Starlink there will still be places that stuff needs to work offline.
It heavily influenced how I work and which tools I use.
Wow. That is so incredibly wrong. People have been building local-first software twice as long as the internet has even been publicly accessible at all.
The tools to build local native apps, the UI for them, and the libraries that they can use, are all still decades ahead of the web-first tools.
I say this as someone who has spent the last 15 years building web apps. We haven't even reached parity with 1995-era desktop apps yet. We're only barely beginning to approach that level of usability and functionality now. And with radically more complexity to do so. (And worse performance.)
Building native desktop software without having any dependence on being online all the time and dependence on having high bandwidth and dependence on a good signal is worlds simpler than developing software that depends on all that extra stuff. All that stuff that is out of your control.
Many people don’t want to share all of their data and would like to keep some local.
For local-first to succeed as a concept, it would gain a lot from having a Rails equivalent that people can get started on. If that exists, promote just that and hide all the other options.
back in the day, we built a real-time sports stats app on top of rails and firebase (before it was gobbled up by google) that provided some affordances in that regard, but truly offline first--saving locally, then syncing when online--would be a great rails use case.
Unfortunately we are far from having a go to platform. The community page's primary objective is to get the word out and connect local first builders.
I guess the answer is that JS/TS has never had a stack comparable to those py/rb behemoths.
Multi-master systems - which you have if the same data is being written to by multiple devices - are complicated, and there's no one size fits all solution. The different approaches all have different trade offs.
A good place to start might be "what do I think should happen when two bits of data that were updated independently are merged?".
PouchDB is fine for trivial local first apps, but not for anything that's critical.
CouchDB, on the other hand, is a very robust app that can be used by multiple local-first apps and those apps can access and share that data, and back it up on a separate local and/or remote CouchDB.
PouchDB and CouchDB used together are a pretty sweet and solid platform to build a Cloud and/or local first on app on.
1) I gave a talk about doing local-first web development that web developers should watch or read at least: https://qbix.com/blog/2020/01/02/the-case-for-building-clien...
2) I recently learned about AbsurdSQL! It's absurdly amazing for this! https://hackaday.com/2021/08/24/sqlite-on-the-web-absurd-sql...
This is the first time I'm hearing about QBIX. Feel free to send a PR with a short description to be considered for the page.
The Sun and later Google vision is the network is the computer. For better or worse, the Sun/Google vision is ascendent now.
Also there's a lot of hobby projects that are great and push boundaries but they're also often the only option for a given toolchain/language/framework. So its a very frustrating exercise to invest in a tool only to realize its half-baked (which is fair for a hobby project!) and you need to reverse course and rebuild or extend it yourself - even for simple things.
I’m guessing it’s because you like the responsiveness of desktop apps for tools in the creative space like photoshop? Or is it something else?
Has anyone tried ElectricSQL? Looks amazing.
The ability to effectively use multi-tenant SQLite but aggregate shared data into a postgres database is amazing. And they help you with migrations etc. This is exactly the ideal setup I've been looking for!
But the main benefit to me would be that user data will be more private and protected and configurable.
This gives someone much more fine-grained control over their data and how their data is stored. I think it ultimately makes the attack vector space smaller for any client data.
https://gfodor.medium.com/rebooting-the-web-in-3d-with-websp...
Lots of comments here, and r/programming are missing the point of local-first. Without local-first, there is no reasonable way to fix the fragmented and siloed nature of data in today's products. Local-first systems rely on data living in user space (vs. product silo), creating an avenue for reusable, portable, and composable data.
Let's take movie streaming to break this down.
1. Composability: Our viewing history is siloed in Netflix, Disney, Apple TV, and Prime Video. You could access better recommendations if these platforms merged your viewing history.
2. Re-usability: Say you signup for Hulu. Since the collaborative filtering model doesn't have any information on you, it takes a while for you to get good recommendations. Even worse still, this prevents new startups from being able to compete.
3. Portability: Web2 companies have almost perfected lock-in models, and this is primarily because you can't move your data. Most people learned the hard way trying to mastodon.
/> Local-first at a high level tackles these by moving data storage to user space (not offline but local-first). Yielding newfound capabilities for:
- ML trained on all your movie-watching history
- Single source of truth for your relationships as opposed to your connections on Twitter, LinkedIn, Instagram, etc., and their specific handles. If you want to partition by platform, you can still do it.
- User-owned private keys and historical data create exciting opportunities to skip signups and re-use historical data with new applications.
- Death to spinners!
Quick responses to some of the common themes,
- Conflict-free sync is hard, but it is not complexity for the sake of complexity; with in-device/p2p conflict resolution, you can finally have a world free of spinners.
- This is not native app development where your app state is a warm cache for BE source of truth.
- Yes, it creates a world where you can use apps offline, which is always a good thing.
- No, its not a service or a stack. It is more of an ideology and can't guarantee people won't abuse it.
I just thought this was a fun list of links to put together (like it's 1994!) and TBH I was not expecting quite this degree of positive reception.
Happy to answer any questions!
The obvious answer to all the questions would be to just create normal software and people can upload/share their files if they want to. That's pretty much how all software works except for SAAS web stuff. I assume you're getting at something different, but it isn't at all clear what it is.
Local/native software has been around for many decades. Long before web software; long before the web itself even existed. What is different about this idea?
Is it some way to make web UIs actually as functional as native applications? Some format to make data from web apps as usable as data from normal applications?
I think a lot of web consumer products (and dare I say SaaS too) are these days _not_ on your own computer like they were in the 90's or early 2000s. This isn't a call to return to those days, but it is a reminder that there have been trade offs in moving our productive lives entirely to other people's computers - both in terms of technical performance, but also conceptual ownership and the like. Can we rebalance a little?
So yes, the idea is exactly _not_ new. But a lot of products are built today without considering this tried-and-tested architecture as an option. This site & list of tools is just a friendly reminder that yes, there is another way.
We have an amazing lineup prepared for you, including @pvh you can find the details here: https://twitter.com/founderYonz/status/1628846898898501632?t...
I'm really hoping WebAssembly will finally bridge the backend and the frontend data structures seamlessly, because right now we are getting to the point of keeping 2 separate data schemas, which is a huge waste of effort.
These two links have the same zip code: https://www.sqlite.org/np1queryprob.html https://engineering.fb.com/2020/03/02/data-infrastructure/me...
One of the reasons I love the local-first movement is that it puts desktop back into the center of things. I want fast, rich creative tools integrated with my OS but I want synchronization and collaboration. This is what local-first aims for, was my understanding.
Ideally "Web" (ie browser tech) steps aside for first-class desktop producer/creative-focused apps.
1. data co-located with its UI
2. works offline
3. synchronizes between clients
4. lets its users own their data
So, except for #3, exactly like classic desktop apps that store data in local files. (#3 used to be called "peer-to-peer networking").
The modern equivalent of classic desktop apps (not that they've gone away) are mobile app stores. Mobile apps are installed and run locally on a phone, even though a key part in their business model is that most frequently sync back data to their server-side.
What's technically different between these and "local-first"? Nothing.
In #4, there's only an implied contract (not even explicit) that the local-first will be not doing "bad" things (like connecting server-side and then sending data there without user permission.) There's no technical limitation that keeps the app from doing that, though.
In other words, "local-first" seems to be a somewhat meaningless buzzword or critique about a business model, not a technical design.
It's much easier to go to a URL than to download and install software, not to mention the process of maintaining the upgrade versions. With web apps, there are no dialog boxes that interrupt you when you're firing up the app to get stuff done.
So while it's not mentioned in the local-first software manifesto, I'd add that we'd want to retain the property of easy distribution of web apps, and it's not just sync between clients.
A better way to think about it is Google Docs (as a desktop app or web app really doesn't matter), and it offers real-time collaboration on a single doc between users without using a centrally located database.
The reason I bring it up is that in spite of this apparent misfit-to-topic is that it's a superb exemplar of what a local-first application can operate like. It does not sacrifice network presence and utility. I wish for more projects to work like Fossil does, only tackling different problem domains. (My personal vote is for blog or CMS.)
It will be great if someone here guides me on how I can solve this below problem.
I am part of a Enterprise company which installs a bunch of managed security softwares in employees' laptops. A couple of these security softwares can/must be used in BYODs as well, to connect to the company network.
I would like to build a application such that, whenever a employee finds that something is not working right on their laptop, they should be able to fire it up on their browser and it will access logs from these security software and does troubleshooting/debugging locally on the machine and provides the results and remediation steps to the person. This will tremendously save time instead of contacting IT team and waiting for their help.
For this usecase, I have some limitations.
1) I want to be able to solve for both managed laptops and BYODs (both Mac and Windows). So I cannot push this as a desktop application using MDM solution.
I want this to be a web-app as I can control the version of the application that they are running. So a local-first web app seems perfect in that sense.
2) I want to be able to access local file system where the logs of these security software are located. I am not a developer, but rather a infra guy, but based on what I know, a web-app cannot access the file system without user actually doing a file picker. But in this case, it is too much friction for non-technical users, especially since I want to build this solution for multiple security software which is dependent on multiple file location.
3) Moreover, I don't want to unnnecessarily reveal too much information of location of the log files for the security softwares. But if it is a local-first "web-app" and the user has to be pick the file using a dialog box, I cannot do it without revealing the location.
4) For "some" use cases, I want to be able to "securely" send meta data to a centralized server for more complex troubleshooting scenarios or to log a ticket, for example.
Sorry for the long question. May I know if anyone have any solutions for this scenario?
This is a concern, but not as much as you would think, because we are going to limit what we are going to send back to the server. Also, we have a dedicated team that will do a audit of the security/privacy implications because any company application goes live.
I am not a developer, but infact part of IT security team. And this is the reason, why I had not gone ahead with any development, despite having this idea for more than a year, because I don't have a proper solution.
* blacksmith_tb 10 minutes ago | parent | context | on: Local-First Web Development
Hmm, I would think browser sandboxing is going to make that hard. If you have local admin rights, you might be able to make a shell script to grep through the logfiles and generate an html page as its output, which you could open in the browser? I would imagine your IT / SecOps folks would not be excited about that data being sent off the device.*
There are two problems to it, which I see. I want my usecase to be solved for managed as well as BYOD devices. And these BYOD devices are managed by the IT teams of different partners we work with. And they have their own security softwares and restrictions. Firstly, we cannot convince them to install a desktop application in their company managed machine. Secondly, a desktop application has a high probability of getting blocked compared to a web-app.
Also, a web-app is in general a better user experience when compared to a desktop app, in this scenario. Because it is easier for the user to launch and use.
I know I am asking for too much. But I have not lost hope that somewhere out there, there might be a solution.
I am open to ideas.
Embed an icon to your software, pin to taskbar or the system tray. User clicks icon -> large, non-minimizable popup window appears, gets users attention -> 'generate report?' -> Yes or No button.
Much better than having the user type some URL. They'll remember the icon but not the URL. Two mouse clicks, no keyboard needed.
(S.V. reference)
also worth noting is that offline first is about way more than just working offline, yes full offline scenarios are more and more rare, but offline first apps provide reliable and instant interactivity where any user action is has a latancy of 1 to 10 ms max and no data is lost. and works as well on crappy mobile as on a high speed landline.
Or you can't convince someone, you can only have them convince themselves if they like your arguments.
Logged with Brave support and as issue on the site’s repo.
It wasn't deliberate omission but more that he is essentially the person who kicked it off and the first link is the ink and switch local-first article
You mean SQLite?
To answer your question, this is not a technical product, it is a call for action and a collection of resources.
Database as a storage and sync layer makes a lot of sense but it's hard to pull of the encryption and device provisioning at that abstraction layer.
Anyways, it depends on use case. It's not entirely an either or situation. Like Obsidian, the note app as an example (even though it doesn't have a web app version officially). It runs great locally, you manage all of that yourself the way you want. However, they also have a subscription to manage syncing for you, in which your data is hosted on their servers for multi device use.
What hasn't changed is that data management is often easier to let someone else think about, and that routing data between different client apps is often most easily and efficiently done with a single centralized intermediary.
I'm also trying to dig up this local-first GitHub client that a hacker built and demoed on YouTube
Not sure what we can fix, does https://lfw.dev also crash (redir to localfirstweb)
* Write me a Python Fabric script to control an SSH session in the following ways: ...
* "What does torchaudio.load do if num_frames exceeds the end of the file"?
* "What's a typical conn_max_age in django for conn_max_age and why?"
* "One-liner to create django secret key and use flyctl secrets set"
* "Fix this toml:"
* etc.
It's also amazing rubber duck debugging. https://en.wikipedia.org/wiki/Rubber_duck_debugging