What if we had Local-First Software?
adlrocha.substack.com
adlrocha.substack.com
Having all of the data local is a superpower. It changes how you build the app, and you just never have to worry about the network. Everything is fast _by default_. It's great.
The app uses CRDTs to sync and it's unlocked a lot of powerful stuff like a robust undo system. We also offer end-to-end encryption.
On the negative side, over time I've stepped back a little from being truly local-first. The mobile app used to connect to the desktop by pointing the phone's camera at a QR code on desktop, and it would literally sync peer-to-peer. Maintaining that was nightmare and users had endless networking problems. Our network infrastructure unfortunately is not built for truly peer-to-peer apps. Maybe ipv6 will help, I don't know. My app now has a "centralized" server but it's really just a backup that gives some convenience - the app is still totally local but syncs through a single server. You can work on it for weeks offline and then sync up. In the future when p2p is ready, I'll be ready for it.
Does Actual expose any api stuff?
* A custom rules engine (to be released in a few weeks). You'll be able to write a list of conditions for matching transactions, and a list of actions to apply when matched. The system will automatically encode what it learns as rules (apply category X to payee Y) so you can see what it's doing and adjust.
* Multiple budget types. Zero-based budgeting is cool, but Actual is just a tool for you. If you want, you can use a simple report based that just shows income vs expense. You choose the type you want adapt it to your lifestyle.
* Custom reports. This is really where having all your data local is incredible. You'll be able to write queries into your data, process it, and render any kind of visualization.
Now that sounds very cool.
Out of curiosity, do you have any plans to handle things that cannot be local first (for example with YNAB I like the Plaid Integration to pull transactions data out of banking).
Something like that for your application would need to clear through your own API right?
It's tough. I've wrestled with this for over a year. Bank syncing has to come from my server, which means by doing so you are giving us access to your transactions. No banking provider has any way for clients to contact them directly. I think this is possible with the right encryption, but nobody is working on it. I've given this feedback to Plaid but they don't care.
The majority of people need the convenience of syncing though. So I plan to launch it, and it'll go through my server, and I plan to communicate clearly the privacy that you are giving up by doing so.
I keep hoping to find a (maintained/viable) open source project that uses headless browsing to login locally and extract transaction details.
For my credit union, I "reverse engineered" the API and export multiple times throughout the day.
I wrote an extension that exports the data for CapitalOne, but I haven't gotten around to trying it either headless or even just in any automated fashion.
Easy automated export of user data, even beyond financials, is something I'd like to see more of. Feels like it could be a workaround while there's so little decentralization.
There _sort of_ is a solution where your computer directly connects to the bank. The format is called OFX (https://github.com/libofx/libofx), and there is a directory of banks that provide these files directly online. This site (https://www.ofxhome.com/) lists the URLs to use for each bank.
I used that for a while many years ago. But it's terrible and requires massive maintenance. For an app that requires connections to arbitrary banks, there's no way developers can support these direct downloads. There's always different errors in data for different banks, etc. Unfortunately, Plaid is solving a real problem
What causes these problems? NAT?
For JavaScript, see: https://github.com/feross/simple-peer
It's just a mess of documents referring to other documents for details.
I'm sure it all makes sense if you just spend a few months making sense of it all, but it's very not much accessible to someone looking do it as a weekend project.
Not understanding the byte protocol is understandable but a quite different topic. You really don't need to understand it, there are libraries for many programming languages doing the hard work for you. And if you want to understand it, it's normal you have to read a lot of different documents. If you would have no understanding of HTTP/2 you would have to read a lot of RFCs too as every version is built on the former version with retaining many of it's concepts
That was another person, I don't struggle with that.
> If you would have no understanding of HTTP/2 you would have to read a lot of RFCs too as every version is built on the former version with retaining many of it's concepts
The turd doesn't smell less because you managed to find another.
The reason I wanted to read about it was because some sources say STUN requires two public IPV4. I wanted to know what would happen if I had only one. Or if I had two IPV6.
GP, I'm curious about the frontend stack as well.
The frontend for desktop/browser React, and Electron is used to create standalone apps. I don't love Electron, and if you don't want it a browser version is available too: https://app.actualbudget.com/
The mobile app uses React Native and has a custom UI just for mobile, but it shares styles and a few small components with web. I don't believe that you can build one UI that works well on both desktop and mobile.
All platforms share the backend code which runs in a separate process/thread. On mobile, I use https://github.com/JaneaSystems/nodejs-mobile to run the JS backend locally.
That's mostly it. No UI frameworks are used. I love UI/UX design so I built it from the ground up to be as usable and performant as possible.
I'm doing a similar quasi-local-first project with p2p. Ive been toying with libp2p[1], which underpins a number of distributed storage solutions, and originated from ipfs.
> [1]: https://libp2p.io
As much as it's easy to shit on the web stack, it solved a real problem, namely the 'operating systems underneath' being absolute clusterfucks at least as much as the web. It's really sad that the only way to reliably access something from multiple devices and locations are websites.
Ironically, if we're ever going to see something like the OP wants, it's 100% going to be locally-run webapps. Whether we like it or not, think it's right or not, it's clear by now that the average joe is not going to learn tools like git.
It is a simple wrapper around the input devices like mouse, keyboard and touch screen and outputs the graphics using GPU shaders (DirectX, OpenGL, Metal). It feels like a mix between a 2D game engine and a UI library.
The immediate mode architecture is a mindfuck how easy it is compared to (functional) reactive programming. You basically get all the benefits of having observers of values with just values.
All this is totally fine for a game UI, and is a much harder sell for productivity software.
Dealing with something that believed it only needs a canvas to draw on is PITA.
100% will require accessibility in some form if your timeframe is long enough. We aren't young forever, you know.
Is it? Web and electron applications have the same issue, and they seem to be doing fine.
I think it's neat that folks can skip the conventional web technologies & hypertext & use the web to create arbitrary systems. But I also think this is quite a malignant & user-de-empowering & tragic development for the web. The DOM has been extremely powerful as a common language, that users can make use of directly & via extensions & scrapers. Stealing this fire back from the users, renegging on the web's promise that it's the user's agent, for them to use to navigate the web as they like: that's a great & sad regression.
In my opinion the web is worthwhile & useful. Bypassing it, treating the web like thin-client, a way to push pixels in people's face, via some auxiliary rendering process, be it local (webassembly) or remote, is a terrible usurpation of user's agency. Projects like Flutter's CanvasKit & others are a great danger to one of the few piece of technology that has enabled users some level of control & freedom, the web.
The article doesn't mention anything at the OS level. Every technology suggested runs in the browser.
While I'm certainly sold, personally, on local-first software (obviously), I think it's going to become an increasingly harder sell for Normal People. 15 years ago, a large percentage of households had at least one desktop or laptop, as it was the only way to get on to the internet.
As smartphones have taken that place for most people, nowadays if you've got a desktop, it's likely only for gaming, and most commonly-purchased laptops (esp that don't come with Windows) have stagnated in performance and capability, as their companies push people to use their cloud service offerings. NAS devices seem to remain niche products. This drove me to have to support everything: Windows, macOS, Linux desktops, and everything that can run Docker. I've been very surprised to find an almost perfect split of users for each edition (desktop vs docker).
Given the popularity of personal-information-selling social platforms (however it may be waning), I don't think most people are concerned about privacy to give up (almost any) convenience. I'd love to be wrong. I'm hoping to make something easy enough to install and live with that it isn't an inconvenience, but it's hard to compete against enjoying someone else's infrastructure and application management teams, for "free" (where "free" === all of your personal information and telemetry).
(For what it's worth, PhotoStructure only runs on hardware you own, and none of your data leaves your computer, except for error reporting, and even that can be disabled).
Know that the free tier will still open and browse existing libraries: I'm not going to hold anyone's library hostage in exchange for a subscription.
My desktop is always network connected, and my internet connection at home is fairly reliable. My smartphone is often in areas with no coverage. There are tons of apps I can't use because they don't keep local data. It would be a perfect time to read and write email, but the platforms that would let you do that are dead.
The default mail app on iOS is perfectly capable of locally writing and downloading/reading email. Can't search all of history since that's in the cloud but for recent usage it's fine.
Did people not grow up on Juno Email services?
Oh wow. An entire generation of modern computer users never had the opportunity, did they? You'd connect to the internet to download your email, then you disconnected (otherwise, you'd use up too much of your phone line and get a big phone bill).
The POP3 protocol still works. You can still get Mozilla Thunderbird and grab POP3-style emails from GMail or whatever. But people want IMAP instead, an always-connected model provides better idea of which emails were "read" or "not read", for example.
Ex: You may download all emails on your Phone, but maybe Email#0 is never opened on your phone. If you used POP3, you don't know if Email#0 was read or not, the protocol just assumed you read it. POP3 was more useful back then, when you only had one computer of any sort and didn't have to worry about keeping multiple devices in sync.
-------
How did forums work back then? Through USENET. A similar concept, where you'd download the messages, then disconnect from the internet as you read all the updates.
BBS machines however were online-only, and the start of real-time collaboration IIRC (not that I ever used BBS, but that's my understanding of that old telnet technology). I don't know the history: whether BBS or IRC came first (or which was popular first...)
The "Killer App" for the internet, AOL Email / Juno Email / etc. etc. was offline-first, by and large. At least, to me (a good ol' "Eternal September" user who joined the internet community well after 1993).
--------
Anyway, you'd read and write emails while offline. Then you'd hit "send", which would boot up the modem, take over your phone line (woops, Mom's talking on the phone. Sorry mom!!). Wait for Mom to get off the phone, THEN you connect to the internet...
And then send the whole batch all at once.
You can still do it if you want to.
fetchmail to handle the mail retrieval. Add notmuch if you want fancy indexing. Any local client to compose. Delivering is a bit annoying to setup initially in my experience.
Ah, those were the days. No endless distractions of always being online :)
Well, IMAP isn't always-online, it supports it but you can just as easily use it, download the messages, and disconnect. With IMAP you do get push notifications for email instead of polling, though, which is nice.
Have any of y'all ever heard of something called "Adobe Creative Cloud", it's...
It is literally trying to answer most of these questions for a set of titanic applications that predate the entire world wide web, it solves, offhand, at least five of these seven principles - it's (1) built around apps that have been working with local files since the eighties, (2) can sync your work between multiple devices, I just carry my laptop everywhere so I dunno how well it works, haven't played with the iPad apps yet so I dunno about those, (3) works just fine with no net access (though it'll cut you off if you don't hit the authorization servers every few weeks), (4) I am not sure how it solves the conflicting changes problem as I work solo, (5) your data's on your local device and in Adobe's cloud for as long as you subscribe, (6) okay well stuff's probably stored unencrypted, unless "I turned on whole disc encryption" counts (7) it's your own files and your own backup strategy.
That's five checkboxes out of seven, better than the 4/7 that "email shit around" and "github" get.
And it is relentlessly uncool. But it has been here for a good while and none of y'all have noticed it because it's built for artists.
I don't want to pay a subscription service to Adobe any more than I want to pay a subscription service for my desk and chair.
More power to you! But this article is specifically about local-first/offline SaaS, which is what Creative Cloud is.
It would be great if we could see more software sold as-is, without updates. But, especially in the world of online-capable tools (which not everything is, but most apps can access the internet), ongoing security maintenance is important. I'm not sure we can have it both ways.
Adobe managed to stay in existence and be profitable for a few decades before switching to software rentals.
What changed in the creative space that caused 'traditional' software sales to no longer be able to pay the bills? And why doesn't that also apply to, say, Capture One?
Businesses often prefer operational expenses over capital expenses. What changed is that your average business IT (bigcorp or mom's basement) now has reliable access to the internet, which means that an operational expense for software has just recently become possible at all.
Capture One has subscription and license:
* https://www.captureone.com/en/products-plans/single-user/cap...
If I had a need for, say, cloud/sync features I understand a monthly cost as servers and bandwidth costs money; totally on-board. But for people who do not need/want cloud-y stuff, what re-occurring cost is there?
I was happy to purchase YNAB4 for budgeting, and would have purchased YNAB5—but it's subscription and I have no need for the sync stuff that's part of the 'deal'.
This is like IT infrastructure moving to cloud: businesses decide that the problems of maintaining IT infrastructure 100% in house is not worth it when you can outsource a big chunk of the business problem for a known cost. In my opinion all of these are examples of the same trend that also includes the direction that k8s, serverless, even companies like uber, doordash, wework etc are moving towards: a global relaxation of artificially maintained buffers down to solving problems continuously, automatically, and on-demand. (Since it begs the question, my critique of uber, et al. is that they're trying to be too big for their problem.)
I think this is the general trend, but I don't know why. My guess is that it is vastly more efficient; not more efficient in cost, but more efficient in time.
From the perspective of their business, it's advantageous to have the consistency of $X / month forever instead of "hopefully a huge yearly sales month, followed by mostly nothing until next year". Businesses like predictable revenue.
Then make the subscription worth something over and above the software:
* https://www.captureone.com/en/products-plans/single-user/cap...
Subscription models are about forcing upgrades in a world where users increasingly aren't actually benefitting from them.
This way you can choose to receive updates indefinitely and support development with your subscription, or buy/stay with a version you are using if you do not need/want the newer ones. (Buying an annual subscription grants you the perpetual licence immediately apparently, I have no experience with this.)
[0] : https://sales.jetbrains.com/hc/en-gb/articles/207240845-What...
In Lightroom what you're describing, with the easy to work with plain files, is called Lightroom Classic. It's still supported alongside regular Lightroom, but it wouldn't be a huge surprise if it gets deprecated at some point. It also only works on desktop.
What is now called regular Lightroom requires Adobe cloud to sync between machines instead of letting you manage your own files & syncing, and is the only way to sync between the newer iPad version of Lightroom or Photoshop and the desktop.
…something that I have to rent and can no longer own.
See also losing user's data:
* https://www.theverge.com/2020/8/20/21377411/adobe-lightroom-...
Cracked creative cloud is a great product, but the creative cloud they sell in stores is really bad. There's nothing worse than being 2,000 miles out to sea with a bunch of footage to edit and no internet to check in with the copyright parole officer.
There are some other comments that highlight why this matters, but I want to add to this: a year ago, the US government decided that they very much don't like Venezuelan government anymore, and issued sanctions. Adobe promptly announced that they'll cancel all subscriptions in Venezuela, no refunds - essentially shutting down the entire creative industry in the country. Now to their benefit, they've managed to get an exemption from the White House just before deadline and eventually did not cancel Venezuelan subscriptions - but was the US government a bit more stubborn, this could have played out in entirely different way.
Point being, a hard dependency on a non-commodity cloud service introduces a new class of problems that locally owned software doesn't have.
Your data is siloed inside a cloud-only app because it means you'll keep paying for it. You can't export it or share it to other apps because that'd be competition for the service you're paying for. It's online-only because it makes the entire service useless once you stop paying.
There's technical solutions for this, and they'll get no traction whatsoever so long as it's more profitable to keep people locked into a cloud-based service.
I really tried. But it's hard, really hard. Like all distributed systems.
CRDTs are far from a panacea:
* No implementation is compatible with another
* There is very few implementation in languages that are not JS.
* The biggest problem is when 2 documents are diverging for too much time (think a blockchain fork) like if you go offline for 4 days while your coworker work on the same document as you do. You have to come back to differential sync, like Git.
And it's just for sync, then to move to P2P you have to handle NAT traversals, distributed identities, rendez-vous places and much more.
A lot of offline-first efforts are trying to solve every possible use case from day one, from single person with a laptop to huge international teams all working on the same asset at the same time and everything in between & beside, and in that direction madness lies. Implementing a useful, viable, relatively minimal product, and expanding its capabilities later would make more sense, though of course that suggestion is immediately countered by fears that the next clone of the idea will implement something first and be crowned king leading the thought process back to "must do it all, must do it all now".
Municipal and Neighborhood level networks would solve a lot of problems inherent in the broader internet, as well as making solving problems as discussed by the OP easier.
For example, it's a lot easier to trust your neighbor than someone on the internet (and better too). For one you can look them in the eyes and shake their hand, for another you are both governed by the same court (though this does make privacy more locally relevant). On the flip-side this means local people don't necessarily have to be great at the internet themselves to use these complex hosted programs. It's just another website rather than setting up a google docs server or using a git repo.
Continuing, another problem it solves is local discoverability. There are a thousand products out there that do local services, from craigslist, on down to "local wikis". But these services don't often care about your community, and it can be hard to know what other people are using (separating local efforts across a global network). Putting them all on a local network, with a local index (and search engines) allows better discoverability nearby (especially if portals into neighboring communities can be connected, eventually perhaps into a federation of sorts), but also services that are oriented to local needs (and issues and regulation).
I'm not saying this would replace the internet. Obviously not. But it would provide a better, safer, and more useful "local layer" of the internet to go to first (to say nothing of the social media security and child safety aspects). That also happened to be faster and more resilient (doesn't go down when the internet does). The problem of course is shit ISPs, though this is why the MeshNet people do what they do. And having applications and a community ready for it. The technology (and culture) would also be useful for communities that can't participate in the global network, from isolated communities (either due to poverty, authoritarinism, or remoteness) to future ones (spaceships, remote colonies).
Just a thought.
Neighbor trust wouldn't last long if it even existed in the first place as soon as easy targets getting hacked became a thing. Most people leave their routers secured by default and even ISPs who use them to distribute wifi "publically" are supposed to leave them secure indicating the preferred level of local network interaction is generally "none".
The locality is a worst of both worlds, arbitrarily limiting discovery if they want to see and be seen and not providing any security as anyone could bridge access within or likely spoof an identity. It seems like something which would rapidly wind up redudant for all terrestrial applications even if it existed earlier.
Distant ones I suspect would either tolerate the higher latency or maintain mirrors or caches - which would likely wind up a subset of the content which doesn't depend upon being current and would probaby become a format which doesn't mind the "necroposting" and delayed reactions as you get hours or days later space responses.
You misunderstood this. My trust of neighbors is more social and less technical. I am less likely to run into bots (and predators) on social media because local administration means those people must be within the jurisdiction of local law enforcement.
> The locality is a worst of both worlds, arbitrarily limiting discovery if they want to see and be seen and not providing any security as anyone could bridge access within or likely spoof an identity.
Part of the point would be to disallow this bridging. If you wanna be seen on both the internet and locally, that would be easy to do. But bridging outsiders into the local network means taking responsibility for them and their actions. And no one would want to do that unless it's people they know.
There have definitely been some challenges with this approach, from available tech stacks to customer support, And user expectations. There were also some unexpected upsides, like requiring less server resources. All in all I think it was worth it and I hope this trend continues.
My apologies, that was an autocorrect-fueled typo, it should have been Dat, not Day.
It currently doesn't have the ability to go to web services for translations you don't have installed locally but that's a feature I'm considering adding at some point. It would seem like the best way to do this we be to track (locally) which translations you use frequently and then save them locally (a one way translation package between two languages is ~100MB). This would give you the privacy/offline access of translating locally for language pairs you frequently use, while also seamlessly supporting the large number of languages supported by cloud services. You could also give power users much more fine-grained control over which translations they keep saved locally.
I've been writing local-first digital archives software. Our system uses git to store collection metadata and git-annex to manage binaries. It was originally designed to be used to allow partner sites with very low connectivity to contribute archival-quality video to a distributed archive. Data can be synced over the network or via physical storage sent in the mail.
Using Git/git-annex for this system has advantages for archival preservation. Copies can be located in geographically diverse locations and synced as necessary. The Git log's use of cryptographic hashes also makes data tamper-evident.
The article linked here talks at length about concurrent edits and the data structures necessary to handle work by multiple application instances. I feel this is (potentially) unnecessary effort. Rather than build applications to support concurrent edits by fat clients, use thin clients that all speak to the same instance of the application with its singular state. Concurrent work would therefore be resolved in real-time by a single application instance in a first-come-first-served basis, without any complex data structures or clever coding.
The root of so much complexity in modern computing is that all of us have multiple application instances when virtually nobody actually wants this. I don't want a separate email client on every device. I want a single email client that I can attach views to and interact with from anywhere.
Most salespeople are always on the go, travelling through areas of bad network, or in flights (which many salespeople consider wasted time because their tools all need a constant internet connection, and that's one of the domains where "time is money" is a core tenet). So when we started building, "local-first" was a strong differentiator for us and became a core philosophy for the company. For the app we have, 95% of functionality works perfectly well offine (except some features that need third party API interaction that we've not found a clean way to set up so far).
Sure, it's so much easier to write a webapp and then add a webview on top of it and call it a "mobile app", but the effort (syncing data which can be edited in multiple places in an local-first world is a crazy engineering challenge) that goes into building a local-first app really pays dividends in the longer run.
[1] https://www.businesswire.com/news/home/20130823005118/en/Mag...
https://en.wikipedia.org/wiki/Solid_(web_decentralization_pr...
You can easily just drop a file in Dropbox to move it from place to place, but its hard to allow multiple apps to modify that file in real time.
https://dropbox.tech/developers/deprecating-the-sync-and-dat...
- It's fast, especially if you run it on a single node with proper hardware.
- It's well suited for multi-device apps. You even get a changes feed for live replication.
- I'd rather bet my money on CouchDB (Apache Project with significant support from IBM) being around in 20 years still than on Dropbox.
Privacy and user control is OFC up to the app developer. Also, it's no worse of a CRDT than the proposed structure, you just need to design the document structure well: In the given example, each todo should be a different document and voilá - you'll only get conflicts if the exact same entry is modified.
1Password comes to mind. If you sign up for the subscription, you can easily access it offline, sync to the cloud now or later. Enable 2factor, and you can still access the vault offline, want to sync with the cloud? Enter the 2factor ID and you're in business.
> Your Work Is Not Trapped on One Device
Microsoft Office does a great job with this and office 365. You have the powerful Word running locally, saving both on disk and virtually in the cloud. But then if you switch machines, want to use in the browser? Easy no problem
> Seamless collaboration
Doesn't Git solve this problem?
> Doesn't Git solve this problem?
The author mentions that:
"The collaboration approach that I personally like the most (and the one I feel could be embedded in “local-first applications”) is the one used by git."
So it's more expensive to write local only software as a result, you will probably deliver features more slowly than your cloud-first counterparts and thus you'll eventually be out competed unless your leveraging local first to deliver something that your cloud first counterparts can't do well, like speed & performance.
They have perpetual licensing (optionally you can subscribe monthly if you don’t want to pay everything upfront) and it really does put power in the user’s hands
It’s fast
The decentralization is absolutely possible nowadays, but there are few incentives to push it forward, unfortunately.
I also remember using an application that did this very poorly. It loaded all data into RAM causing it to consume 1.5GB for very basic functionality.
Do you have an example?
Working on PouchDB I spent a good amount of time thinking about how to "sell" local / offline first apps. It seemed a lot to me like the problem was plain old capitalism, offline / local first software is faster / and more robust, for most circumstances its a better decision for the user. However the economic incentives arent there to build the best software, the economic incentives are to slow webpages to a crawl with adverts and planned obsolescence.
Aside from that there is the issue of portability which comes with the power - namely supporting everything and barrier to entry. But the bigger difference is probably convenience. Web typically needs no install and less commitment and gives a single set of servers to work on for everybody.
Take a general referral/word of mouth. Web has more or less "go to the URL and try it with no commmitments or extra steps" and web shopping found not needing to register was crucial for people actually trying. Now compare local first "Hey you should try and download Notepad++/git/etc." And that is before IT policies or licenses get involved which can mean a certain segment get a hard block they wouldn't for a web service unless they specifically decided that say "pastebin is a hard no" and then missed that the new hotness is say clipboard.com anyway.
We had an approach which tried for its own best of both worlds with its own sandbox and that was in browser Java applets.
But true, if our customers wouldn't pay for our app and we'd have had to resort to ads instead, offline first wouldn't have been been a priority, perhaps not an option at all.
If you can't access your mail provider, you still have all the data from the last time you did, you can search old emails because they're stored locally (you can set how much you want to keep for these situations), and you can write an email and it will be stored in the outbox to be sent when connectivity is restored. In other words, your email experience is exactly the same as with connectivity, except for a banner that reminds you that it hasn't been able to download new mail since HH:mm.