Reviving Sandstorm
sandstorm.io
sandstorm.io
Now Letsencrypt support free wildcards so perhaps it'll be easier for Sandstorm to get momentum.
Sandstorm uses a new hostname for every session https://docs.sandstorm.io/en/latest/administering/wildcard/#...
https://docs.sandstorm.io/en/latest/administering/sandcats/
...but yeah, now that let's encrypt does wildcards it'll be easier to do it on your own domain. I'd like to see sandstorm have some nice integration for this in the future.
(sandcats still works of course, and is still a good option to get started if you don't want to mess with DNS and certs right off the bat).
* I'm also the tech lead of Cloudflare Workers.
For those who dont know what it is:
Single Sing On with an App Store. Better explained:
You create one account and then can create/edit/share/copy projects of completely unrelated software . So lets say you create a gitlab installation, 3 etherpad projects, 5 wekan (trello clone) boards and then share them with your team/client/family by just adding their email adresses - awesome tool!
Security is critical for a platform for self-hosted apps. One vulnerable or malicious app shouldn't be able to compromise your entire platform. Docker-based solutions don't provide this.
Does that make sense as a comparison?
If not for that mismatch, one could just wrap most existing apps that support pubcookie authentication in a layer that translates Sandstorm APIs into HTTP.
If I make an app, I can safely assume that the user has a bank account, an e-mail address, etc. What would happen to the app economy if we could also assume that the user had a server? Imagine all the apps that would be made possible if the app makers didn't have to pay for and scale a back-end service!
Sandstorm.io, https://solidproject.org by Tim Berners-Lee, many people have tried, hopefully someone will succeed.
My parents can understand something like a physical box that sits next to their router.
They just need to start specializing rather than being a "jack of all trades, master of none." For example, being a best-in-class way to host a Matrix server. Become a best-in-class Google Photos alternative, etc.
I think it's hard to sell people on a cloud box that can't do everything people want to do on the cloud. One missing app can (and sometimes does) make or break someone's interest in a platform.
So in that sense, having too many highly specialized (or low quality) apps can also be a turn off because it just confuses people.
https://sandstorm.io/news/2014-07-21-open-source-web-apps-re...
Unfortunately, it's unlikely that we (at least in the US) will find the political will to change the law that says if you give your data to a third party, it is no longer protected by the fourth amendment and doesn't require a warrant to obtain.
So self-hosting your own data seems to be the only way you can keep your data protected by the fourth amendment.
Something like that could even be integrated into iCloud if it becomes popular. Then your "user provided server" would probably only work with other Apple products, Apple style. But for many users that would probably be good enough.
And you wouldn't be limited to having only one of these "user provided server thingies", either.
Deploying a decentralized app that can go from Zero to Twitter scale overnight is the goal, and seems to be within grasp.
Of course, Holo doesn’t solve the problem of governments or the mob quenching your liberty, but it’s a start.
It’s a stepping stone to fully decentralized distributed applications where every user is also a host of a full node.
The most reasonable thing to do, of course is something like Sandstorm did. You get the technological 'status quo' of the web and mix it into a self-sustained, autonomical node.
I think that when you need to have this level of autonomy, the design meant for the web doesnt solve this very well.
This is the reason why i think both Sandstorm and Solid didnt 'hit the nail' yet, because we need a couple more "iterations" to solve this in a more satisfying way, and it goes deep down into the architectural choices.
Having said that i think Sandstorm, Solid, DAT and IPFS have the right mindset, and the sort of meditation that are pointing us to a future.
The real danger here, is that the web on its own will get trapped by the mobile walled gardens of both Apple and Android were you dont have any choice of distribution or reach without saying yes to a digital overlord that can terminate your channel within the digital realm over any bogus reason presented.
And unfortunatelly our legislators are still not prepared to deal with our current digital citizenship and to make the proper stand on our civil rights.
Without a proper solution, we are heading to a dystopic digital feudalism, and this is one of the reasons why im also working hard to launch something that will at least try to make this dark looking future not so unavoidable.
(And thats why i have a utter respect for projects like Sandstorm and Solid)
I repeat; The web stands no change and will become a "channel" within a bigger internet ecosystem, and a symptom of this is the web being embedded as a app platform for mobile and in the near future desktop apps.
The web has a great future as a application platform, but this free, utopian world where we have a more free and descentralized way to share information is losing a lot of ground, and if we dont keep some great values the web stand for in new tech incarnations, we stand no chance to keep all this.
Sure, but that doesn't mean you assume that the user operates their own bank. They have some third party they trust to hold their money for them. There are risks to that, but in practice it turns out those risks are much less grave than the risks associated with everybody holding their life savings under their mattress.
This is why I've always been skeptical of the "everyone should have their own server" vision: servers require administration, administration requires technical skills that most people will never have, and even lots of those that do have them aren't necessarily skilled enough to be able to keep them reliable and secure. For all those people, having some professionals at a third party deal with that stuff will yield better results than DIY will. There are risks here too, but it'll probably be easier to mitigate them via regulation than it would be to teach everyone in the world to speak Unix.
(The one way out of this trap would be if we could come up with some kind of server that did not require administration -- that had guaranteed 100% uptime, that never required security updates, that scaled automatically to meet any level of demand, that didn't have physical parts that wear out over time, etc. It's telling that the closest thing we have to that today is less a server and more something like AWS, which... requires trusting a third party.)
> The one way out of this trap would be if we could come up with some kind of server that did not require administration
To a large extent this is actually a goal of sandstorm.
I'd contest some of your requirements though: /Nobody/ has 100% uptime, even AWS. And most individuals are not in a position where an outage is going to cost them millions. It might suck, but people trudge through worse problems with their PCs; no reason a personal server ought to be different. So I think the bar is a bit lower than you suggest; I think it's possible to get a server to the point where it can be "administered" by someone who's capable of "administering" a laptop.
We're obviously not there yet; you still need to set up a Linux box before installing Sandstorm itself, and we don't really have a great Story wrt backups yet. But Sandstorm itself is already mostly fire-and-forget; it auto-updates itself, our security track record is rock solid, and I can't remember the last time I had to do anything that felt like sysadmin work for my Sandstorm box. There are a couple things I think still need to happen:
1. We need an automatic backups & recovery story.
2. We can't require the user to install Linux first; we'd need a "sandstorm distro" so folks can do the whole business together. The OS should be hardened by default and self-update with Sandstorm, as well as integrate with the admin panel for scheduling reboots.
3. Ideally, there'd be hardware you could buy that's just already running the sandstorm distro.
It should be the same with whatever someone comes up to as a "bring your own server" solution imo. I'm completely fine with having to do more manual work to set up my own box at home or something like that. Maybe I'll install it on Azure or AWS instead. Or just source it out completely. Would be nice to at least have the option :)
A server is also a mashup of functionality. I can’t see any reason why it cannot be made as easy as a smartphone sometime not too long from now. And making things easy and inexpensive enough can also create a market, even if there didn’t appear to be one.
On the other hand another approach would be to use more client-side processing[1] in the existing environment. This would not completely remove the need for the server on app maker side, but could possibly offer a significant reduction of the resources needed.
The third approach could be to change the billing model only, where processing would be done on a (very small and short term) cloud instance run by the mobile operator and the customer being billed for usage of that. In other words have computation-as-a-service as a part of mobile operator service. This would also have quite complicated privacy implications.
[0] And if the router-route would be taken, it would make things much cheaper since the routers CPU could be used. Even if additional cores would be required this is still much cheaper than having a separate server-device. Also much better energy-wise since for most of the time the server cores could be off/stand-by and a small process on router-cores would wake them up only if needed.
[1]Given how diverse client-devices are it's not that easy to have client side processing on all platforms. I wonder if WASM will make client-side processing more common.
The reason I stopped using it was development... It was amazingly hard and confusing to develop a "hello world" application for Sandstorm.
Sandstorm had a great GUI and workflow for users, but it seemed like new developers on the platform were ignored... which reduced the growth of the platform and eventually killing it
Glad to see it's being revived though and maybe I'm smart enough to make my own Sandstorm-Compatible app now!
Regarding 2) I wondered: was this the "new heroku." Or, better than zeit? Or better than docker?
I still don't know.
After installing it, I went to the marketplace and looked at the set of apps. Looks like there is button to quickly install piwik (alternative to Google Analytics). It worked really well. There is a spreadsheet app. A git server.
But, I can do basically the same thing by going to the marketplace in Google Cloud Compute and choosing GitLab, or Wordpress, or whatever.
Sandstorm looks ugly. I know that does not mean it is better or worse, but GCE certainly looks more professional when I install one of their marketplace apps.
So, what am I missing about Sandstorm? If I choose apps like GitLab or Wordpress where there is a documented and easy way to migrate my data so that I can easily move off GCE and over to another provider. Then I feel that my lock-in risk is mitigated, but is there something else that Sandstorm gives me here that I'm missing?
The big difference, in my mind, is that each app is integrated with a permissions/capabilities model and strongly sandboxed by individual instance or document. Sandstorm itself handles logins, and handles capabilities to access documents; then e.g. there's no way that Etherpad can accidentally leak your doc to someone else unless you've granted a Sandstorm-level permission to that person.
It sort of flips from the "walled-garden app" model, where the app is the boundary and individual docs and users are within the app, to the traditional "computer with filesystem" model, where the user login and the file with permissions are system-level concepts. (Except instead of "file", they call it a grain, and it's a separate instance of Etherpad or Gogs or whatever.) That's also what makes this more than just Docker containers -- deeper integration into the app.
That also gives you flexibility to have a bunch of different instances of a single app, and IIRC, they have functionality to import/export those instances in a well-defined format from one host to another.
At least, that's what I remember from playing with it 4 years ago. In any case, I got the impression that the model was much more secure, and flexible, than just "install Gogs on vanilla Linux".
If I knew that I could create a private walled garden with a bunch of disparate apps that are all connected, that I would have been excited about. That's hard work they did, and it is a shame they don't promote that up front. I'm confused who they think they are talking to, since "open source apps" would only appeal to developer/sys-admin people anyway.
Communicating everything that any potential user might want to know in a single sentence turns out to be really hard.
If I look at Cloudtron, I see Gogs (git), Wordpress, GitLab. etc. If I go to GCE marketplace, I see those same things. Both of these sites have a button where I can install those apps easily.
What does the addition of "self-hosting" mean for me?
For most people, paying for a VPS is the easier and cheaper route maybe, but for some of us, that flexibility and total ownership is a much better option. I suppose it mostly boils down to who you trust more: Google Cloud, or your own skills. (and time/patience)
To your point, I'd be curious if the data selling/using policies of Linode vs Google Cloud are different.
Honestly, this is why I stopped using it after I installed it a few years back. Well, both the ugliness and the UX.
It would be very important to describe and make it much easier for the community to ship self hosted Sandstorm packages.
I'd much prefer if Sandstorm just shipped with an IdP and contributed OIDC implementations to downstream projects.
Others have a fair bit more difficulty, but I think it comes down to market share: If a lot of their users are on Sandstorm, they're going to care about maintaining a Sandstorm package. If the Sandstorm project isn't as widely used, people aren't going to be that excited to do all the extra work for packaging.
Wekan has Sandstorm packages as part of the release pipeline: Every single release is submitted to the Sandstorm app market, actually. But Wekan is both considered a core app for Sandstorm, and has been available on Sandstorm since it's very early days as a project, so the group of users using Wekan on Sandstorm has always been pretty healthy.
I know exactly one non-cannonical project that maintains a reliable out of the box experience for snaps, Nextcloud. And that's because they use and package it themselves.
Yet I'm never quite sure if I'm about to shoot myself in the foot installing any other snap.
Same with Sandstorm. When I ran Mattermost for a bit I was initially shocked how seamless and fast it worked, followed by immediate disappointment that apparently just a handful of versions had been patched and it was lots of major versions behind by now.
Will anything significantly change in contribution activity going forward? Are there again some people working full-time on Sandstorm, which is realistically necessary for a project of this size? The announcement doesn't say anything about that.
But, even without that, I do think there's a reason to be hopeful that this is more than a momentary stir. The company shut down so suddenly and without really on-boarding new contributors, and I think much of the problem was that of getting over the hump of "all of the people who've worked heavily on this codebase disappeared at once." I think it might not have stalled in the first place if there were a couple folks outside the company who had been doing regular work on the platform. I still feel like our bus factor is a little low wrt to hacking on core, but I'm hopeful that we can get to a point where there's enough shared knowledge that we won't end up in quite that kind of slump again.
I wish you the best of luck, and really hope that Sandstorm succeeds! Thanks for putting in the effort!
There's obviously some security cost to doing that, but it's a cost most people would accept in the context of a grain they want to make easily findable to the public.
So why would someone pay for your project? Even a very novel solution like Sandstorm, which six years later has no real equal in the security department, struggled with convincing people to pay for it.
While there are some warts, and development has been slow for a while, the project is fundamentally sound and still quite useful.
As a community, many of our conversations and notes are spread across five or six servers in different people’s control. We’d like a better experience for that long term.
Email login is pretty good now though if you don’t want to deal with Google or GitHub, and now that LDAP and SAML is available for free, intranet is pretty doable with standard conventions known to most IT departments.
Sandstorm explains why they don't use Docker: https://sandstorm.io/news/2014-08-19-why-not-run-docker-apps
But the problem with the ease of adoption remains.
However, Sandstorm’s security model is a step beyone what something Docker based can offer.
I don't think so. Here's their license: https://git.cloudron.io/cloudron/box/blob/master/LICENSE
https://docs.sandstorm.io/en/latest/administering/faq/#why-d...
Though that doesn't address the intranet issue of course.
Additionally, as long as the question "WTF IS THIS??" is not forbidden in the HN comments, even those lacking the domain knowledge, or a sufficiently useful description on the site itself or a useful Wikipedia page, I think the extent to assume is fairly low.
That is, I think it's safe to assume the internet, and its users, exist, and that the reader is sufficiently intelligent to make use of the resources available to them, when writing a post to publish online, if it's not intentionally an introductory post
It makes sense for some services that are fundamentally user-fragmented (File backup/Evernote/Flickr) but less sense for things that require multiple users to be credentialed like social networking, since social networking structurally benefits from everyone being on the same service.
Is it a non-profit organization or an actual company? I didn't find anything about the pricing plan
Sandstorm was (maybe technically still is?) a company with a business plan. However that's effectively been disbanded. From 2017: https://sandstorm.io/news/2017-02-06-sandstorm-returning-to-...
Nowadays it's a community project, still getting security updates from Kenton. Spinning out some sort of non-profit funding situation, probably unaffiliated with the initial company, is plausible. Right now it's all speculative though, which is why you won't see anything about it on the site.
Dut doodoodoodoodoot.
Oops, apparently that's the wrong Sandstorm...
And then half of the time the docker containers you need aren't built for ARM so your RPI home server cant run it without building it all yourself or finding an alternative container that is built. And then docker has no built in update mechanism so you have to constantly manually check that nothing you are using has any updates.
The only time docker actually is simple is when you use one of the "all in one" containers that has everything you need but then it binds to port 80 and 443 and now thats the only app you can run.
Docker could be used in combination with a sandstorm style tool. The real value in Sandstorm was meant to be that you didn't have to configure everything, you just press "install" and it would be all done for you and kept up to date.
Dokku is single-server only, however. So if you need a Docker swarm, you'll have to do that yourself :)
Another option is CapRover, which is also free and open source. CapRover is a simple interface for Docker Swarm, Nginx, and Let's Encrypt that is designed for deploying web services. It optionally supports scaling web apps across multiple servers, and also includes a web interface which can be used in place of its CLI, if you prefer to manage the system in the browser.
That only says you haven't spent enough time. Say, 15 hours would be just enough to understand postgres container and figure out the best working configuration for you. Apps on top of it? Good luck spending extra 30 hours.
The thing is docker has never claimed that you don't need to learn about the software you're gonna run. It simplifies only deployment (a.k.a. installing and executing binaries) by providing uniform environment, and by making configuration files more reproducible, but it never reduced the complexity of software configuration.
But that doesn't mean Docker isn't a viable option for simplified server-application deployment. All we need is a set of well maintained recipes. It's just that there are too many recipes out there, unreviewed, untested, unmaintained. If one central party takes the responsibility here, Docker (or any other container solution) can outshoot Sandstrom in no time in terms of both quality and quantity.
You're right, a Dockerized alternative might be as good or better. But until such a platform exists (I do not count Cloudron because it's closed source now), it is not viable for the people Sandstorm is targeting.
Being with 0 experience with Docker and only with bare Linux experience, I'v set myself to set up nextcloud + mysql + nginx-proxy (with some custom config) + letsencrypt via docker containers on arm platform.
Learning docker basics, some docker-compose, I'v got that set up in a few evenings, maybe some 6 to 10 hours total (including first time experience of setting up raspberry pi with external storage). nginx-proxy being hardest part for me, as perfect image that fits arm platform was made available just after fair amount of searching and trying to understand what it takes to make it compatible with arm.
After all, I was really pleased with what I felt how much docker saved time for me. Maybe not really fast, but some productive full working day that is. I'v learned something about a new technology that gave me nice preconfigured webserver with hardened TLS settings etc. I thought, if I'd have to go down the route of manually installing this dependency, configuring this and that, etc - uh, that would surely have taken me longer.
The other side being a joy was - hey, I can host this all on a single raspberrypi, the frontend is isolated from backend and I can still add stuff behind nginx-proxy (and it will be isolated from nextcloud frontend), as my RAM usage is somwhere around 512MB.
Just wanted to share how I feel about Docker when putting my hands on it (single host, swarmless scenario)
If your widget takes 6 steps, and another otherwise equal one takes 1 step, then whether that widget is for users or developers, the choice is a no-brainer.
1) temporarily decrease the # of supported apps.
2) new custom react UI
3) call it project headless. try to discard a ton of code. keep the schemas, fork the services, grpc/capnproto them etc, tear out the good bits of the server sides of these apps. If an app has nothing special in it - discard it.
We all know in the long run this would be the least labor intensive way to execute a project like this. I bet you'd get a huge amount of contributions flowing in too.
More generally, it's a project-specific news section, it doesn't need to introduce itself in literally every post - check the site instead.
> Sandstorm enables non-technical end users to install and run arbitrary software on servers they control. Apps may be downloaded from an app store and installed with one click, like installing apps on your phone. Each app runs in a secure sandbox, where it cannot interfere with other apps without permission.
It's trying to allow anyone to self-host.
Source Sans Pro 300 is too thin for body text on most platforms. It’s tolerable on Apple platforms with high-DPI displays due to their stroke thickening (where they essentially ignore the font author and do their own thing, though you could also argue that it’s just a form of gamma control), but on other platforms like Windows it’s much too thin, to the point that on many low-DPI monitors it’s genuinely painful to read.
Be very careful in general when using light font weights for body text. 400 is a much safer option.