Setting up your own Nextcloud instance to get off of Google
kiramclean.com
kiramclean.com
I find that that reduces the maintenance burden to just keeping a machine running and patched.
I also set up a read-only web frontend with NGINX's directory indexing + htaccess so I can share files easily.
Thanks!
Compared to the maintenance cost of something equivalent 10 years ago, it's a lot better now. I don't find it meaningfully inconvenient, even with kids.
My problem with them is, i don't need as much storage and would orefer more CPU
Synology has this habbit of equipping their lower end models eith too little RAM and underpowered CPU
It is also harder to share it, as the 'owner' can usually see all data thats in it
sudo docker run -d --restart=always -v <LOCAL_PATH_TO_NEXCLOUD_DATA>:/var/www/html --name nextcloud --network=host nextcloud
sudo docker run -d --restart=always -v <LOCAL_PATH_TO_MYSQL_DATA>:/var/lib/mysql --name=mysql --network=host -e MYSQL_ROOT_PASSWORD=hunter2 mysql1) could I fail to detect a failure of an auto-update or novel exploit because I’m more busy now?
2) if so, could something very bad happen?
I got two “yes”s. It still annoys me to have everything in iCloud, but I also have local copies so I’m not too stressed about account lock out.
Otherwise to each their own, it wasn’t an easy decision to give up the autonomy, but the risk/reward equation changes when my time became more scarce.
For folks who would use nextcloud if they didn't have to care about all this server admin stuff I would definitely recommend Hetzner. Cloudamo [1] is also a good option with servers in North America, for people there who care about location.
I don't know what type of hardware they're running their Nextcloud instances on, but I've been using calendar/contacts and public file sharing (since Firefox Send went down) on the cheapest plan for a while now and it's fine. I also dabbled a bit in using Nextcloud's maps, video calling, bookmarks and rss reader applications and it all works OK, although I didn't stick to them because they're not the most rich in features. I had to Hetzner email support a few times and got a helpful reply within 1-2 work days, sometimes quicker, which I thought was fine.
Scaleway is pretty impressive with its pricing too with 75 GB (storage + egress) free per month [2]
[0] https://wasabi.com/cloud-storage-pricing/#three-info
Paying a monthly fee for data you don't access seems like a dishonest pricing model. (I can see that a part of the fee has to be per-month, but it shouldn't be so large).
I just want a to pay a fair price. Not a price based on ungrounded assumptions.
It's like going to a restaurant where the food is cheap but the drinks are crazy expensive.
If you want a fixed price you have to do it your self. It will be cheaper, but less safe, less accessible and more involved.
In my opinion, Nextcloud favours quantity over quality, and many of its core features are subpar. Upgrades are messy, and the docker images fail at reproducible builds. The android client is also buggy.
In summary:
- Nextcloud failed to update from the version 17 to 20 - it did inform me that it can't do this, but not before attempting to update the version number and essentially breaking itself. That said, other software, such as phpBB is also known for brittle updates between major versions.
- After a re-install of the latest version, it failed to allow clients to connect to it through HTTPS (reverse proxy setup), though this proved to be a problem in the default configuration.
- It then also failed to synchronize all of my files once i dumped them into the sync folder, though that's a lesser issue.
- The new Talk ( https://nextcloud.com/talk/ ) application is enabled by default and for some reason noticeably slowed down the whole system, even when not in active use - page loads went from 1-2 seconds to approx. 3-10.
Despite all of that, i would still recommend Nextcloud, because there isn't much software like it out there, and when it works, it is actually pretty okay.
Provided you follow these suggestions:
- Always have working backups. I personally use BackupPC ( https://backuppc.github.io/backuppc/ ) which i self-host, cloud vendors also offer managed options.
- Stick to a single major version, update manually. In the case of Docker, i just stick to a particular image tag, such as :20.0.2-apache.
- When updating, have the backups ready and be prepared to wipe and reinstall the application. Never store anything in Nextcloud that you're not prepared to have unavailable for a while. As for the apps (like Talk or Calendar), where you can't export the data to later import it, be prepared to lose it, or use another alternative.
Sadly this will mean that for a while your instance will be insecure, but the current circumstances force me to prefer working software over secure software.
https://nicolasbouliane.com/blog/nextcloud-docker-upgrade-er...
However, this is the straw that broke the camel's back:
https://github.com/nextcloud/docker/issues/420#issuecomment-...
It does feel kind of unfortunate, however, that while containers themselves are immutable most of the time, the tags themselves will point to different images over time.
If you do want something like a rolling release model, where you always use ":latest" (or ":master" or ":trunk", or ":stable" or whatever), then it feels like you'd need to constantly pull the latest versions of the image, with something like shepherd for Swarm ( https://github.com/djmaze/shepherd ), watchtower for regular Docker ( https://github.com/containrrr/watchtower ) or whatever the Kubernetes alternative is.
The whole DB user thing feels unfortunate, however! Especially on systems, where you have a single "fat" DB instance which is shared amongst different pieces of software (with separate schemas for each), as opposed to a more distributed approach, where each DB instance is separate and used by only one application.
Personally i've come to use the latter approach and run all of my DBs as containers, with bind-mounted /var/lib/$DATA_DIRECTORY_FOR_DB_HERE directories, for easier data backups. Though i guess that's also in part because i want to cap the resources available to each instance and also because i don't trust any piece of software to let it impact others in a shared resource situation.
And its not because I want it to be JS web app or anything like that. Just the core HTML/CSS is poorly implemented and just doesn't look good.
The calendar and addons system just work so poorly. I just cant use it. I just use Seafile for filesharing.
I've run my own email before, also a pain in the ass for spam management. I do own my own domain and use ImprovMX.com to forward it to a free Gmail account. So if I ever need move, I can do it quickly and easily by just changing the forward.
I'm not privacy conscious so I don't really care to run from Google for that reason, I just am tired of them shutting down and killing services I use often.
I just haven't hit an email service that matches the spam fighting powers of Google yet. or the UI. Everything else looks so traditional. and I don't want to pay $99 for Hey, mostly only do month to month so I can easily cancel and not waste money if I want to stop using it.
Anyways, I digress. I just haven't found the right combination of tools for me, but Nextcloud was not it. I honestly think its horribly designed. We need an alternative to it, but I know its a huge undertaking to make something like this, so I am not sure it will happen any time soon.
That said, I would never put anything important on it. I had at least 5 upgrades in the past 2 years result in total failure, with an unrecoverable instance and corrupted DB.
I used Nextcloud for a few years already and never, ever I had a corrupted DB in any of the upgrades.
I tried to use Nextcloud's email app to backup my gmail, but after sending it up I realized that it is just an alternate interface to access email on Google's server, not a proper backup solution. (It only supports IMAP, not POP3.)
If I can't find a better solution, I may just set up Thunderbird and put it and POP3 mode, but I'd really prefer something that could sit on a server and run automatically without me having to touch it.
(For what it's worth, I ended up keeping my Nextcloud server set up because I found that I really like the notes app. I also have it doing a daily backup of my contacts from my phone and it's replaced Dropbox for small file storage. I already had my photos backing up to a Plex instance on the same server, or else I would probably be using the photo backup feature as well. )
You could then use this maildir as the backend for a dovecot IMAP server and use any webmail client of your choosing to connect to this and search it etc.
I'd avoid POP3 - very old protocol and it removes your mail from the server (is this your goal?). You're definitely better using IMAP. Even if you want to delete mail from the server, you're probably safer to fetch it by IMAP, then delete it.
And, yea, I'm not so concerned about which protocol to use or deleting the email from Google's server, I just want my own local copy of it.
I have it run with a systemd timer at regular intervals and it’s backed up with borgmatic at regular intervals using another systemd timer.
Huh? One of those is finite and the other is not.
Docker wi5 a tool like proxmox can go a very, very long way in a matter of a few days... and provide near identical vps maintainability.
Hardware is much more reliable and self monitoring than it used to be.
Ransomware attack vectors are often clicking malicious websites, emails, or software opening stuff that is risky.
Keeping you access point patches system patched and browser up to date is a big line of defence, with or without cloud backups. Those identities can be as easily hacked if the weak spot is our computer.. Using browser and email plug-ins to increase security and safety can be helpful too.
One can run pihole locally at home as well and filter out a great deal of trackers that can get hacked.
It's also relatively easy to setup multiple vlans on your local network to separate your devices.
Having multiple current backups of your computer, physically and locallu is also an important defence against ransomware.
But yeah I also don't think Nextcloud or other alternatives will really take over unless security/privacy/ethics become a higher priority than cost for most people, which seems unlikely to me.
Edit: Typo (of -> off)
I would like to host my own Matrix instance, but I have no idea what the cost would be monthly.
Their cheapest offering (€4/month including tax) is more than enough for even the heavy web applications and their lowest storage tier will net you 300GiB to play with. The big downside is that networking is limited to 100Mbps, but for casual use that's not that big a deal in practice. That's plenty for a NextCloud instance, a matrix server, a bridge or two, a mail server and more with RAM to spare. The benefit of the classic VPS model over auto scaling cloud compute with a free tier is that a runaway script won't cost you a few hundred bucks if something goes wrong.
> As much as I love Sandstorm, it’s hard to come home from my successful day job to work on an unsuccessful side project. And so, I have been spending less and less time on Sandstorm. I still push updates every month to keep the dependencies fresh, but hadn’t worked on any new features in about a year and a half before adding mass transfers recently.
> Meanwhile, without leadership, the community has mostly disbanded. The only app that gets regular updates anymore is Wekan, thanks to its maintainer Lauri “xet7” Ojansivu. Jake Weisz heroically continues to carry the Sandstorm flag, reviewing app submissions (mostly from Lauri), replying to questions and bug reports, and advocating Sandstorm around the internet. A couple others lurk on the mailing list and IRC. Most people have moved on.
> Almost all the app packages are from 2015-2016; many of those apps have had significant updates in their standalone versions since then which are missing on Sandstorm.
I run nextcloud on a $5 vps just like in this article, it's still underutilised. Out of the box my family can have access to certain folders and their own account, they can watch videos in the browser and I can have a selfhosted google docs/sheets alternative. All encrypted at rest.
- SyncThing: decentralized, fast & unix aligned
- CryptPad: end-to-end encrypted, real-time collaborative editing
Same. Setting up v20 for a customer these days. Heavy, slow. Buggy LDAP. Not easily containerized: official Docker image tries to rsync ALL the distribution from /usr to /var/www/html every time your start run the container, and you can't mount individual volumes (e.g. just data), because they keep the config at /var/www/html/config.php
Customer just wants a UI for a filesystem, and I'd gladly replace it if I could.
The cloud seems maintenance free until the plan or features change or go away altogether.
Many people don't think there are viable Gmail alternatives, but on Drew DeVault's recommendation I recently switched myself over to using Migadu[1] and the experience has been fantastic.
That said, I am running CalDAV and CardDAV at home on my own hardware, but they do offer a simple version of both that should be usable.
I'm now at the point where I'm trying to find a different solution. I don't even use all the web UI stuff, but desktop sync is unusable to me now. It all started a few months ago when the new desktop client version came out, and would always crash when opening that new view. When it was updated on the server side, I started getting signed out every time I shut down one of the computers I'm using, which gets especially frustrating on the computer where I'm using two nextcloud accounts, because I have to log in, wait 30s because apparently there are bad logins from my IP (??), grant access, log out, log in with the other account, grant access again, and then do the same dance again on the next computer I start. And ever since I've been using it, I kept getting "undefined states" and weird database errors.
It just seems like they don't do a lot of testing and focus more on features than stability. Any suggestions for a very stable open source service hosted in linux that does good desktop sync with a windows client?
I've always had misgivings about the security of using and running PHP. Does PHP software still tend to be insecure as in the past? Why are there so many web apps (especially wiki servers) written in PHP instead of e.g. Python, Rust or some other language?
All the advancements on PHP are great, it's becoming a JAVA more and more, but likely any PHP legacy apps will never see an upgrade to the latest version. And starting a new project in PHP, nah, there are better more well designed programming languages.
Currently considering this vs. a few standalone docker instances for calendars/contacts for one of my downtime projects over the next few weeks...
In general, Nextcloud has been transformative in allowing us to move off Google products. All our music and photos are on Nextcloud too.
My biggest gripe with Nextcloud is that upgrading between major versions is a pain and they release upgrades often. If my VPS didn't have other duties as well, I would probably switch to a hosted version.
apples can access it via builtin caldav/carddav, android needs an app.
I employ a "spider" system. I use paid online storage like Onedrive, S3, Azureblob or Gdrive, or Fastmail for email and calendar and contacts, but pull all data from their primary hosting location and then disperse it into multiple cloud copies plus local plus distributed local offline (in intervals). All spider copies are encrypted either using GPG or symmetric encryption. I can access all spider target locations with credentials I know by heart or have offline backups of them, and I can also access or know the encryption keys required to access the data itself.
I also keep an encrypted emergency copy of my most important data on my phone's storage.
In such a system, it's perfectly fine to use Google storage + Azure storage + S3 and whatnot provider, as long as you have a sound system and encrypt your data. They can deliver important features you cannot deliver yourself at reasonable cost.
Nothing is free.
Essentially, to safeguard the most important personal data, the exact same methods and knowledge that a system engineer who designs a storage layer as part of business critical infrastructure should be applied.
Nextcloud is just too messy and complex. I don't want to spent a lot of time managing its lifecycle, testing updates before deploying them to production, etc. I trust accomplished vendors to host production copies of my data as a service, and take responsibility from there. Sure it's easy enough to deploy some piece of open source software and update it continuously, but I suppose the work that goes into making it a high-quality service, like testing updates with duplicate data before letting a new software version lose on your important personal data master copies, is something I doubt few people are putting in.
— what software and algorithm do you use?
There could be problems with some of these programs.
— how do you handle the key management to ensure that you are not locked out of your data?
I am not sure if memorization of the secrets is a good approach.
Their technology is miles behind OneDrive/Dropbox and they seam to not care but add features that enable them to attract new customers instead of fixing the core. When you need to use Office products nothing beats OneDrive.
I really want to use nextcloud for managing files but I could not find a satisfactory answer.
Oh, I have a daily file backup meaning I can get any copy of a file from last year so that is covered.
FreeNas feels much more stable. If you only work in an office I'd use network drives they are less hassle than syncing (though mixed environments with osx and smb are a pain) If you need the files with you, use NextCloud.
I kinda havent switched because of that reason only. I dont need data outside of the lan network and for outside I use zerotier so I always have that network drive.
Good to have options. Maybe one day I might switch everyone but that would be a lot of trouble
If my memory serves me well, then at the beginning updates caused a lot of issues, but for past couple of years I'm pretty confident when doing updates. I update only via web UI when new version becomes available for my instance. Currently running v19 though v20 has been released 2 months ago. That way I believe I avoid some of the bugs when major version is released :).
Some negatives aspects that I have encountered, but don't see as a big deal -
newer Nextcloud versions require newer PHP versions that are not provided by Ubuntu LTS, that means, that I'm no really benefiting from LTS and I have to upgrade the distribution if I want latest Nextcloud (don't want to upgrade PHP separately).
and couldn't get Talk working reliably, whatever I'm missing, it feels that documentation could have been better.
[1]: https://homedrive.io
the only recurring cost is electricity cost for self-serving at home.
The only thing in the pack that I didnt like that much was the document editor, where letters appeared in the wrong order when typing too fast. this could possibly be due to running the ARM version of the backbone.
But then there are other ways to edit documents, and the rest of the default tools well done.
it is also possible to get different plugins as well as there are many apps on f-droid that integrate nicely
As someone who's worked on more than a few collaborative text editors through the years, this just sounds like a correctness bug in how they're processing typing events.
At most I spent 10min/month to update to the next version.
Its one thing to set it up, and it's another thing to keep it running stable and secure.
Storage - the biggest issue with rolling your own cloud. For some reason storage cost is usually the killer in these scenarios.
The thing I hate most is ads. And YouTube (unless you pay them) gets ever more aggressive with showing ads.
The pre-roll ads are bad enough, but interrupting a playing video with a couple of ads (as happened to me the other day) is way beyond the pale.
I guess I need to take a closer look at integrating youtube-dl into my Mythtv environment and otherwise just ignore YT (as I do all of Google's other herding tools).
They are not. And I generally don't. In fact, I rarely watch YouTube. When I do, it's generally to find a specific song or video clip (both usually decades old) to forward along to someone else.
In those situations, I would be shocked if the "owner" of the content (the YT channel owner) had actually paid license fees to the actual copyright owner.
As for paying $12/month to get rid of ads, that seems an awful lot like a protection racket. "It'd be a real shame if you had to waste all that time watching ads. We don't care about the money, we just want to help you make your viewing experience better."
Given that the video in question[0] wasn't actually created (or even directly licensed) by the owner of the YT channel, your point is moot for that particular piece of content.
As for actual content creators, I absolutely believe they should be fairly compensated for their work.
However, simply repackaging the work of others isn't "content creation."
Regardless, I make every effort to shun all platforms that attempt to show me ads. That's my choice. Because I don't care to be presented with a bunch of garbage or pay the same people who profit from it to not see such crap.
If there's something I want, I'll buy (not stream or rent or license) it.
There's far too much of that going on and I refuse to support it. If I buy music or a phone or a car or a tractor or seed or anything else, I should be able to use it as I see fit, without restriction, data exfiltration or a bunch of scum trying to sell me stuff.
As a general rule, I don't. And if you'd actually read/comprehended the post you replied to, you'd know that.
I don't tolerate that garbage from TV (via time-shifting/fast-forwarding), why should I accept it from online sources?
And I don't. Show me a site that has ads I can't skip and I'll show you a site I won't use.
1- It uses SQLite, you can't change this easily as the same SQLite is also accessed by MIAB to sync contacts and calendar functionality. The Nextcloud devs don't recommend SQLite for multi user servers. [1]
2- MIAB supports using Nextcloud for managing contacts and calendar but file management is not officially supported so if you want to use that same installation of Nextcloud for storing your files, you won't get support from the MIAB devs in relation to any issue related to files storage [2] (in that case you will have to seek help from the Nexcloud community).
3- As a result of the 2 points above, for multiuser installations you may want to have a second instance of Nextcloud, backed by Postgres or MariaDB, just for your files storage, while you can keep using the version included in MIAB for contacts and calendar. However, for a single user installation, you would be fine using the version of Nextcloud that comes in MIAB (I have been doing that for a long time and I'm fine, mine is a single-user system).
4- The version of Nextcloud that is included in MIAB by default locks access to the admin interface, this can be unlocked using the provided script [3]
5- The automated backups that MIAB creates, will also include all your Nextcloud flies, not just email, calendar and contacts. So if you have a lot of files it's a good idea to use the S3 backup functionality instead of the default on-disk backup method (otherwise backups files will soon eat up all your storage). In my case I use the MIAB S3 backups feature, and I save my data in Digitalocean Spaces which is S3-compatible [4]
[1] https://docs.nextcloud.com/server/stable/admin_manual/config...
[2] See par. "What’s inside the box?" https://mailinabox.email/
[3] https://github.com/mail-in-a-box/mailinabox/blob/master/tool...
[4] https://www.digitalocean.com/docs/spaces/resources/s3-sdk-ex...
At this point, the only solution I’ve found to the kind of self-hosting stuff nextcloud provides are various utilities built on top of SSH — like rsync (which works perfectly) - or samba (which works OK). I haven’t found anything that really works well on iOS.
It worked fine at first but after a while it got really confused, to a point where Nextcloud randomly decided that the older version on the server was actually the more recent one, and synced that folder back onto my local, updated repository. And I was the only person touching those folders, the Nextcloud server was only used as a backup during that period.
It's got a couple neat features, but problems like this are really not excusable. When the one core feature breaks in everyday situations for users doing version control it's not a good look.