Hacker News Reader

Comment by EchoForge | Hacker News Reader

153 pointsEchoForge61 comments

I had to stop using it after loads of conflict files piled up over the years in my notes folder.

I'm trying to switch to WebDAV for notes. Considering Caddy+WebDAV[0] or Peergos[1] or SeaweedFS[2], but not Next/OwnCloud.

Is still consider using Syncthing for other files types.

[0] https://whhone.com/posts/webdav-syncthing/

[1] https://github.com/search?q=repo%3APeergos%2FPeergos%20webda...

[2] https://github.com/seaweedfs/seaweedfs/blob/master/weed/comm...

I had similar issues years ago and it was frustrating. What I do now is have syncthing on a 'server' that is always on and everything syncs to the server and not between each device. That reduced my conflicts by 99%. Another issue I had was accidentally deleting a folder and having it deleted off of all my devices.

To solve that issue I have my server do ZFS snapshots on my syncthing folder, which makes for easy recovery. With those 2 things being done, I can't say enough good things about syncthing. Compared to nextcloud it is almost maintenance free.

Syncthing has staggered versioning that keeps several versions at different timepoints this may have solved that issue.

Why not just use rsync -a periodically over an encrypted ssh connection? It works across Mac/Linux etc. and is just as efficient, no?

If you really want to be fancy, you can deploy a script to watch files for changes and do rsync immediately when it happene.

> If you really want to be fancy, you can deploy a script to watch files for changes and do rsync immediately when it happene.

I use a tool that does that, it's called Syncthing.

Yeah but if you’re just syncing everything to and from a server anyway, why bother with it?

Plus, rsync -a never deletes files.

Because you want syncthing to delete files more often than not. When you want the reverse, it's a single property on the shared folder (ignoreDelete), feel free to set it.

Is that ignoreDelete a Syncthing setting or something on the file system?

Syncthing setting, per shared directory: https://docs.syncthing.net/advanced/folder-ignoredelete.html

Plus why doesn't sync thing support rsync protocols anyway? It's not like it couldn't, right?

They're not the same thing. rsync is unidirectional, and only between two endpoints. One side is always overwritten by the other side.

syncthing is bidirectional, syncing changes in either direction, and supports syncing any number of clients, not just two.

Watching and syncing immediately - yes, that's what syncthing does. Why reinvent it with something worse?

You can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CVS on the mounted filesystem.

Not sure why this would be easier or better

FTP is not encrypted. Unless you use SFTP maybe. (I wrote an SFTP and FTP-over-SSH client in Visual Basic like 20 yesrs ago lol).

And even then, you miss out on all the optimizations with prolly trees and other goodness that rsync has.

No, sorry, that solution sucks. CVS? That’s super outdated.

It's a reference to the infamous dropbox comment: https://news.ycombinator.com/item?id=9224

Anyways, if you're at the grocery store and you need to see your shopping list, and you're happy with your current workflow of opening up Termux and typing `rsync -avrsomgwtfbbq me@server/shopping-list.txt` with your thumbs, then by all means keep doing that! For the rest of us, there's Syncthing.

Yeah but on the substance, none of your countrexamples make sense

SyncThing doesnt actually have an iOS client that syncs reliably. I tried some, when I was test driving it and Resilio Sync and others.

https://forum.syncthing.net/t/syncthing-for-ios/16045

On your phone you’d probably just visit a website, and access your server via HTTP.

I sync files across Windows, FreeBSD, Linux and Android. Your solution wouldn't work for me as easily as Syncthing.

That isn't going to handle possibly conflicting changes from more than one source.

Conflict files are good. It means you changed the same file in multiple locations and Syncthing didn't overwrite or delete any of them.

Much like forest fires, only you can prevent conflict files.

Nah. They're not alone. For me, syncthing would just randomly decide an updated file conflicted; I had two computers and a smartphone and was pretty careful about this stuff.

It was its only fault - but it was easier to figure out what the actual most-current file was.

Nextcloud does the same thing occasionally, and it's not intuitive at all trying to figure out which one is the proper one to keep.

There was a bug in the android client a while back that was causing that it is much better now.

> I had to stop using it after loads of conflict files piled up over the years in my notes folder.

SyncThing is still still _loads_ better than Next/Own-Cloud unless you also need the extra baggage those two bring.

Having said that, "conflict-sync" files are my current issue.

I have several devices that sync to a local central device and a remote central device and I'm struggling to determine if the "conflicts" that are _not_ conflicts are because of this "dual-honed" approach or not.

I've had to reset it multiple times due to the same sort of issue. There doesn't seem to be a good way to resolve conflicts; at least, none that reliably works.

You have to ensure there's continuity - at least something online and connected at all times, to ensure that the system can know and have distributed the latest version of the files.

I just have it running on my desktop, and at this point, the system's been running for nearly 6 years, and the file conflicts are rare. They only happen when I e.g. do edits on my laptop offline, then go home and do more edits on my desktop before letting my laptop sync.

Since I started using Syncthing, I have found many cases of "just make a new folder and share it with selection of other devices" and now, that I am running it on a server, it is also always online, ready to share with any device that comes online. It is just so neat and handy!

Syncthing is probably the best sync software on market right now in my view. I have had very positive experience with it.

I have an always on device that runs various applications including syncthing. Works great and sync is fast. Like send a movie from a computer to phone, and it gets there fast.

I have started to use encrypted folders (beta feature). If you have a VPS, this might be a useful feature for you.

I like that the transfers are peer to peer, and syncthing has been very secure so far.

SyncThing has been the only piece of open-source software which I've fire-and-forget installed and it's working indefinitely.

I use it on so many platforms and even sometimes in three-way like my Obsidian vault that syncs between my laptop, my phone and my tablet. I barely ever had any conflict. Amazing software.

Yep, good software and the dev (nutomic) is the main Lemmy coder, meaning the fediverse is in good hands.

Seconding the sentiment <3

I've had the intention of testing it out for YEARS and finally got around to it last week or so. Really pleasantly surprised how well the NAT punching / relay servers handle keeping machines synced.

I wish I had tried it all those years ago. I wish excluding directories was a little more intuitive and there was a good way to administer config's on a machine not local to me. Maybe over SSH, I'm still learning.

You can use SSH forwarding.

ssh -L 12345:localhost:8384 remoteserver

Keep the ssh session open, maybe “watch date” or something

Go to localhost:12345 in your browser

Bosh

> ...a good way to administer config's on a machine not local to me.

If you can't access the web interface, is Wireguard an option? That's how I do mine.

This system works extremely well unless you have a LOT of small files (like 200-400k or more). Then, unfortunately, it stalls/breaks in very weird ways.

We had to move on and go with more manual synchronization methods like using robocopy and so forth.

It's unfortunate given how awesome SyncThing + SyncTrazor works on Windows Servers. But I cannot recommend it highly enough if you don't have a system with a huge number of individual files - it works very well.

I found that it struggled with larger data sets. I have a directory that is 70TB and about 300,000+ files and unfortunately, it just can't handle that. Resilio (formerly BitTorrent Sync) handles it nicely and they just opened up to allow home users better access to it.

I find Resilio to want to fall back to relaying, even though ports are open and if I restart I get a direct connection immediately.

Would be nice if I could tell it to never use relay servers.

Anecdotal data, but it feels like a majority of the nice software I look up on Github ends up being written in Go.

Golang definitely inspires confidence. My employer also uses it. I like that it's defaults do a good job at mitigating supply chain risks.

https://go.dev/blog/supply-chain

https://security.googleblog.com/2023/04/supply-chain-securit...

https://security.googleblog.com/2023/06/supply-chain-securit...

Go made static binaries popular, and those solve a ton of distribution challenges.

syncthing automtically connects with bunch of external servers automatically. Relays/discovery servers/stun etc. You can disable but they are enabled by default. They connect on first run and before you get a chance to disable - atleast on android. Everything may be encrypted but I still prefer to not connect to external servers

Yes, IMO the external connections and relays should be disabled by default/opt in for privacy reasons.

I've had to stop using Syncthing as my primary means of file syncing due to either Android 14's scoped storage or the way this OEM implemented it. At least it still works on my backup Android 10 device.

If you use the android app you got to do it via the app ui instead of the web ui, the app ui will ask for permissions to a folder if you select it.

I've tried both methods.

I would like to reflect on all the positive sentiments here. I have been using it for years and I rarely have to look at or think about it more than once every two months. Even when I do need to look at it, it's just to realize I filled some drive and that's why it's not synchronizing anymore.

I have no issues day-to-day and it's relatively easy to set up. I haven't played with the zero trust encrypted folder syncing yet but I will eventually :p

I'm running it on Linux and Android but I started out with it on Windows. Thanks for the reminder about this software. I don't think I've donated in a couple years now.

Whenever I need to share files between two Windows computers in a LAN, I'll use this tool.

It sounds kind of ridiculous, but just goes to show how crappy/difficult the Windows native file sharing is.

Localsend is excellent for all the use cases SyncThing doesn't cover well

I recently set it up for syncing files between Windows and Mac. That results in many sync errors because of differences in allowed special characters in the file names. Anyone having recommendations for avoiding that? Thanks.

Huge fan. Been using it for years with very few issues across Win10, MacOS, iOS, Ubuntu / PopOS, Steam Deck and even Quest 3. Fire and forget. Like Tailscale, don’t know what I’d do without it.

Syncthing + Raspberry Pi + Wireguard + Gitea = There's no "cloud"! It is just someone else's computer.

...+ Nextcloud + Joplin + Jellyfin + ntfy + Lychee + ....

I'm hooked on this stuff; the amount of high quality free software available now it's just incredible!

Related. Others?

Syncthing – A decentralized continuous file synchronization program - https://news.ycombinator.com/item?id=41281152 - Aug 2024 (23 comments)

Syncthing: A continuous file synchronization program - https://news.ycombinator.com/item?id=35879039 - May 2023 (124 comments)

Syncthing Anywhere with Tailscale - https://news.ycombinator.com/item?id=32375662 - Aug 2022 (42 comments)

Syncthing Stats - https://news.ycombinator.com/item?id=32012712 - July 2022 (4 comments)

Syncthing: The Data Deduplication Master - https://news.ycombinator.com/item?id=31278867 - May 2022 (2 comments)

Syncthing – a continuous file synchronization program - https://news.ycombinator.com/item?id=28859521 - Oct 2021 (230 comments)

Syncthing – continuous file synchronization program - https://news.ycombinator.com/item?id=28371802 - Aug 2021 (1 comment)

Syncthing: Syncing All the Things - https://news.ycombinator.com/item?id=27929194 - July 2021 (172 comments)

Open Source Continuous File Synchronization - https://news.ycombinator.com/item?id=27149002 - May 2021 (146 comments)

Syncthing is everything I used to love about computers - https://news.ycombinator.com/item?id=23537243 - June 2020 (159 comments)

Do not use Syncthing (2019) - https://news.ycombinator.com/item?id=23116462 - May 2020 (1 comment)

Emacs' Org-Mode and Syncthing = Perfect (2017) - https://news.ycombinator.com/item?id=23058358 - May 2020 (77 comments)

Syncthing: An open source Dropbox replacement - https://news.ycombinator.com/item?id=20466469 - July 2019 (41 comments)

Syncthing graduation day - https://news.ycombinator.com/item?id=18832517 - Jan 2019 (114 comments)

Syncthing Usage Data - https://news.ycombinator.com/item?id=13856552 - March 2017 (119 comments)

Syncthing - https://news.ycombinator.com/item?id=10331031 - Oct 2015 (1 comment)

BitTorrent backups – BTsync and Syncthing - https://news.ycombinator.com/item?id=10207935 - Sept 2015 (1 comment)

Syncthing: Open Source Dropbox and BitTorrent Sync Replacement - https://news.ycombinator.com/item?id=7734114 - May 2014 (184 comments)

https://hn.garglet.com/similar/stories/27929194

I had a small issue with Syncthing-Fork on Android. The app defaults to having silent notifications so when I shared a new folder from other computer I didn't noticed it on my Android device and I thought it wasn't working because notifications are the only way you can accept and configure new share. Nothing else shows up even though I had the app opened all the time I was doing that.

you can open the web-gui from the menu and the notification should show up there

Syncthing is a wonderful open source project that I thoroughly enjoyed using when my mobile device was on Android. Everything I needed was synced, all the time. Great performance, too. And it was really easy to set up syncing of a particular folder with random people for various projects.

I switched to an iPhone for greater privacy, and was crushed to find Syncthing doesn't support iOS.

I see that 5-6 years later they still don't. Sad, given the iPhone has a majority market share in the US (60%).

I use Nextcloud and it's....okay. There's some weirdness where files I've specified should be always available somehow aren't, but luckily the keepass client keeps an internal/backup copy and is smart enough to open that.

iCloud is just...a rolling dumpster fire. And easily the most frustrating thing about it is that there's no way to set up an iPhone without it immediately trying to sync everything to iCloud, like say, your iMessage encryption keys...

There's a 3rd-party commercial app that brings Syncthing to iOS [1]. It's essentially a paid app, but periodically they'll release TestFlight versions as betas.

[1]: https://mobiussync.com/ [2]: https://github.com/MobiusSync/MobiusSync/discussions/194

> I see that 5-6 years later they still don't. Sad, given the iPhone has a majority market share in the US (60%).

What goes around, comes around. Apple knows they make it hard on developers that want to make the iPhone behave like a normal computer. Same as any Apple platform, you pay the price premium for compatible software and not the other way around. Magic solutions don't manifest on developer-hostile platforms like that, and Apple makes sure of it. After all, they profit off every solution sold on their problem marketplace.

> there's no way to set up an iPhone without it immediately trying to sync everything to iCloud, like say, your iMessage encryption keys...

So much for "greater privacy" then, huh? Or... do we still think this is a negligible fact?

I switched back to android for syncthing, totally worth it. I am on a galaxy s24, the battery life is not great compared to my last iPhone, and the camera takes poor photos, but other than that everything is better.

> I switched to an iPhone for greater privacy, and was crushed to find Syncthing doesn't support iOS.

There is an app that wraps syncthing that is available on iOS but it's still quite gimped[0] due to how iOS does file management.

[0]: https://forum.obsidian.md/t/sync-mac-pc-and-ios-using-syncth...

On IOS there's Möbius Sync, which works perfectly with Syncthing.

You probably should've switched to GrapheneOS instead.

Reply on news.ycombinator.com