Show HN: TagSpaces – An open source personal data manager | Hacker News Reader

Show HN: TagSpaces – An open source personal data manager

tagspaces.org

361 pointstagspaces83 comments

Looks nice, but for personal use I don't like the idea of a tool like this changing the file names, and then I see the option in the Pro version to use "sidecar files". Nice! It then brought up another question I have often had with projects like these that start out on Unix like systems and are then available for Windows as well: why do they use dot files on Windows and tell the user to hide them manually, instead of using files with a "hidden" attribute? Part of porting should be adhering to the local conventions, should it not? Since this is not the first time I stumble upon dot files in Windows Explorer I am left wondering there is a real reason for it?

Probably it's not a high-priority issue, so they didn't implement it yet. Should be easy enough to implement for someone, who uses Windows, if they accept patches. It's a common pattern, when developers think about their environment first.

Actually that looks like it's a feature they sell. If you scroll down you see mentions of TagSpaces Pro with it's number one feature:

* Ability to save tags in sidecar files

Here is their feature page: https://www.tagspaces.org/products/pro/#tagSidecarSaving

The original comment clearly mentions that feature. The question was why cross-platform applications don't set the hidden attribute on dotfiles.

Even worse are Unix utilities that leave dotfiles in every single directory they touch. Just because they're hidden doesn't mean that they're not dirty! Pleas keep your crap restricted to ~/.config, which exists for this very reason.

If you place something in the ~/.config it is not portable, ones you copy your files and folders to a usb stick or sync them with dropbox, you loose the meta data.

Then the app is poorly designed. It should sync metadata over a different channel.

Otherwise you get inconsistent behavior such as metadata being synced when you copy entire folders, but not when you copy individual files. Then you have to rely on even more hacks to get around that, like detecting when a file is copied and automatically copying the corresponding dotfile. Sooner or later you end up with a complete mess.

Well

1) This is not default behaviour, you have to specifically enable this

2) They actually note that if you copy files outside their app, metadata does not get copied, it's doubtful they'll fix it

So basically, NOFIX, working as intended. Not poor design.

> ~/.config, which exists for this very reason.

On GNU/Linux systems maybe, it isn't part of POSIX.

It does not conflict with POSIX though. Why not do it on BSD as well?

There are more UNIX systems out there.

How will you enforce something like that to all UNIX eco-system?

Convention. The concept of configuration in dotfiles is no more posix than the concept of configuration in ~/.config.

The point was enforcing UNIX applications to use ~/.config.

On the one hand, I wouldn't want my filenames to be changed (and in addition, one would likely run into problems with the path length limit, at least on Windows).

On the other hand, using sidecar files would essentially mean giving up freedom to move/rename your files. Sure, you could move/rename the metadata files manually, but it'd be much more work, and you're bound to mess up sooner or later.

I'm not sure how to solve both those points without making the application "own" the files (as e.g. Camlistore does).

NTFS can have additional data streams associated with a file. It's used for stuff like the warnings you get when you try to run an executable downloaded from the Internet. This seems like a perfect solution for me, at least on Windows.

It's not just the correct way on windows, it's the correct way on macOS, and any other file system with resource forks or extensible file metadata.

However file metadata is usually pretty poorly understood by most developers (based on a personal sample of my past experiences) and file system resource forks are even less understood...

Perfect is the enemy of good and all that sort of thing ...

I would say it mostly misunderstood by those not raised coding GUI applications for Mac OS, Windows, Atari or Amiga systems.

The problem with Alternate Data Streams is interoperability. In many cases, the data will silently be lost when copying between different file systems and/or operating systems. I believe Windows itself will warn you when trying to perform an operation that would discard alternate data streams (for example copying from NTFS to FAT), but many tools would likely silently strip it.

Think of things like any of the cloud drives, bittorrent, screen-sharing or IM software. Are they aware of ADS (or equivalent for that platform). What would the right thing to do be when syncing between platforms that have support, and ones that do not?

Well, this is open source, so I guess this could be added:

https://github.com/tagspaces/tagspaces

Windows even has a standard NTFS metadata field for Tags. You can display it as a column in File Explorer's detail view. You can edit it in the Properties > Details sheet.

Outside of Office sometimes trying to encourage using it to make searching easier and Windows sometimes using it for ID3 tags from audio files, I've never really seen anyone try to use it.

Wow! I've been looking for something exactly like this for a long time. I am a motovlogger (which is someone who rides a motorcycle while vlogging) and I generate tons of small videos that I've always struggled to organize. I'll need to use random videos from months and months ago sometimes and it's so easy to lose track of them when you have thousands upon thousands of small short video clips.

One question/request I might have is if it is possible to attach a general note to a specific file. It's helpful to know what a video is about without having to watch the whole thing.

Thanks for the feedback. We are already working on a feature allowing saving description to files.

FWIW KDE does metadata for files, eg comments/tags/star rating, that you can see in the dolphin information panel and search using their standard toolset.

(You can see the add comment "link" in the images here https://userbase.kde.org/Dolphin/File_Management#Information .)

I like it (even if it is a js app), but I don't like how it renames my files, especially without asking me if that's alright first.

It broke several other scripts and shortcuts I had due to the file name changing without me knowing.

Camlistore is another open source project which works well. It's a bit more technical than this one appears to be.

I love Camlistore but have been hesitant to use it. They're just doesn't seem to be enough tracking behind it.. if more people were into it, i would be really happy.

Instead of renaming the file, TagSpaces could create a symbolic link and preserve the original file name. All such links could be managed in a TagSpaces folder.

> ln -s ../fluffy.jpg "fluffy [cat].jpg"

Tags could be added by linking to the latest one:

> ln -s "fluffy [cat].jpg" "fluffy [cat] [yarn].jpg"

This would allow TagSpaces to have temporal knowledge as to when tags were added or removed.

If the links are cascaded as I have shown, then when the original file is moved, TagSpaces will only need to rebuild the first link. The rest will still work.

The issue of the tags causing the name to be too long can be addressed by a different tag naming strategy.

> ln -s ../fluffy.jpg "fluffy [+cat].jpg"

> ln -s "fluffy [+cat].jpg" "fluffy [+yarn].jpg"

> ln -s "fluffy [+yarn].jpg" "fluffy [-yarn].jpg"

This would require TagSpaces to have some simple logic to determine which tags were relevant at a particular point in time.

This is great idea, but is very Linux/OSX specific, since TagSpaces is multiplatform application, this proposal will not work on Windows or Android platforms.

That system won't work for very many tags. Unix systems have limits on the number of symbolic links you can resolve when trying to access a file. I can't remember the limit on GNU/Linux off the top of my head but I'm fairly sure its something ridiculous like 32 links.

Thanks for pointing that out. Instead of the endless cascade, just cascade once off of the first link. That way, there would never be more than two references to resolve and you still get the temporal benefit.

> ln -s ../fluffy.jpg "fluffy [+cat].jpg"

> ln -s "fluffy [+cat].jpg" "fluffy [+yarn].jpg"

> ln -s "fluffy [+cat].jpg" "fluffy [-yarn].jpg"

The screenshots look nice, but they only contain a couple dozen files and a similarly small number of tags.

I've got a good fraction of a million files on my home partition, and I'm likely to end up with hundreds if not thousands of tags if I were to go about organizing them. I'd like to know whether this tool will work on such a scale without inconvenience. Too many similar tools fall apart once you hit a hundred tags or so, not to mention a few hundred thousand entries. It's a lot of wasted effort to start using a tool like that, only to realize that it doesn't scale.

I think git-annex's metadata[1] support and metadata driven views[2] aim to solve a similar problem.

[1] https://git-annex.branchable.com/metadata/ [2] https://git-annex.branchable.com/tips/metadata_driven_views/

I wish there was a git-annex metadata browser and editor like this.

I've always wondered what would happen to an OSS project if someone sat down and implemented all the PRO features and PR to git.

You expect to type PRO and have people know what your talking about? I certainly don't, and unless you are talking about the wikipedia entry https://en.wikipedia.org/wiki/PRO_%28category_theory%29, which it doesn't sound like, I don't think it's reasonable.

They mean the Pro version, as in – the paid version of the app. I think only the free features are open sourced.

It was obvious to me even before I read the article (which has a "pro features" section).

Usually it takes a large amount of dissatisfied, paying customers nagging you with bug fixes and feature requests to eventually build a quality pro product.

However, by the looks of it TagSpaces seems very polished and useful right off the bat.

As tags end up in file names how can you do backups with this? rsync creates duplicates when tags are added since filename changes

I believe -y or --fuzzy would help with rsync. At least if the file contents hasn't changed.

The pro version offers saving tag in metadata instead https://www.tagspaces.org/products/pro/#tagSidecarSaving

But it won't recognize if an external tool moves the file

I'd much rather it stored it in a central "database" of some sort, but I'd also want automatic detection of renamed files.

That is what we do at http://www.cloudfindhq.com, picking up the renamed file issue from the underlying Cloudstores was quite a challenge to implement. A lot of subtleties to get right with not always helpful API endpoints.

A separate tool which helps with problems like this: rdfind

It will let you create hardlinks between identical files to reduce disk space usage. You will still have two files, but the impact will be reduced.

Use a backup system with deduplication like Tarsnap, ZBackup or bup, then identical (and even just similar) files will occupy the same space.

I assume only the regular version (non-Pro) is open source? If not, I'm really curious how this works together with Pro version being paid.

Looks like the license is AGPL so that nobody can clone them and do a similar 'pro' version.

Now, when you contribute to their OS project you need to sign a CLA which pretty much consists of the usual 'all your base are belong to us'.

  You hereby grant to the Owner and to recipients of software distributed by the  Owner a perpetual, worldwide, non- exclusive, no-charge, royalty-free, irrevocable  copyright license to reproduce, prepare derivative works of, publicly display,  publicly perform, sublicense, and distribute Your Contributions and such derivative  works.

Looks like the license is AGPL so that nobody can clone them and do a similar 'pro' version.

Well, they can, they just have to ship the source to the buyers (and no, they don't need to give it to non-buyers).

But that source would also be AGPL and a buyer could share it with the rest of the world.

But the odds of someone except the original developer wanting to work on a paid version are low anyway.

The contributors have to sign the CLAs only if they want to contribute to the core of the application. BTW, all the tagspaces extensions are licensed under MIT, so here no CLAa are required.

Funny how sometimes you think "You know 'x' would be really useful, I would use an application like 'x'." Then the next day its the top link on Hacker News.

Or worse: "You know 'x' would be really useful, maybe I should build that." Then you decide to let it rest for a few days (most of my projects have been bouncing around in my head for a while before I have a good idea of what product I want to make) and the next day it's on Hacker News.

Tagging is an old one for me. I settled on using tmsu but never actually got around to use it. The photo organizer sounds like something I should be doing, I start losing more and more photos. Problem is, Tagspaces either renames my files or leaves hidden directories in every directory (what is this, OS X?). Guess I might give tmsu another spin or, if that isn't satisfactory, write my own after all.

Sounds like these guys https://cloudfindhq.com/ expect they don't modify the file name

Yeh, come and check out Cloudfind - we have just launched a new product that integrates nicely into Dropbox and Google Drive - free to use for personal and small team use. Lets you organise the tags into logical groupings to help you organise your stuff.

I notice the free version only allows 100 tags. I'd hardly have more than that but it might be a limitation some people should be aware of

I actually would not mind having a simple database (that is cross platform like SQLite) which would act as a separation layer from the files (and store any tags, and minimal meta data)...only because I'm not crazy about any system changing file names on me. Call me old fashioned ;-) That being said, the UI - especially the editor - looks pretty neat.

I've seen comments for git-annex, but the lack of a neat gui/browser gives me pause. Can anyone recommend an alternative to Tagspaces but which does not alter filenames...and is of course cross platform (because i work with a couple different machines every day), and is open source?

Tagging the filesystem and making a personal-cloud manager is very similar to one of the directions I'm headed in with meta64.com, which is built on the Java JCR, Lucene, Solr Cloud, Polymer, and is a sort of tree-structured repository that does lots of these types of shenanigans.

I have to say that it would be nice if the sidecar functionality was available in the free edition. I don't want to try the free version because I'm unsure of the effects that renaming will have. I'd be afraid of losing potential customers because they didn't know about the renaming. I will still buy the Pro version because I definitely need somethings like this and this looks like a nice project to support.

p.s. is it possible to upgrade to Pro Unlimited if I like the Pro version?

Most people aren't very diligent with naming their tags and organizing their data. I am definitely guilty of this.

Are there any data management services out there that can use some kind of semantic understanding to let you search through your own data? In essence, "google" my own personal collection of data? For example, I might make quick notes throughout the day and tag them with vague or maybe even incorrectly spelled tags. Can I type something like "note I wrote yesterday about dogs" and get correct results?

Cortana / Siri / Google Assistant?

Seems to be a place all of them are trying to head, with varying degrees of success thus far.

I'd be more interested if this integrated with macOS tags.

Ponies aren't free. Review their contribution guidelines and create an issue. Just saying, lamenting a missing OS-specific feature is a big ask.

I didn't say I wouldn't pony up. I'm offering feedback which I hope may be useful information.

TagSpaces aims to be a cross-platform tool, implementing such a feature will be very restrictive for the use cases supported by the application.

Cross-platform doesn't have to mean lowest common denominator.

When I hear "X is impossible", I like to reframe the issue. Instead of making a statement, why not ask a question such as "What would make X possible?" I write more about this at https://medium.com/@xpe/reframing-the-impossible-42c588abcba...

Is there a trial for the pro-version. In case somebody wants to test the sidecar functionality, full text search capabilities...

I've been using TagSpaces for a while (mostly ass desktop note taking app). Nice project, but went back to Tomboy for two main UX reasons:

* no ctrl+n for new file

* no auto-saving of the file itself (I have to say last version I've been using was 2.1.1. I've downloaded / updated the software on my computer and will check this soon (apparently is missing)

I have been looking for something similar to this that would allow me to also share the files with other people using a group permission structure per file. Has anyone seen something like that?

Am I the only one who read this as TabsvsSpaces?

So, for someone using only macOS/Mac OS X, how is this an improvement over the built-in file tagging?

Other operating systems don't posses the easiness of the tagging system build in OS X. The issue we see in the OS X tags is that they are locked to a particular operating system. Ones you copy/move your files on some other other platforms or the cloud you are mostly loosing the tagging information. BTW, TagSpaces is not only for tagging, we use it for example as a Evernote alternative.

So, given that my question was:

> for someone using only macOS/Mac OS X, how is this an improvement over the built-in file tagging?

The answer is "it doesn't"?

I'm not saying it doesn't have a market, I'm trying to understand if it offers anything above the OS included functionality for someone purely in the Apple ecosystem.

The title I got in my RSS reader was "File manager supports over 100 file extensions(Open source)." I was expecting a recent discovery of something from like the 60's being the earliest known document using the term "Open source." Shining example of the over-overloading of terms in tech.

Why organize files by massive manual tagging if you can organize them with a simple system based on conventions [1]?

You can predict where something is most of times.

[1] https://github.com/galfarragem/hamster-gtd

In order to organize files with tagspaces you could use tags, but is not mandatory. You can still relay your local file and folder structure, which is displayed in the application. In a way you can use any conventions with TagSpaces, without a bounding to any file sync cloud service or Evernote.

I've been wanting to build something like this for about a decade but never bothered, and this looks so much better than anything I could have built. Thanks Ilian!

That must be the slowest webpage I've ever used.

From the comments it sounds like I'd prefer the way git annex handles tagging.

Really fast for me, here in the UK.

It's still loading...

Hah, it's sitting there waiting for several fonts to load from fonts.googleapis.com, but I have my browser set to ignore any font choices set by webpages. Apparently it's just going to leave the dom empty forever.

I have fonts.googleapis blocked in my hosts file, loads fine for me in Firefox. Maybe it's your browser? Or the method of blocking?

I checked that last night, actually. A traceroute showed packets winding their way through comcast's network, hitting google's, and then vanishing. Whatever the problem was, it's been fixed. The moral of the story is to never rely on third-party assets, especially for synchronous stylesheets that block the browser from even parsing the rest of the page.

Reply on news.ycombinator.com