Comment by dcz_self | Hacker News Reader

Comment by dcz_self | Hacker News Reader

Parent Full thread

Too many ideas given the time, but now I'm focused on two:

1. Building an open source bicycle computer. It's been over 20 years since they arrived at the scene, and there still isn't one that you could hack! An outrage! No published sources yet, I'm getting stuck on not having much experience building physical things.

2. Getting rid of the directory hierarchy. I have 10K photos, 30K emails, 10000km of GPS tracks, and 10 years of chat logs. Why can't I find anything among them? I own a computer, after all. I have 10 folders called some variation of photos/Cologne/2020/flowers! Having to organize them myself is tedious and a fool's errand, so I'm leaning towards using a database as a file system, to let me just query for files. Geo queries using a map? Yes please. Selecting bounds on a timeline? Oh yeah!

Turns out I'm not alone, Microsoft tried this with WinFS, and failed. But the idea lives on: https://www.nayuki.io/page/designing-better-file-organizatio...

3. Writing. I hope I can find the time to expand on the above on my blog.

I've been working on 2. since January.

All your photos, chats, e-mails, messages, health data (eventually all your data) into a single database, which can be distributed onto multiple removable 'volumes' (disks).

Plug in the volume and the db index is seamlessly merged and queries run on all the connected volumes. No files. No cloud. Data on your disks only.

There are processing nodes that run in the background indexing the content, finding people and locations in the pictures, chats.

It allows you to build up a fully search-able archive of all your data, which you can physically keep in a locker or safe.

But also, if you store and keep all your data on disks that you control, then many of the services that we use could just run as local apps on your server, reading and writing from your archive via an API.

Social apps are just two people's archives sharing content between each other, no 3rd party services required.

I worked a lot on it until about 1.5 months ago, when I got discouraged by how hard and big this project is. Breaking up with my girlfriend around that time did not help either.

But now I see you mention it and it's kind of inspiring, since I've been doing a lot of thinking and gathering strength for another crack at getting a prototype out.

It would be interesting to see what other people have been thinking/doing in this area.

But since there's at least one person who wants it, I might actually jump back to work ;) Thank you !

Do you have any writeups/examples in the public?

I am on purpose not addressing the exchange of information between people – never considered it, no vision, would just confuse me.

I'm also not so keen on indexing. I'm envisioning this more as a file system layer, where data is added to the database as it gets saved. Ideally there would be no way to access the data outside of the database (that would cause desyncs and the need to reindex stuff). We don't need no POSIX file systems ;)

I can empathize with this to some extent. I realized that homebrew is too slow and bloaty for me, so I decided to write an alternative from scratch (also meant as a learning project).

Eventually got discouraged by the sheer amount of real-world workarounds and corner cases that need to be accomodated rather than some inexplicable technical hurdle. And broke up with my long-time girlfriend.

Now I have little desire to go back, and things just feel meh. I play basketball sometimes to keep my sanity.

For your second point, you might want to have a look at Perkeep: https://perkeep.org/

It supports the kind of queries you are after, but it’s a bit of a passion project and development is slow. What’s already there works well though.

Thanks for confirming. I intended to check on perkeep, but my frustration never reached that point before.

I hope it can be adjusted to my needs: I expect I'm going to have to find solutions for tracking files' sources, for snapshots, and for remote/removable storage, in addition to all the UI work.

The included web UI is quite nice (they really should include a screenshot on the website), and setting up backups/remotes is very simple, everything is content addressed so you can just copy stuff without fear. Snapshots I suspect wouldn’t be too hard.

Have a look at Bangle.js. It's nominally an open source smartwatch, but the difference between that and a bike GPS is just a handlebar mount. v2 has just gone on Kickstarter - I've ordered one specifically for bike mounting.

Nice, thanks! Now I need to reconsider. My goals were to have 180+ days battery life and wired connectors, but maybe no one really wants those.

If you do go for Bangle.js, ping me if you get to the stage of putting any code up on github - I'd be very interested.

Would you be interested in cooperating on that? You can ping me over email (see my profile or the blog).

My second impression about the Bangle.js is: "oh no, JavaScript only?". Still, it's much better than building the same thing from scratch, except it may still take effort on the software side.

Having another interested person involved would be amazing for my motivation! Btw, one of the reasons I wanted hackable hardware was to map road surfaces on OSM ;) GPS alone doesn't add much to OSM in my area any more.

Maybe the answer is tags/attributes for filtering, instead of nested directories? Could be more tedious but given the image recognition ArTiFiCiAl iNtElLiGeNcE maybe it could be easier

That's what I'm thinking about too. Most directories are describing what's already available for the computer to extract: the place of something being created, the time of it, the source where it came from (e.g. who sent this as email attachment). Only a minority needs to be explicitly added, like emotional meaning ("my favorites"), or external organization ("experiment 1", "driver's licence"). While we can't use the computer for the latter, it's nonsense that we leverage computers so little for the former.

I think it is. Directory hierarchies are trees, tags are graphs. Trees are great when you need to optimise the physical retrieving time, but don't map well to stuff like pictures. For example, tags of who is in the picture are way better than making folders by person. Same thing for places.

StagesCycling Dash L50/M50 are open to hack: bootloader is not protected, SD card can be replaced, and ssh access is open (when device is connected by USB).

Do you have any links confirming that? I'm failing to find any resources.

https://www.reddit.com/r/linuxmasterrace/comments/ehn19r/how...

How much would you pay for a program that would help you do 2 way more effectively than just by querying it?

That depends on how easily I could compile it, and how much more effective it would be than querying. At the moment I can't come up with anything better than queries.

I'm willing to part with some money to have someone implement it for me if I see there's demand.

Queries kind of suck when it comes hierarchy, do they? And you sure have some folder hierarchy, even if you're not happy about it?

I identify the hierarchy as the source of my problems. I don't need hierarchy, and when I use it, it's because that's the one tool I have. There's no hierarchy between time and place, and I shouldn't be forced to establish it.

There might be some hierarchy between tags: PhD/laser experiment, PhD/dissertation, but there's rarely a need for it to be more than 1-2 levels deep, so I prefer to suffer the pains of inadequate hierarchy rather than inadequate queries. Or at least I would if I could try it out.

I was mostly referring to the fact that filesystem kind of enforces hierarchy, so you have it even if you dislike it. When I'm sorting out the mess that is my hard drive I defenitely value information about hierarchy overlap (due to (incomplete) copies), but I still have not found a tool that would help me an I can't imagine a query language that would be helpful.

I'm not sure about a file system enforcing hierarchy. Historically, they didn't even support directories at first.

Which use cases would a query language fail to help with?

Hierarchy is the only tool it gives, you can't really have one dir with all your files, unless there's very few of them.

My typical scenario is multiple copies of something in progress, like traveling with friends, copying photos to each other phones on the way and then downloading the whole mess to one computer. Query language would give me precise information for every file but it's overwhelming.

I want the opposite for 2.

Keep all my photos in the directory hierarchy, but have a tool to sort them and sync them between devices

If I sync it with rsync, I might accidentally delete important files. Git would sync properly, but then it keeps the data in the repo rather than the directory hierarchy

I presume you've checked out the likes of Lightroom for photo organizing? Though don't let that put you off something better!

Lightroom is closed and comes from Adobe, so it was never on my radar. Being able to modify the software is a must: photos are only part of the problem, so any solution must be open to integration with other sources of data.

It would have been interesting to see how it organizes the UI, but there's no demo version for Linux :P

Fair enough on the closed source thing!

My take is that we have specialist requirements for photos that are probably better served by a photo specific UI. Lightroom's doesn't inspire me to say it's great, but it's adequate. Can search by metadata like camera or capture date, save previous search results as 'smart folders' (e.g. I have one for all pictures since I last had a book printed). It indexes thumbnails for quick scrolling. My ancient version doesn't do geosearch but maybe later ones do. You can clone pictures (like a hardlink) rather than copy into different albums. You can also clone pictures but apply different non destructive edits to the same source file. Almost all the edit operations I want are close at hand. Order is preserved in albums. Rate photos 0-5* and tag with different colours. As to how all that is organized, I'm sure it could be improved on.

I've also yet to see a good photo solution for my parents who (like many non tech folk) still struggle to understand folder hierarchies and where they saved stuff. Picasa used to be ok-ish but discontinued. Google photos is a bit of a pain over their ADSL.

Good luck with it anyway!

Thanks, that's a great summary. It does seem it's most of the way there already. How does it handle removable storage?

Ironically, the fact that a topic-specific UI is needed for topic-related tasks is another strike against closed software. Something open would have been a starting point for other sorts of views (email? PDFs?), but when there's no modification, there's no ability to do that.

Import dialog for copying SD card to wherever you keep your photos on disk. Different dialog for adding photos to the db without moving the files. I don't know how it would handle having your library on something removable, haven't tried.

Reply on news.ycombinator.com