Before taking off on this endeavor, I want to make sure I have a good system in place for where things go.. I have thousands and thousands of various folders and files, things like projects, music, photos, zip files and other downloads, personal notes, backups, etc..
How do you organize your digital library?
Media - An auto-backup of photos, videos & music is what I like. Use google drive for cloud backup. Have a premium account to accommodate for all the space needed
Notes & Documents & Writings - Public ones - Already part of the internet space. The rest are really things which I can afford to lose. They are like a closet which I don't really mind cleaning out once in a bit and losing it.
Work - Tax filings & income documents are in a digital private locker. The rest of it like work related repositories, code and other stuff. Either open sourced or in private git repos.
Movies & Books - I used to collect things like movies, books and then realized that could just rent and buy them over so no more into hoarding the stuff here.
Best advice, I can give after trying multiple organizational systems is constant pruning and sharing out what you can in public domain.
- Have one obvious spot to put things. For example for photos: have one directory for all of them, and make subdirectories there by either the device that took it (I remember better whether I took a photo with device A or B than whether it was in 2017 or 2016) or whatever else you prefer. Use broad, obvious categories ("photos", "audiobooks/English", "audiobooks/Dutch", "games/downloaded", "games/self-made", etc.) for the main folder names.
- Don't let data get stuck on old devices. Keep track of in which folders you've stored data and, once those are copied over, give it e.g. a few months. If you didn't need to go back to the device, wipe the device. Don't be me. I have a phone from 2012 that I still haven't finished migrating.
- Have backups. Make making a backup easy. Just take the whole disk. (Games will store your save files in places you didn't know existed.) Mark the backups as backups, so you don't have to think "eh, am I going to find any original data on this 2008 hard drive or was it just a backup copy I can wipe?". Don't be teenage me.
- When in doubt, use year-month-NAME for files and directories, then the rest of the name matters a lot less as you're likely to remember the year and season that a given thing happened in. (System timestamps are prone to being lost upon migration, or get updated when you fix a typo while reading.)
- Folders can have an 'archive' subfolder for things you may need one day, but that you aren't likely to ever touch again. I used to keep one folder per school/internship/employer, and it got a bit much. Moving old things into an archive subfolder helps and I don't have to feel like I will lose the data (it's not like I marked it for expunging).
- Have a temporary folder whose entries are prefixed with an expiry date. This contains "just in case" files, like a copy of my Signal messages database before I start messing with the database (I'll know latest in a few days that I didn't need the backup). I also put call notes here, like when someone calls me and I want to make sure I've saved the file but don't have a project folder for it, but it might turn into a project, I could put it here with some months' expiry. If nothing came of it, it can be safely deleted and I don't have to review the contents to know whether it's irrelevant.
The problem is that this doesn't scale for immediate or extended family members. I guess the realistic option is for them to pay for cloud storage indefinitely, but that's not really ideal either.
> For a couple of years, I taught PIM in a lecture at the St. Pölten University of Applied Sciences. My students are surprised, how inefficient they were doing even «common» things like handling email, managing files like photographs on your computer, or searching for information. It is so rewarding to see students realizing how much improvement is possible with some background-knowledge and a few hints here or there. Starting with 2021, that lecture started at Graz University of Technology.
> I hereby claim that I am able to save you at least twenty minutes of your computer time a day just by looking over your shoulder and suggesting more advanced tools, better methods, and efficient workflows.
I just brain dump what’s most important in the morning and triage my work. Then get to it. If something comes up instead of reacting immediately, I paste it to the TODO list and maybe a short note.
Otherwise I’ll just work on first thing that comes to mind, or first message I react to, not the most important.
- Calibre to manage my offline collection: In case the Internet goes down or some site disappears
- Zotero (Papers and books to learn from)
Images:
- Apple Photos: I only take pictures with my iPhone
- Screenshot: Automatically goes to my desktop, which is synced with iCloud.
Movies:
- I have a old Mini that I use as a file server. Everything is there
Documents:
- PARA method (I barely produced documents on my personal devices, so I haven't upgraded anything)
Music:
- I have an offline FLAC library, but streaming is more convenient right now. I'm using the `Collection/Artists/Albums` folder pattern.
There's not much else, I only hoard books (and my favorite albums). Anything else can be deleted.
It's tempting to design the perfect system for an archivist, but you might realise that it's too much maintenance for a regular guy who has a life.
Personally, I put my photos on a timeline, and the rest in a reasonable folder structure. That's about it.
I use Digikam[1] to do facial recognition and tagging on them. It's finally gotten to the point where it doesn't crash all the time writing metadata, and the facial recognition is acceptable. It's nowhere near as good as Picassa was, but the last release had a horrible bug that sometimes swapped face tags if there were multiple people in a photo. Those would them lead to bad training data, and it kept getting worse.
At one point in time, I JPEG2000 compressed all my photos to save disk space.... it was a huge mistake, and now I've got to try to recover the original jpegs from CDs and old backups.
I have photos of my kids (who look similar, obviously, I guess), going from age 0. The face apps all get confused, and often match younger kids, and older photos of older kids, and fail to match younger and older photos of the same kid.
Do any of the apps use date-of-photo to help guide this?
I am fairly certain that the dates on photos are NOT used.
If you're not an Emacs fan and you din't want to be, Logseq is another tool that looks like it can be used in a similar way.
I used to have lots of subfolders but found that it was easier to just prefix with the date name and have everything flat so I can search in a single directory for things like the washing machine receipt from 8 years ago or whatnot.
Then I have diary files with text descriptions that link to the docs.
For work, I use obsidian with lots of notes in markdown notes linking to files with deep nesting structures for projects. I have more time to organize and work pays for OneDrive that seems to find things better in deep folders.
I have a todo that I manually roll over from day to day to give me a chance to prune or revise.
I have a standing wish to explore PARA or some more systematic organization scheme but for now just my own ad how structure works. No one else sees it.
I'll just give examples of folders you might find on those drives.
- /Applications/Audio/Renoise/
- /AppsPortable/Images/Krita/
- /AppsData/SublimeText/ <-- project files go here
- /Drop/Downloads/Browser
- /Drop/HTTrack/SiteName
All "my" stuff (mostly things I made or personal documents, or things I curated for personal use) goes into a /Docs/ directory structure, which is similarly straightforward:
- /Docs/Projects/NameOfProject
- /Docs/Resources/Icons/Objects/Tech/joystick.png
- /Docs/Images/Created/ <- everything I made that's an image but not a photo
- /Docs/Images/In/Canon50D/012 <- I just increase that by one each time I dump my CF card, and then take my sweet time to sort that
- /Docs/Images/Out/Pub/2023/ <- uploaded RAW photos named [year]-[month]-[day]_[hour]_[minute]-[second] go here, in case of events or vacations I might make a subfolder for them
- /Docs/Images/Scanned
I also use synthing extensively, so there's a subset of that structure for things I want to have on my phone, too.
- SyncThings/Thang/Audio/AudioBooks/
- SyncThings/Thang/Audio/Music/
- SyncThings/Thang/Audio/Talks/
- SyncThings/Thang/keybass.dbx
etc.
Contacts I keep on a little Baïkal installation, and thanks to DAVx⁵ that means I can edit my phone contacts in Thunderbird, which is so awesome.
I don't want to make this post go on forever, but FWIW I made good exsperiences with going overboard in this way: because at this point, I know where any given thing "belongs". Sometimes I even download something, put it "where it belongs", and see it's already there. Of course I also have plenty of unsorted files here and there, but knowing I could sort them rather easily is nice.
IMO first you want to:
1. Gauge how likely you are to need things, and to split them apart on that basis. Don't bother organizing stuff that doesn't (yet, perhaps ever) need organizing.
2. Determine which things you can filename/text-search for if you need them, versus which things _need_ a good folder-organization because they are not easy to index or find metadata for.
I feel no shame having my MIDI collection from the 90s buried somewhere inside "Old Music" if I never really end up looking at it.
https://f-droid.org/ja/packages/com.github.axet.smsgate/
Also there's a play store version
I haven’t reverse engineered the schema for MMS so I lose photos. I don’t much care but that would be something to do for completeness— adding the photos to the messages as MIME attachments.
Who may have this knowledge you may ask?
https://en.wikipedia.org/wiki/Library_and_information_scienc...
Literature, I use UDC which is a Dewey Decimal variant that fixes alot of what's wrong with DD. Audio, Video, and Images are Plex.
The "not-named-well" directory still gives me trouble years later, and the concept makes sense to me. I keep (food) recipes in it, as a subfolder, but also stl physible files. And though I've never managed to find any anywhere, if I do, I'll keep house building plans there. I even spent an hour or two the other day learning what those would look like (I was surprised that plumbing plans are never included, and that most places charge extra for the materials list). It was just after the Sears home kit story hit here on HN, felt briefly inspired.
No point in hanging on to anything you don't use at least once a year.
I have many documents that I use extremely infrequently but will need eventually.
Marginal cost of storing 1,000,000 files over 1 is near zero. If I have to keep track of one file, I may as well keep track of many.
When’s the last time you looked at any of that stuff? If it was all deleted today, what would you honestly miss most? Save that and forget about everything else.
Which area will documentation on project management go to?
Attempts to organize complex data using exclusive taxonomies, in my experience, never works. One needs at least something like tags.
For file organization, I tried to organize so many times but it was too big of a job to start. So I finally decided to do it by year. I have a documents folder with folders such as "2023". Inside that I simply make a folder for anything such as "taxes" or "CV". For some years everything is in one "unsorted" folder. It's low friction to throw stuff in there as you find it, and not too overwhelming to sort through one year at a time. It tends to be fairly easy to remember what year things are from later.
I have a limited number of exceptions that make sense to use different organization, for example a top level folder called "gitlab"
My biggest piece of advice, as someone who struggled to get organization done for years is to be very deliberate when moving stuff and delete things from the old location as soon as you've confirmed the move and backed it up. Use a tool like rsync that can give you piece of mind that nothing was missed. If you don't delete it, you'll end up like me and finding duplicate files later and wasting time making sure it's really a duplicate before deleting it.
Same goes for backups. Put your old ones on a drive and put that in a closet or delete them if you're sure you didn't miss anything. Going forward, you'll keep your new system backed up and you don't need the confusion of nested old backups-of-backups later.
[1]I also have some of these documents in google drive (from before icloud was good), some on a file server (from a period of a few years where I thought I wanted to self-host everything). It's on my to-do list to get everything into icloud only except for files which are too big or copyrighted, then figure out a good system to mirror icloud to my file server so that I am back to having one single drive to backup and trusting that everything is there.
Say, I back up the home directory in my computer to a NAS. The computer dies at some point. I get a new computer and recover home from a backup (or part of data in the home directory). New data is added to the home directory. New computer is backed up to NAS. Over time, most of the data in backups are the same.
Any idea how to deduplicate, or avoid this problem?
Files in nested folders. Symbolic links, when a file or folder does not fit into a single category.
- Photos (shared via iCloud)
- Notes
- Music
- iCloud for file storage
- I also use google docs for spreadsheets etc.
This pretty much covers my digital life and mostly meet my requirement to be on auto-pilot.
tools:
vim, rg, fd, fzf (ff as wrapper on Windows), sometimes ctags
Concepts learned with books:
- Make it Stick: The Science of Successful Learning
- How to Take Smart Notes
- Building a Second Brain
Can be found here and some more:
https://github.com/vbd/Fieldnotes/blob/main/booklist.md#self...
immutable backups of HOME in s3[1].
usb drives with encryption keys and s3 creds stashed all over.
always setup new pc/os via backup restore with usb drive keys.
every new pc/os gets a new backup location.
My guess is that my answer will not be common or popular with the Hacker News audience!
(1) Emphasize simple text. For handling the text, use my favorite text editor KEdit with 100+ macros in the editor's macro language Kexx.
(2) Keep data in files in directories, and use the hierarchical file system to construct a taxonomic hierarchy of the contents of the files. I have no files in any cloud. I'm shopping for a smart phone but so far don't have one.
(3) Use text windows with command line scripts written in the interpretive language Rexx. In each text window, run a simple Rexx program that acts like a shell. The editor's macro language Kexx is a version of Rexx.
So, to do some piece of work, I start with a text window, use some tree walking scripts to get to the relevant directory, start KEdit with that directory the current one, and then get to work.
Each directory has a special file that has some basic documentation of the contents of the directory.
(4) Have a file FACTS open in KEdit and with some useful macros. When I see a fact I want to remember, I put an entry in that file with some appropriate keywords. Since my use of the hierarchical file system does not always have an obvious, unique directory for each file, for an important file I put its tree name in an entry in FACTS.
Over the last 8 years, have put on average 3.66 entries a day in FACTS, and on average an entry has 566 bytes. So currently FACTS has 4,414,649 bytes, and KEdit finds things in the file very quickly, even without using the key words.
(5) As a last resort I run a little Rexx program that puts all the tree names in my most important drive letter into a file. That file has 230,159,949 bytes with 6232 lines. KEdit can read and search this file quickly.
(6) I do file system backups using carefully selected options of ROBOCOPY.
(7) My most important directory is DATA05. That name is reserved so that my scripts and macros can work with the tree rooted there independent of the drive letter. So, drive letters can change, and I can essentially ignore the change and still just keep on working.
(8) Due to a disaster, I had to rush out and buy a laptop. It runs Windows 10 Home Edition. I'm working to get back to the desktop I built with an 8 core AMD processor and running Windows 7 Professional. I like that version of Windows and see no reason to change, but I do intend to plug together a server and there run Windows Server 2019.
(9) For writing important letters, math, other technical material I use Knuth's word processing software TeX. I have about 100 macros I've written in TeX, e.g., for verbatim content, cross-references, putting annotation on figures, .... I do not use LaTeX. The TeX distribution I use came with a spell checking program Aspell -- I like it a lot; it's my main means of spell checking.
(10) For Windows and Linux, I wanted to emphasize exactly 1 and selected Windows. For software development for my startup, I program almost entirely in the Microsoft Visual Basic .NET (VB.NET) with ASP.NET, ADO.NET, etc.
From my imperfect memory, the book by Kernighan and Ritchie claimed that the programming language C had an "idiosyncratic" syntax. And, then, similarly for C++ and C#.
Supposedly at one time there was a source code translator to go, either direction, between C# and VB.NET. So, the semantics were the same and the only difference was syntactic sugar.
E.g., I wanted something like Redis so wrote it quickly in VB.NET using two instances of a collection class. The Microsoft documentation for the class seemed to be essentially the same for C#, VB.NET, etc. So, I guessed that, really, C# and VB.NET were, at least in practice, essentially equally powerful.
To me VB.NET has more traditional syntax, more like the original Basic, Fortran, Algol, PL/I, Pascal and is easier to teach, learn, read, and write. So, I like VB.NET.
If I have to write some C#, then I will. Similarly for C, C++, Python, etc.
(11) To write software, e.g., in VB.NET, I just type it into my favorite text editor KEdit. I've never used an IDE (Integrated Development Environment). Once I tried but gave up in a few minutes.
The software I write has a lot of documentation, and some of that is in the same file as the code. So, to me, in the file there is the documentation and the code; given one of these two, with some work, can try to reconstruct the other; and the easier direction is to read the documentation and reconstruct the code -- that is, having just the code and then trying to reconstruct the documentation is harder. Then, writing the documentation is more like other writing in a natural language that I do with a text editor. So, I just use a text editor, not an IDE.
One final point: Sure, early in my career, I eagerly learned all the algorithms, program languages, etc. I could so that I could do the work of employers. Now that I'm doing my own startup, the technical tools I use are the ones I need for my startup. Then, so far, for my startup, I see VB.NET as just fine and see no reason to spend time learning other languages.
RSS feeds
saving:
documents/subject folders
pictures/subject folders or
pictures/yyyy mm dd subject folders
videos as for pictures also youtube channel organized chronologically