I think this is it. I have a single markdown file for all my casual notes (1), and a folder structure for my permanent notes (2). Notes of type 2 are the most valuable long-term, and it's worth the time to select what to store, and to structure it right.
I'm always on the hunt of my ideal open source, lightweight, non-cloud, cross-platform, mobile-friendly hierarchical note taking app, but I know I'm asking too much, maybe I'll never find it. Meanwhile I use plain text editors.