I was like you, until my library grew to a certain size and I needed/wanted to organize it and display it better. Calibre lets me tag books, so I can pull them up by arbitrary categories which I can't EASILY pull off only using filesystem features. With Calibre I split my ebooks into two libraries (one for computer books, one for not computer books) and tag to my hearts content and requirements. I keep it simple to 2-3 tags for each book so I can later find my computer books that are "python, security" or my "reading challenge, crime/mystery, unread" books without dealing with extended attributes and the find command. Calibre also travels between linux, macos, and windows and so do the tags; unlike the filesystem where I'd get the joy of retagging all this stuff if I am on a different system (say, copy of my stuff on Windows means I have to retag with NTFS alternate data streams).

Same thing with pdf's - I used to keep a bunch of docs in my own subdir, until it got to be 1000+ pdf's and I discovered Mendeley Desktop and its tagging feature. Now I can find stuff much faster.

Same thing with my music collection, same thing my with photos.

Your needs may be minimal enough and your computer skills sufficient enough so that you get by with the filesystem, but I would venture to say the average user is much better off with an app/library method of organization. e.g., I can't fathom organizing a non-trivial photo collection via the filesystem only. I know I'm better off with the app/library method of organizing data, something I came to realize after my initial resistance.