Yeah, apps wouldn't be able to only listen to the firehose.
There are some proposals for private files. However, I'm outside the AtProto world so not sure what exactly the suggested implementations are. I just hope they give enough control.
I think the technology could potentially be used for way more than microblogging. I would love to use webapps that store the data on my devices and share it with specific people. The data and access under my control.