You're going to capture hours of walking and/or seemingly doing nothing, exchanging pleasantries/small-talk/banter. Without access to my thoughts, this is stuck in some superficial layer -- useless other than to maybe surface a reminder of something trivial that I forgot (and that's not worth it). Life happens in the brain, and you won't have access to that (yet).
Curious though: if there were a way for an AI to understand your thoughts, would that even be something you’d want? Or is the whole concept off-limits for you?
It's an interesting question -- I've thought about it a lot in the context of some hypothetical brain interface. There are a lot of unknowns but I personally would go for it with the very hard constraint that it be the equivalent of read-only (no agents here) and local (no cloud).
As potentially scary as it seems, I would not be able to fight the temptation to participate under those conditions. It would make elusive thought a thing of the past.
If it was networked, it would need to have much tighter security than the current internet.
If it was just a terminal to some corporate server running unknown software for purposes I wouldn't necessarily agree to, nope, nope, nopity-nope. Even if it didn't start off as a device for pushing propaganda and advertising, there's no realistic expectation that it wouldn't evolve into that over time.