When you just have to think about buying something, and Alexa orders it for you :O
If the software assistants are sufficiently useful and tightly coupled with the human mind, I think it quite likely that the line between self and software might get blurry for some users. The ability to think a question and hear a correct answer as a voice in your head is the sort of profoundly powerful user experience that I think might plausibly alter the assumptions people make about what it means to be themselves.
If these software assistants become apart of the users' own mind in their own perception of themselves, what responsibilities do the owners/operators of those systems have to their users?
I guess we'll cross that bridge when we get their, but the relative immaturity of FOSS software assistants is starting to unnerve me. In 2040 when Amazon starts selling "god in a box" to the general public, a two-way telepathic connection to a state of the art quasi-AGI living in the cloud, will there be a viable FOSS alternative?
Edit: strange I didn't think about it directly, but that is the Borg from Star Trek.
Toddlers are, today, growing up with iPads, YouTube, and Alexa/Siri as just a natural part of the world. As far as I've observed—admittedly not much—educators and parents are far behind, I would speculate both because grasping the indirection-of-agency that this sort of technology creates is a heavy abstract task of the kind that doesn't seem to filter through those parts of civilization well and because the technology has the ability to change too quickly to counter any attempts to pin it down. And pinning it down in too static a fashion could have its own horrible effects.
“FOSS” in the original sense is largely a distraction here (even though it is still an important idea), given that we've wound up in the “programming is specialized” world. The dynamic characteristics involve how agency flows through systems, and in the presence of highly distributed and often SaaSS (Service as a Software Substitute, as the FSF describes it) systems, being able to alter the source code isn't a solid defense even at “skilled programmer” speeds: going against a rushing current just means you get torn apart as soon as you touch the world. I think we need a new word for what you probably meant but which I don't know how to articulate well.
I guess the tighter coupling than what it is now is what makes the idea repulsive.
Also, as far as FOSS alternative goes, I wouldn't count on it. It's not so much the code - in time it'd be the huge data-crunching that counts, something only big corporations are able to do.