First, I think this is an interesting question. However, I would recommend not asking directly about a work issue in a public place. It's kind of airing dirty laundry and could possibly upset people unless you have advance permission.
But to answer the question, it really depends on how you want to do it. If you have a single team doing everything and everyone can grab any ticket, then having a single backlog means that the priorities are obvious. I can't tell you how much that improves the planning process and it could very well override the problems with having a mixed board. If you are doing kanban, then having the priority correct is pretty important.
If you have multiple different teams or multiple different roles, then it can help to split out the overall board into streams. However, in my experience it helps to have an overall priority list for as long as possible. If you are using software to display the board, then it might be possible to automatically split it into streams. Once you get too big, though, it becomes impossible to do central planning for everything, so it becomes a moot point afterwards.
Different people react to context switching differently as well. Personally, I have literally 0 problems with it. Some people end up going something like twice as slow if they have to context switch. The normal solution I've found is to allow people who have a problem to cherry pick. If people start to game it, then you might need a manager to intervene, but I've rarely had problems with it.
When in doubt, talk to your teammates and come up with a compromise ;-)