The realtime executives are not operating systems. Executives and operating systems are two different things.
QNX is a realtime operating system. It addresses the niche of a realtime operating system that isn't being met by anything else out there, and has been doing so for 40 years, even if free versions for hobbyists haven't been available for many of those years. It's not about market share since that's always been there. It's about mindshare and given the comments here, that's obviously missing.
Are you sure they want to open source it? Is this just an assumption, or can you provide a reference?
I actually tried to find out whether the source code of the kernel etc. is included in the free version they provide, but they require to install a license manager on my machine, so I stopped; and it doesn't look as if it was open-source when there is a license manager (would be very easy to get around).