I've advocated this too. One (large) practical issue is that Mathematica includes several libraries and APIs that have been licensed from third parties to fill out its suite of algorithms and data. Removing these for an open source edition would leave gaps in the language, though it's unclear to me how 'neat' the holes would be, or how much they might cripple the experience.
Even if Mathematica is never open sourced, there's still value in having a commercial language that can incubate new ideas or applications of existing ideas, for open source efforts to either ignore or riff on. The document+repl model used by IPython/Jupyter is an example of something that WRI didn't pioneer but certainly helped popularize in its modern form.
Another example that I think should be more widespread is the idea of 'superfunctions' whose arguments exploit a simple grammar to scale from simple invocations to arbitrarily complex ones. In effect you have a lightweight DSL -- a large or infinite family of functions fused into one -- which I think can buy a great deal of concision and flexibility.
GroupBy (https://reference.wolfram.com/language/ref/GroupBy.html) is one small example of this idea; in its simplest form it just partitions a list into equivalence classes, but scales up via its argument grammar to something more like (optionally nested) MapReduce. Part (https://reference.wolfram.com/language/ref/Part.html) would be another example. Again, this isn't something unique to Mathematica, you see variants of this idea in e.g. Clojure and Haskell, but my impression is that we've incorporated it into a wider cross-section of our language and made it more light-weight and convenient to use.
Another thing that I often think about is the notion of having a deep web of interrelated structures and behaviors, for example colors both have a visual display form (little colored boxes), IDE edit mechanics (you can click them to edit them in a palette), and polymorphic properties like distance, so that functions such as Nearest and FindClusters can 'automatically' operate on them. That's of course hardly novel, typeclasses in Haskell were designed to do precisely this, arguably more elegantly and user-extensibley.
But I think Mathematica shows how it is really valuable to have not just the capability to do this, but having it built in and working -- with many of those n^2 possible interactions between functions and data already curated and implemented. WRI is far from having done this completely, but again, we're trying, and I don't know of other groups that are trying to do this in an as integrated a way, across so many areas of computation.