+ Making instant web apps and APIs just by specifying two functions -- "front-end" and "back-end".
+ Easily creating impressive-looking programmatic 2D and 3D graphics and sounds.
+ Innovative idea making visible graphics and colors as arguments and outputs of functions.
+ "Knowledgebase" of facts accessible using natural language. (Though, this was often slow due to server calls.)
My main critique of Mathematica is that it is the epitome of a kitchen sink library. There is an overwhelming number of functions for every possible thing. So, it is often not feasible to know what is offered without a lot of reading. (In fact, Wolfram even states this in his book.)
This is the real problem with their "knowledgebase". I would love to enter any natural-language query and get an answer, but in reality I need to check their list of supported areas and know the seemingly arbitrary list of facts about each area. To really make this useful, Wolfram has to figure out how to make it work without requiring me to be so knowledgeable about what it contains a priori.
Also, having such a large standard library makes the language very "flashy". It is indeed impressive and inspiring to create a sphere or musical piece in one line of code. However, real programs end up being much larger. In practical programs, Mathematica's functional syntax results in long unwieldy statements with forced formatting. This is occasionally impressive, but it is just as often hard to decipher.