Mathematica 9 Is Released Today
blog.stephenwolfram.com
blog.stephenwolfram.com
When I had first encountered Mathematica in college, I thought "this is neat", used it for a few calculations, and forgot about it. Fast forward to a few months ago and I decided to install a demo just to learn about term rewriting and pattern matching.
What I discovered blew my mind: It's just a lisp! Every expression has a "head", which is just a namespaced symbol. FullForm[x+y+z] is really just Plus[x,y,z] and Head[Plus[x,y,z]] is simply Plus. Squint, and it looks like (+ x y z) to me... The really interesting part is that the entire computational library is built around term rewriting. While there is mutable state, it exists almost primarily at the symbol definition level. Everything else is pretty much pattern matching on trees of symbols. Even the rendered output is just a drawn representation of a box-oriented markup built out of Mathematica forms. It's quite amazing.
Mathematica owes much of its lineage to Macsyma, a computer algebra system that really was Lisp on the inside; its spiritual descendant, Maxima, is written in CL.
Wolfram and Chris Cole started developing a symbolic math system, which they named Symbolic Manipulation Program (SMP), when they found that Macsyma was unable to handle the computational needs for their physics research. For problems that they considered to be medium problems, Macsyma could do the work, but it was very slow. For problems that they considered to be large problems, Macsyma ran out of memory. Thus, they needed something more efficient in both time and space.
Wolfram and Cole planned on writing in C. They asked researchers in the symbolic computation community about this, and were told that it was a terrible idea. C was too low level. To do a high level system like a symbolic algebra system, you had to work at a much higher level. Using anything less than Lisp pretty much guaranteed your project would take forever.
Wolfram and Cole decided on C anyway. (BTW, C at CITHEP was not quite standard C. Norman Wilson had hacked the compiler to handle float and double expressions the same way FORTRAN handled them, making C almost as good as FORTRAN for numerical work, and C had mostly taken over from FORTRAN for physics computation there).
I recall seeing the factor of 100 number, or something like that, in one of the first public presentations of SMP. On the research problems that Wolfram and Cole considered to be medium sized, SMP was beating the Lisp-based systems by around a factor of 100.
SortedSet[x_List] := SortedSet @@ Union[x];
SortedSet /: Plus[a_SortedSet, b_SortedSet] := Union[a, b];
SortedSet /: Plus[a_SortedSet, Minus[b_SortedSet]] := Intersection[a, b];
Format[set_SortedSet] := AngleBracket @@ set;
integrate((x/sqrt(x * x * x * x + 10 * x * x - 96 * x - 71)),x)
AXIOM is written in lisp and latex (literate programming), it's more powerful than MAXIMA.
Call outs to R code http://www.wolfram.com/mathematica/new-in-9/built-in-integra...
Social network analysis http://www.wolfram.com/mathematica/new-in-9/social-network-a...
Computation bar http://www.wolfram.com/mathematica/new-in-9/next-computation...
Sigh. I dream of Mathematica being open source.
Take a look at Maxima then. It is a good system too but not even close in quality, features, etc.I blame its open source nature for this.
Why is it's open source nature to blame here? Can you elaborate?
Is it the product vs. project argument?
In my not so humble opinion and only in my opinion open sourcing of Maxima (or Macsyma) put it in the wrong hands. Not to blame the actual developers of Maxima. Maxima is a very good project by itself just not as good as its commercial competitors.
And anyway, having a(nother) open-source symbolic mathematical program is better than not having it, even if it is substandard compared to the commercial offerings.
[1] http://www.wolfram.com/products/applications/excel_link/feat...
As a heavy user of both, my experience is that Mathematica is more difficult initially when writing your code, but is ultimately easier than Excel because the thing you would be doing in Mathematica would be very hard to do in Excel (it is both a technical as well as maintenance nightmare).
That's what Google's Brin was working on when he interned there.
I share the dream of an open source Mathematica. It's so rich, elegant and powerful, using it would be a no-brainer. But in its present state? Nope. I've been bitten too many times by the proprietary nature of Matlab, in which I have invested a lot of work, to repeat the mistake. More specifically and from the top of my mind, I found that proprietary means:
- I cannot run my code on my personal PC
- some people I'm collaborating with cannot use my work
- when I move to a different job, there is a good chance I cannot reuse my code
- even when my new workstation (or my colleague's PC) does have the software installed, there is a good chance the version is incompatible with my code or the license doesn't include some toolbox I'm using
- some work I want to make freely available is of no use to someone who cannot afford the software
DownValues[GraphPlot]
Only a smallish subset of the language is written in C (for speed).
Edit: It's also kinda fun to use
Names["*`*"]
Names["Internal`*"]
Names["Developer`*"]
The amount of restraint you've just shown in describing NKS is superhuman.
Each time I've gotten to the point where I can do basic derivatives/integrals/polynomials/etc and realized the real power is in the huge library (and brain trust behind that library) available to you.
It really is quite enlightening to come back to other languages. It feels like you're going back to tin can and string communications.
I am incredibly excited about the improvements for operations research markov chains and queueing, as well as timeseries. These will simplify my code hugely, better yet speed up work.
I dream of a world where people don't compulsively fantasize that for any given piece of software, it should be supplanted by some other piece of software whose main differentiating feature is that it's a FOSS clone.
Mathematica being open source would mean that I could look at the source though and see what the hell is going on inside. I'd love to see how they are doing call outs to R and trace through all the interactions between the kernel, the user interface, and W|A. If it were open source, I could see the implementations of all the Mathematical algorithms and look at all the neat optimizations and new results they have found during their work. I care about the "math in Mathematica and am interested in seeing how it all works.
EDIT: Mathematica can do almost anything, but how do you use it actually in your work environment?
On those rare occasions I need to solve equations or do some calculus, there's nothing better.
In particular I love to build up some complicated function of many variables and wrap a Plot[] command in a Manipulate[] command so that I can drag the slider bars around and see how the shape of the function changes. This allow for me to get an intuitive understanding of the problem.
While this could be an extensive debate, I prefer Mathematica to Matlab whenever there is an option. I am a heavy user of both, though have never used Symbolic toolbox in Matlab. It starts a lot more complex in Mathematica, but becomes significantly more powerful shortly thereafter.
I used it extensively in designing my side project synth[1], where I used its visualisation capabilities to figure out everything from panning algorithms to delay lines. It earned its keep most often late at night when I was tired and trying to figure out daytime-simple-but-nighttime-hard routines. The Manipulate[] function is really good for this, as it shows you the output of a function as you drag the inputs up and down. I also used it for turning mathematical formulae into C code - the CForm[] function does this very well.
It's also a great way to experiment with functional programming, since that's how it mostly likes to operate.
I reckon there's a lot of places you could use it, but it's too expensive to justify. I don't really care if it's open source or not, but I'd like Stephen Wolfram to have an epiphany on pricing and realise he could sell ten times as many Home Edition copies at a quarter of the price. Its adoption is absolutely held up by its sky high price.
Anyone with further interest in it should check out the Wolfram blog[2] and one of the friendliest StackExchange communities out there[3]
[1]http://www.omnivoresoft.com [2]http://blog.wolfram.com [3]http://mathematica.stackexchange.com
That alone would almost be reason enough to upgrade.
So if you undo your last 5 minutes of work because you thought you would undo one line or something there is no way to get it back. T________T
If you deleting something inside that one thing you are currently typing, that deletion is also completely impossible to undo because if you do you will undo typing that entire block. (Undoing a small accidental delete is usually the cause of the scenario i described above)
The holy grail as I see it would be storing the entire edit history of the document as a git-like tree of modifications, which is entirely possible because Mathematica notebooks are themselves built out of Mathematica expressions (see for example Cell http://reference.wolfram.com/mathematica/ref/Cell.html). This would be similar to Rich Hickey's work on language-aware version control (http://blog.datomic.com/2012/10/codeq.html)
One of these disadvantages is that editing the notebook does quite weird things to this hierarchy, like embedding new cells inside existing ones, merging cells, and changing mutable state in the frontend in various ways. Figuring out how to intelligently support multiple levels of undo without leading to rapid garbage generation and/or corruption is I imagine a Hard Problem.
For the record, another disadvantage is that source versioning and diffing is nigh on impossible (unless it takes place at a higher level, which is an interesting idea I think the company should pursue).
I can attest to that! I have directories full of -v1.nb, -v2.nb, -v3.nb, -v3-fixed.nb files. Here's to hoping someone figures it out anyway :)
I'm particularly pleased by the addition of units. I can't count the number of times I've done some symbolic work in Mathematica, copied the results, pasted them into WolframAlpha, added numeric values and units, and then computed the value there. It's become a common refrain in my workflow and a rather annoying one at that, so I'm really glad they're integrating things from WA into real Mathematica.
A lot of the new features fall short for me, e.g. face detection. Sure, it's nice to have over not having it at all, but when alternatives like OpenCV offer so much more functionality, why bother? I was trying to find an easy way to do facial landmark detection recently. If Mathematica included that by default, now that'd be a selling point.
Also misleading is their new "audio spectrogram" feature. Wasn't this already available with their FFT functions? An actual useful feature would be, e.g., automatic formant detection from voice files.
Often times, their new features list are just more nicely-packaged versions of existing features that in practice just save a line or two of code, rather than an feature that didn't exist at all before.
If you read the blog post you'll see that the units support before 9 was 'tacked on' as a separate package. The units in 9 are much more extensive and flexible, and more importantly are integrated with other functions at a deep level. To give an example, if you find a solution to an equation that involves units, the solution will possess the correct units (and so on for curve fitting, etc..).
> Face detection
Having used both Mathematica and OpenCV for image processing, I challenge you to explain exactly how OpenCV goes "far beyond" Mathematica. Particularly in morphological processing and image component measurements, Mathematica is quite far ahead of OpenCV (in fact, Mathematica bundles OpenCV to do some of its image processing). However, OpenCV does expose more lower level stuff, and comes with some ML and object detection routines that Mathematica doesn't yet have. But the maximum speed at which you can prototype new algorithms in OpenCV is pretty darn painful.
> Spectrogram
The audio spectrogram is certainly not tacked on. Even doing a Short Time Fourier Transform yourself is far from easy (it is not equivalent to an ordinary FFT). Making it performant on large audio samples also requires some subsampling smarts. And doing a correct wavelet spectrogram is highly non-trivial, because the feature size depends on the frequency -- i.e. low frequencies have lower time resolution than higher frequencies.
> No new content.
Look at this list:
http://reference.wolfram.com/mathematica/guide/SummaryOfNewF...
There's a massive amount of new algorithmic content there. You can't seriously claim that, say, symbolic calculation of antisymmetric tensors and continuous Markov processes are 'just convenient wrapping'.
Disclosure: I work at WRI, and love using Mathematica.
>You can't seriously mean that.
I didn't say that. For the record, I use Mathematica too, and while I find it useful I'm more often that not left frustrated.
Clearly they've improved these features, they just haven't improved them enough for my uses. I've personally used the Units package before and made audio spectrograms, without much issue but without much success either. I always had to turn to other tools to get the result I need, and this is still the case with Mathematica 9. The improvements aren't sufficient to make Mathematica useful for my particular uses. Let's take face detection as an example:
http://reference.wolfram.com/mathematica/ref/FindFaces.html
Do we know how it finds the faces? What if we want it to find other facial features, or train it with another data set? With OpenCV, I can tell it to use a custom Haar cascade. The only options Mathematica exposes are face size!
Having a feature in a bullet point isn't sufficient if the feature isn't fleshed-out enough to be useful.
More generally, I find myself using Matlab more often even though it doesn't work quite as well for my purposes. And why? Because it has easily readable source files, more in-depth documentation that tells you how its algorithms work, it has an open-source alternative that's more or less source-compatible (Octave; Matlab gives me a nice GUI and a pleasant working environment, but a colleague can run my files) and also uses open source software in a lot of cases so its output can be independently verified. It's that kind of thing that Mathematica needs to work on, not surface level features that probably aren't going to be useful for people who actually need them. If someone's project relies on good face detection I doubt very much they'd find FindFaces[] sufficient. Which is not to say Mathematica isn't better for having it than not having it at all, and I am sympathetic that they have to start somewhere...
Edit: I just want to add thatI have no doubt a lot of very smart people are working at Wolfram, and a lot of these features are great. I just can't help but think that developer effort is somewhat misplaced. The graph in the blog post of number of functions against time only goes to highlight that. When you see the number of people clamouring for undo support, or a better GUI, or existing features being better-explained or more fleshed out, I think that's what they should be concentrating on rather than adding new features to the list. Though the new Tensor functions are pretty cool..
The image processing group thought about allowing you to use custom Haar cascades with FindFaces, and decided instead to wait another version and do a proper Viola-Jones detection function + training framework.
The thinking was that enough people will find some basic face detection useful to justify the early release of this particular facet of the whole object detection/recognition problem.
But, you are right. There is definitely a balance between 'solving' an entire domain at once or expanding the frontier to new domains. I think the Mathematica team is navigating that balance pretty well, though. My most common reaction when seeing new M functionality is 'wow, this is amazingly deep', not 'gee, this is a gimmick'.
and a bunch of spit and polish on the Mac notebook interface - undo as people have mentioned, scrolling performance, responsiveness, etc.
Thanks from a 17 year MMA user.
I've been trying to plug Mathematica into my workflow in place of a number of tools I'm more familiar with and I keep running into issues where I either try to do things in a very un-Mathematica way or the default behaviors I assume of the program are wildly incorrect (e.g. Thread operates in the reverse order I would expect, importing an extracting a column from a csv file seems clunky, wrote up a worksheet with a number of E variables with subscripts only to find out its treated like Exp[], etc.)
There seems to be a wonderfully large amount of available training and tutorials all over the web but I don't know where to start to most effectively acclimate to Mathematica.
At least, exploring is how I learnt it.
These video courses also exist and are all free: http://www.wolfram.com/training/courses/. I haven't tried any of them, but they're supposed to be quite good.
I bought a 1-year license just a few months ago for version 8. Only £36, sure, but I'm a student and can't afford a second £36 now just for a handful of new features, even if they would be useful. Upgrade pricing would be nice. I'm sure working out a fair upgrade price shouldn't be too hard either. I bet there's even a formula for it somewhere...
In the past couple of months, Mathics had made quite a lot of progress.In particular the upcoming release will run on PyPy. Obviously it still had lots of catching up to do, but being written in python (and some Mathematica) makes development easy and fast. Come join us!
Disclaimer: Mathics vice dictator here
You can always download it and try it locally: http://www.mathics.org
Edit: If anyone is interested in getting involved, we are always looking for contributers. The full code is on github: https://github.com/poeschko/Mathics
I used this to "retina-fy" Mathematica and it worked, the notebooks are clear now.
We're all blind to things our current tools don't permit us to explore.
As a random example, I bet that many cloud-based services could (but probably don't) do reliability analysis of their services under different independence assumptions. Are the weak links what you thought they were? What if you updated your model as various things failed / rolled over, so that you could run to put out the fires that are most likely to burn the whole village down? For a modest organization a smart intern could probably put this together with Mathematica in a couple of weeks.
My one gripe thus far is that there isn't as much keyboard control over the prediction interface as I'd like, but that could just be that I haven't found the shortcuts yet.
One thing I quite like about it is that there's a built-in "feedback" feature- any time the "predictive UI bar" is showing, there's a button you can press to send semi-structured feedback about what's currently showing, suggest other things you'd like to appear in similar situations, and so on. I've used it already to suggest a few changes the prediction UI for ListPlot.