szhorvat

I find that the biggest practical issue with using GCC's analyser is that it's just so darn difficult to set it up and have readable output. Have you considered focusing on this a bit more? Writing documentation (or even producing tools) for integrating analysis into one's usual workflow, integrating with common build systems (e.g. CMake), making the output properly readable in the context of the source code? I feel that at this point this would be much more helpful than ASCII art or more kinds of warnings ...

I did attempt to use the SARIF output just after your previous blog post a year ago, with a CMake based project, but after an hour I still wasn't able get the warnings to show within the context of the source code.

The -fdiagnostics-format option isn't even in the GCC documentation, your blog post is the only place where I saw it mentioned.

In short: I'd love to use GCC's analyser, and tried it several times, but my bottleneck is usability, ease of setup, and a proper interface to help sort out the many false positives from the true issues.

I'm quite surprised to hear this. What do you get from GCC's analyser that Clang's static analyser doesn't already report?

I tried to use GCC's analyser several times, but I couldn't find any good front ends to it that make the output readable. Clang has multiple (reasonably good HTML output, CodeChecker, Xcode integration, etc.). How do you read the output?

Furthermore, I find that GCC produces many more false positives than Clang.

There's lots of ideological discussion here. I would have loved to read some experiences based on actually trying the thing!

You don't have to settle for the command line. It works with Jupyter, which seems to be the most pleasant way to use it.

Now you can give it a try easily. Why don't you? Maybe you will discover a new world ;-)

Don't think of Mathematica as a computer algebra system. It is much more than that, fully capable of numerical computation, and just the unique term rewriting programming language makes it worth checking out.

For practical purposes, nothing. If you have used Mathematica in the past, and know what it is: the "Wolfram Engine" is basically the Mathematica kernel.

Note that this free version only includes the kernel, not the notebook interface. But there is nascent support for Jupyter, so productive use is very much possible.

"I'd expect that many of the things I'm used to doing in Mathematica wouldn't work here"

What's amazing is that this isn't true. There are no _technical_ restrictions. You can't do Manipulate because the notebook interface is no there (or rather: it's there but it's read-only). But all the computational power is available.

"I’m a representative sample of my fellow software engineers"

I'm a bit puzzled about how strongly you seem to be feeling about this topic. Given your comments, you seem to be quite far from the target audience of Mathematica. It's not for software engineers. Most productive work done with Mathematica is not "building software" ...

But given all this hostility, have you downloaded it, and have you tried it? Or is all this just philosophical (i.e. the least interesting) criticism? There seems to be no experience backing your comments.

We were considering making http://matlink.org/ compatible with Octave. The feedback I got on this was part of why this wasn't done. To make MATLink user friendly, it needs to come with compiled binaries, which would be linked against Mathematica's closed source MathLink library.

If it is the case that GPL doesn't forbid this, I'd love to hear about it.

Note that I'm not saying that one can claim that a result is correct, a theorem is true, etc. based on the fact that some undocumented algorithm produced it. That's clearly unacceptable.

Nor am I saying that it's never necessary to rely on an algorithm to get such a result.

What I'm saying that when people use Mathematica or other closed source systems, they do not usually commit these mistakes.

Also note that Mathematica programs can be open source and documented (many are). Several built-in packages have accessible and documented source code (e.g. Combinatorica). There's nothing wrong with using these to obtain such a result, and cite the (public and documented) program used to create it.

I guess it depends on one's field.

I've never cited a software I didn't write myself, simply saying trust this software, here's the reference. I wouldn't trust a result from Sage any better than one from Mathematica, simply based on which system produced it. I'd trust that x is a solution of an equation if substituting it back verifies it. I'll trust that two graphs are isomorphic if the software gives me a vertex permutation that makes the adjacency matrices identical. It doesn't matter how that isomorphism was computed.

When publishing work, I'll aim to make it verifiable this way.

I believe the vast majority of the use of these systems is not of the type when one needs to blindly trust the software and refer back to it in the paper. At least in my field (physics) it isn't. Yet I use programs like this daily, and I clearly depend on them for my work.

Most of the functionality available in Mathematica (or, I'd argue, most similar systems) are not of the type that one needs to cite. They either use standard and well known algorithms that are available in a multitude of systems (do you cite the methods for matrix multiplication or eigenvalue computation, and would it make a difference?), or the results are much easier to verify than to compute.

In those cases when I need to rely on a published method, like you mention, the method is very unlikely to be a built-in part of any system. So I either need to re-implement it, or use the original code of the authors. If the authors implemented their method in Mathematica instead of Python, does that make their program less reliable? No. It's still a published method, anyone can verify it.

My point is that I hear this argument about Sage very often, and the typical generalization is: "if you used Mathematica for your research, that's wrong, because it's not verifiable". This is a fallacy. It completely ignores how these software are used in practice, and implies that results from open source software are somehow magically reliable (they're not) and don't need verification (they do).

I've yet to come across a situation where the argument does apply at all: point me to a paper which goes truly wrong by citing Mathematica/MATLAB/Maple/etc this way.

I should add that every time I asked WRI support about implementation details, I did receive an answer with references to the method used.

My impression is that you don't hate to be that guy, but actually love to express this opinion.

Open source systems like Sage always look desirable, simply by virtue of being open source. But every time I look at it I'm left with a very bad taste in the mouth because of the constant badmouthing of non-open-source systems that is going on in that community. Companies do that sort of thing, and it doesn't inspire trust. But we know that it can happen just because a few people in the management made bad decisions. But when a community (!) around an open source (!) system takes on that attitude, it looks much worse. Don't you realize you're driving people away?

Why not put all that energy into improving your own system instead of trying to actively hinder others? Examples of that are forking GMP and making in GPL (not LGPL); actively pointing out to people (as Mr. Hermoso did to me) that no you can't link Octave to Mathematica because Octave is GPL (which is just a hindrance for my research, as well as to others); building on the fallacy that results obtained with Sage are inherently better because Sage is open sourced software is _theoretically_ verifiable. All software is buggy, and the only thing that makes a research result more trustworthy is if it is indeed verified, not if it's theoretically verifiable, but no one ever does it. Practical verification is almost never about reading the source code. It's about making sure the result is consistent and computing it with alternative tools.

You need to get used to the notebook concept. In the end you'll find that it's much more convenient for interactive work than a traditional command line. For non-interactive work you can still use Vim if you like.

In fact Mathematica shows its advantages most when used interactively (compared to languages, like Python, which were not really designed for this).

Yes, I agree with you assessment :-)

"You import someone's changes to a local notebook" <-- there are no existing tools that reliably automate that. (I'm aware of the notebook diff utility in the AuthorTools package, but that's not reliable). There are many tools if you're working solely with plain text.

Notebook interfaces are nice for interactive work (I love the concept and advocate for it), but they're no replacement for an IDE for writing packages. I spend most of my time in the notebook interface, but when it comes to writing a polished package, I use an IDE or a separate text editor. (Check out http://www.mathematicaplugin.halirutan.de/)

Having worked on a medium-large project collaboratively (http://matlink.org/), I can't imagine how we could have managed without using plain text files and version control (notebooks currently cannot be efficently used with version control systems).

IDEs are not used just out of habit. They have their uses, and notebooks can't replace them, just as IDEs or simple REPLs can't replace notebooks.

The Front End isn't very good at taking the job of an IDE ...

But it is a very good notebook interface (doesn't the notebook concept come from Mathematica originally?). Notebook interfaces are excellent for interactive work (a large part of scientific computing) and improvisational programming. It's a pity that they're only becoming popular recently (IPython). Many people still don't get it what notebooks are really good for. For example, the R community seems to think that a "notebook" is for report generation, and none of the "notebooks" for R allow any interactive work (knitr).

What's good about the notebook interface is the workflows it allows. Even MATLAB adopted "cell mode" a few years ago which enables the same basic workflow (except it does't have the equivalent of output cells, so it's not quite as good).

It's here: https://billing.wolframcloud.com/pricing.html?p=wpc

What will it be able to do that Mathematica won't have? (Will Mathematica also have cloud integration?)

And what about the Discovery Platform and Data Science platform? How will these differ from Mathematica?

Thanks, that explaines it!

Can anyone enlighten me about why the title of the post got changed? I tried to squeeze all the relevant information in there, to make it easy for people to decide whether they're interesed or not. I really hope it's not because some of the commenters don't like this system not being open source, and find the "try it for free" bit not politically correct.

Yes, exactly. I should have said, try Mathematica.SE for questions after having read through the inroductory tutorials. :-) Thanks for the link!

Try it out here: https://programming.wolframcloud.com/app/

For people new to the Wolfram Language: it's essentially a new version of the programming language of Mathematica. if you're looking for help on StackExchange, go to http://mathematica.stackexchange.com/

The out of code data processing is something that was sorely needed, and I was wishing for this for a long time. One of the big drawbacks of Mathematica for data processing was that it's only convenient to use if all the data can be read into memory.

What you're saying about distributing a computation on a cluster sounds very interesting. I used Mathematica for a hybrid Mathematica/C++ calculation (LibraryLink) where the complexity was handled by Mathematica and the (simple) heavy lifting by C++. I used the standard parallel tools to run it on a cluster, which means that communication was done through MathLink.

I never went above ~70 CPUs, but people say that problems start to appear above that (too many MathLink connection): http://mathematica.stackexchange.com/questions/20356/mathema...

Another possible problem with my solution (LibraryLink, then Mma parallelization) was that it required a Mma license for as many kernels as I was running, even though most of them were only running the C++ code. But that's easy to fix on WRI's side.

About the "fictive power law" thing: this is THE paper to read: http://arxiv.org/abs/0706.1062 (it's an easy read, explaining what the maximum likelihood method is, etc.). Despite what they say, fitting the log-log CDF usually gives pretty good results when done right (fitting the PDF does not)

Several questions :-)

1. I noticed the interactive slider, embedded into the webpage. That's not what vanilla Mma 9 can do. Is there a simple way we can do the same without the CDF plugin (e.g. a package I'm not aware of) or is this future functionality?

2. The graph with the migration data looks nice. Mma 9 can't do an edge layout like this (curved edges) by default. Is this again custom code (custom Graphics or custom EdgeRenderingFunction) or is it future functionality?

3. There's the part with the frequency of various graph motives (the number of edges, triangles, and other weird shapes in graphs). How did you count these? Was it done using Mma? Some of these motifs are easy to count (there are simple expressions in therms of the adjacency matrix), but some others like the (1-2, 2-3, 1-3, 3-4) subgraph are not so easy.

4. How did you make the word clouds? Is the algorithm written in Mma? Here's a nice but slow one (the worse answer by myself): http://mathematica.stackexchange.com/questions/2334/how-to-c...

If you choose to learn a new language, I'd suggest to do it for the joy of learning and discovering something new. The fact that you ask which one is the most valuable shows that you don't feel you need another language for any practical purpose. Don't look for the easiest or "most valuable" (whatever that might mean). Look for one you'd enjoy.

You can check out some online language learning communities such as UniLang or HowToLearnAnyLanguage for a lot of discussion on this.

Looks interesting, thanks for sharing it! Your algorithm is also based on the same "spiral out" technique described by Jonathan Feinberg on StackOverflow. (It's also what I used in my reply to that Mathematica.SE question.) Heike's solution is interesting because is always finds the tightest possible fit with a not-very-bad performance (the image processing operation used can be implemented in terms of correlations, which can be fast when done using fft).