Hacker News Reader

Comment by picozeta | Hacker News Reader

I am not sure Wolfram Research realize what they have.

If they would make their stack FOSS they would be absolute heroes and shape the future of applied math and computation software with a historic impact. Otherwise they will fade away, when other interfaces (Jupyter & Co.) catch up and/or surpass it in its capabilities. More and more people will just not accept the closed nature as it inherently contradicts with the idea of exploration.

There are basically 3 scenarios:

- they go FOSS now: best outcome for people and Wolfram

- they go FOSS later: good outcome for people, more difficult for Wolfram, as they lose developers/community

- the don't go FOSS (or not in near future): Wolfram SW won't be used anymore as open alternatives surpass it. People won't care.

Even Microsoft got it.

View on HN

> If they would make their stack FOSS they would be absolute heroes and shape the future [...]

... and instantly stop making money. I have the impression that making money is quite important to Wolfram Research.

You may be right that in the long run they're bound to get their lunch eaten by free alternatives, but "in the long run we are all dead" and it's really not surprising if they prefer "continue making lots of money from selling Mathematica, and maybe one day find that free alternatives take away our market" over "immediately make vastly less money from selling Mathematica, but keep market share for this big codebase we can no longer make much money out of".

Also, open-sourcing their stack would mean relinquishing a certain amount of control. Have you ever heard anything about Stephen Wolfram that would suggest he'd be OK with that?

There are different target groups for this kind of software. People who like open source will never use anything else, even if its inferior to closed source alternatives. Other folks will gladly pay to get a system that is robust, stable, and easy to use. I think there is enough market share for Wolfram to thrive for a long time (and open source alternatives too).

> People who like open source will never use anything else [...]

I like open source, but I _have_ to use mathematica to be as productive as my peers in my field. The open source tools that exist are indeed inferior and that's why I cannot use them.

However, this is not a good argument, as my field is physics. I do believe that the use of mathematica for science is a inherently bad thing, and should not be considered an allowed part of doing science -- because it is closed source.

I consider the reductionistic part of science very important, so when I use some function in mathematica I do not know exactly how it is implemented hence I cannot reduce my result beyond the point of "Wolfram says it is ok.". But that is not ok.

I have been lucky with all of my results, in the sense that one can check them by hand or by inferior products. But finding the result to begin with is why mathematica is almost necessary for a lot of calculations.

You have to think of Mathematica code as a specification of what your science does. As long as the expected output of the function is clearly defined, then your work is replicatable/verifiable (with enough effort) which makes it good science. I am much more concerned about occaisional places where the documentation of Mathematica is poor, than whether the soruce is viewable.

Open Source is just the ultimately precise but horribly inconvenient documetnation.

Fair enough. Perhaps my issue more correctly lies in the close-to absolute trust some of my colleagues have for mathematica.

Since mathematica is so much faster and feature rich people use it and only very occasionally is it verified by some other software. I would prefer, and would make things easier and faster, if we had (verifiable code) + (one result), instead of (no code) + (one result) + (independent check of result), since only rarely one bothers/have the time to make the independent check. In some cases there is no option to make an independent check (e.g. "with enough effort" is usually too much effort).

Mathematica is not verifiable, it doesn't have the authority to define 'good science'. Code spec =/= proof.

> The open source tools that exist are indeed inferior and that's why I cannot use them.

Can you be more specific? It might be interesting to hear if people found viable solutions to specific concerns.

I have only used some of the modules that are available for SymPy. Most of them are not complete enough to use professionally, but I try to spend some time to learn them since I would prefer to use them when they get more complete -- but that's one reason why they are inferior.

At one point I was using SymPy and I wanted to invert a symbolic matrix, a rather small one but it had some off-diagonal elements. This took a few seconds in mathematica, after a few hours in SymPy I had to halt the execution of that line. I never got around to look at the code to see what caused it, probably some simple bug, but that's another reason.

I have used Sage somewhat as well. It has more features than SymPy, for what I do professionally. Still some features are missing that I would need. But this might also just be a problem of the amount of experience I have with these tools.

The last problem is that these are much slower than mathematica. I would certainly not say that I'm a good programmer, so my code is probably very slow and badly written. In mathematica I have done some calculations that take days to complete, but they were quite heavy calculations. For these projects I could in principal have used SymPy, I know it has enough features, but with how much slower it is, it would be useless (unless I was able to improve my code to compensate).

With more experience I could give more specific concerns.

Thanks. I must say that except the inversion example, this is still not specific enough for me to decide if SymPy (and other tools) are inferior, as you claim -- in particular, I can't tell if they are going to be inferior in doing the kinds of things I need done.

I don't really see sympy and the like surpassing Mathematica. Even maple is kinda having trouble. If when I'm buying Mathematica I'm paying for a proper implementation of the risch (integration) algorithm, it will have been money well spent.

Doesn't Maxima [0] have a proper implementation of the Risch algorithm already?

[0] http://maxima.sourceforge.net/

I don't think so. Last I checked, the most complete implementation (at least in open source) was Axiom.

How does REDUCE compare?

I'm not sure. I haven't looked at it in a very long time. The open source implementations I know reasonably well are: Maxima (and therefore Sage), SymPy, and Axiom. I did some work on a integration test suite for Sage before I had to put it on hold. It was based on a suite from Axiom.

There is no complete implementation of the Risch algorithm. None of the implementations fully implement the algebraic case. Besides, the Risch algorithm doesn't solve all symbolic integration problems in practice:

* Even when it succeeds, it doesn't necessarily give you the simplest possible form of the solution.

* It is concerned with elementary functions, but real-world problem often involve non-elementary functions (like Bessel and hypergeometric functions). You can extend the Risch algorithm to work with such functions, but this is complicated.

* It doesn't allow computing parametric answers for common parametric families of integrals (this needs to be done using heuristics and lookup tables).

* It doesn't give you an algorithm for definite integration, except in simple cases. Even with proper definite integrals, you need to be extremely careful with branch cuts when applying the fundamental theorem of calculus. For improper definite integrals, systems like Mathematica generally try to go via convolutions of Meijer G-functions, using huge lookup tables and simplification heuristics.

* Even more fundamentally, the Risch algorithm (and symbolic integration more generally) puts heavy demands on the underlying symbolic computation engine. In particular, it depends on the ability to decide whether an expression is equivalent to zero, which in fact is an undecidable problem, and this strictly speaking makes the Risch "algorithm" a non-algorithm (though it is a proper algorithm when restricted to a ground field where zero testing is effective, such as Q).

I can only say that I am very impressed by the knowledge you are displaying about the Risch algorithm and symbolic integration. If I may ask, what is your background and how did you learn about them so deeply?

PhD in symbolic computation (RISC). Symbolic integration is not my field of research, though, and I don't really know anything beyond the basics (covered in courses on computer algebra). Passively attended plenty of seminars and conference talks on symbolic integration, though, and for a couple of years, I did share office with Clemens Raab who is one of the experts on the Risch algorithm.

I am working on a project that requires symbolic computation, which I am learning a bit by myself. For now I can manage, but could I get in touch with you in case I would need some advice about that? My email is in my profile (thanks!).

> I don't really see sympy and the like surpassing Mathematica.

Given enough time, it will happen. There are several reasons. One, people who publish mathematics academically are reluctant to use a closed-source tool to create their results, because those results can't be examined and verified at their source. This issue pushes people toward sympy/Sage/Jupyter and other similar open environments, even though at the moment they aren't as powerful as the Wolfram environment.

For example, imagine that the four-color map problem (the first significant computer-aided result in mathematics) had been solved by a close-source environment (instead of an open, freely readable source as it was). If that had been true, people would still be arguing about whether the result was valid.

Two, an open-source environment like sympy attracts technically skilled people intent on improving it, knowing that they're contributing to a tangible kind of progress that's clear to everyone, and then freely copied by like-minded people into different environments, all with the intent to advance human knowledge, rather than make stockholders happy.

Three, this may not be apparent, but the existence of environments like Sage and sympy are putting tremendous pressure on Wolfram to price their products more reasonably. Instead, and so far, Wolfram's strategy has been to design enticing free samplers like Wolfram Alpha as gateways to their expensive offerings.

> If when I'm buying Mathematica I'm paying for a proper implementation of the risch (integration) algorithm, it will have been money well spent.

This is a perfect example -- until Mathematica is open-sourced, no one can know whether the existing algorithm is either complete or optimal. Also, given that research into this algorithm has enormous practical value in both mathematics and computer science, it's a shame that a large segment of the effort is being carried out in secret. I'm sure those responsible for its present form would love to publish their results, if only for the fact that this would get them fired and possibly prosecuted.

There was a time when people who made an original contribution to human knowledge would publish their results in the open (think Einstein) and take pride in their contribution to the common good. It seems those days are past.

imagine that the four-color map problem (the first significant computer-aided result in mathematics) had been solved by a close-source environment

Mathematica isn't magic and the source code for your mathematica program is perfectly readable text. Reproducing the result from a mathematica program without mathematica is often time consuming and occasionally painful, but it's rarely a hard problem. So if the four-color map problem had first been solved using Mathematica, it wouldn't have taken many weeks for the solution to get reproduced in a different languages. Hell you could probably reproduce the necessary code just from reading the paper without ever looking at their source code.

Do you have a background in this topic that informs that opinion?

Not beyond having written a reasonable amount of Mathematica code and having a passable understanding of the basic 4 color theorm proof.

> Mathematica isn't magic and the source code for your mathematica program is perfectly readable text.

Yes, but the user's program only tells Mathematica what to do, not how to do it. How Mathematica does it is proprietary, and this represents a serious transparency problem for academic work and publication.

> Hell you could probably reproduce the necessary code just from reading the paper without ever looking at their source code.

If this were true, Wolfram wouldn't be able to charge thousands of dollars per copy of Mathematica. The reason Mathematica is expensive is because of what it conceals, not what it reveals.

I don't think they are losing mindshare. Mathematica is simply unparalleled in its features. Any stats showing they are hurt by the closed nature of their software?

Mindshare is a bit fuzzy, but at least among academics, I do think its closed-ness has made it lose some of its luster in recent years. Mainly because the bigger focus on reproducibility and open code and such has made people feel like it's not quite the "right" way of doing things, especially if you're preparing something for public consumption (vs. private experimentation), compared to e.g. the more community-based and open Python math ecosystem. Ten or fifteen years ago people didn't think about that as much, so people used Mathematica happily. Nowadays of course many people still use Mathematica, but, unless my circle is totally unrepresentative, a good number feel a little more uneasy and maybe slightly guilty doing so, and have at least a desire and vague plans to switch if/when they find time or the alternatives get better.

As a researcher currently working in Matlab, if/when I release code publicly I'll be porting to C++ and writing bindings to other languages. It doesn't feel like I'm really contributing much to the community if I release code that requires thousands of dollars to run.

GNU Octave is a free/open alternative for Matlab, which is mostly compatible with things built in Matlab. Just make sure your code runs in Octave and you don't have to do any porting.

There's no viable alternative to Mathematica, let alone a compatible one. Even for the basic stuff I've used Mathematica (student license) for, basic symbolic integration, differentiation and simplification, I get inferior results from Sympy and Maxima. I haven't tried XCas yet, some say it is better.

And this is only talking about the computer algebra capabilities. Mathematica has all sorts of nice tools for doing plots, interactive UIs and other stuff.

Here's a quick slideshow on Software Sustainability and Reproducible Research in Remote Sensing from Robin Wilson, https://speakerdeck.com/robintw/software-sustainability-and-... he also writes a blog that often discusses the subject in the broader sense http://blog.rtwilson.com/in-praise-of-projecttemplate-for-re...

> Any stats showing they are hurt by the closed nature of their software?

No reliable statistics, only the conjecture that Sage, sympy and other such environments allow people to produce a significant percentage of the results Mathematica can produce. Mathematica still has a substantial territory of advanced results unmatched by the free environments, but the gap is narrowing as the years pass.

I know this -- students are much more likely to acquire Sage or IPython than purchase even the lower-priced student version of Mathematica, simply because they can get the former for free, and there's plenty of high-quality documentation and examples available (example http://arachnoid.com/IPython). And how many undergraduate courses require the kinds of advanced results that only Mathematica can produce? And given those specific results, in an academic environment how much of a penalty is it that only the result is produced with no explanation or context?

I think Wolfram Alpha / Language is cool, but generally agree with your analysis. Is the Wolfram engine supposed to be a tool to better access indexed human knowledge on the net? Or is it about teaching people to think in a structured way about querying and manipulating this data? For the former goal, we really need to know what's going on behind the scenes of his program to weight the results. For the latter, what are we teaching if the language obfuscates the details of how it is interpreted?

The thing about Mathematica (and any other math product) is that it lets you increase the abstraction level of your thinking.

If you have a smaller problem, first type it into mathematica, and see if it comes up with an answer. It means that you don't have to divert your efforts into that small problem.

That said, mathematica has a function to have it explain exactly what it did to solve a particular problem. Example:

http://i.stack.imgur.com/V9dxn.png

Sure, Mathematica is great. But showing the formulas does not mean showing how the code was interpreted -- ie, translated into machine instructions. So it's good for exerting power over equations that have known algorithms for solving efficiently. Not to knock his achievements. I mean, what have I done that's so cool?

> The thing about Mathematica (and any other math product) is that it lets you increase the abstraction level of your thinking.

The difference between Mathematica and other environments is that in the latter, you ascend from the trivial to the abstract, hopefully learning the entire structure as you go, but with Mathematica, you're deposited on a mountaintop with no obvious pathway down.

> That said, mathematica has a function to have it explain exactly what it did to solve a particular problem.

Yes, but this doesn't work for the more advanced features. If it did, someone could simply copy down the conversion sequence and recode it for a different environment. As a result, Mathematica users are in the position of accepting results whose genesis is concealed.

I don't think Wolfram will open source their engine, ever. Their income is mostly from selling the software licenses an I don't know how they could replace that income.

However, it would be nice if they would open source parts of their application so bugs could be fixed in the user-facing parts. Even if their engine remains closed, the UI could be open sourced. I'm fine with them keeping their "secret sauce" proprietary and asking money for it. They spent decades making it work.

I am saying this because Mathematica for Linux doesn't work great. It's violating the X11 protocols and doing some crazy things with XSendEvents, which makes it not work at all on some window managers (window layout is completely fucked, mouse clicks aren't received properly, etc), while other WMs (e.g. i3wm) have Mathematica-specific hacks to ignore some of the messages it's sending.

I'm not sure what to do with this situation. For some of my projects (outside of paid work), I'd need Mathematica. So far I've used an educational site license for my university but now I've graduated and I won't have access to that.

So I could pay them $130 (student license, I still have my @uni.edu email for now) to $300 (normal license) but I don't know if they'd fix the issue. I could start using a more mainstream desktop setup (KDE, Gnome, whatever) in the hopes that it works, but that's not ideal either. I have not contacted customer support because I'm skeptical that they would do anything for such a small minority of their customer base.

Mathematica is almost the only closed source application I'd need. The open alternatives are not good enough and I'm not educated enough to improve them.

There is one thing I don't understand about your argument, Microsoft did not open source their entire stack, but what you're asking of Wolfram would be akin to releasing the whole Windows source code into the wild. Sure Wolfram would still exist in some sense, maybe as a consulting company, maybe just selling support, but it would become a fraction of its current (not huge anyway) size.

Given that FOSS still cannot properly replicate the workflow experience of Xerox PARC workstations and Wolfram tools are one of the best success stories of Lisp inspired work environments, I don't see that happen any time soon.

As another example, when will we have FOSS Notebooks that can match what Apple is doing with Playgrounds?

Having the means to the right funding makes a lot of difference, specially when many seem to be wary to pay for developer tools while gladly pay for any physical tools.

I'm not familiar with the workflow experience of Xerox PARC workstations. I am interested in novel and useful workflow experiences. Do you have sources that describe the benefits of the experience? If it is based on personal experience, can you elaborate?

My personal experience is based in Smalltalk VisualWorks and Oberon.

Smalltalk, well being Smalltalk.

Oberon being the product of Niklaus Wirth, after he learned Mesa/Cedar while at Xerox.

That experience, coupled with lots of archaeological digging for old manuals and papers is how I got to learn about it.

Basically Xerox had three workstation environments, Interlisp-D, Smalltalk and Mesa/Cedar.

All enjoying the fact of the workstation and OS blended into each other.

The CLI was a REPL, so you had full access over the OS. Any public API could be accessed in the REPL, you could interact with running applications, all the same way.

Also the REPLs were graphical with inline generation of data, so imagine something like Swift Playgrounds as the CLI.

Lots of nice documents here:

https://archive.org/details/bitsavers_xerox

Filtered by language:

https://archive.org/details/bitsavers_xerox?and[]=subject%3A...

Thank you for the clarification! I have worked with smalltalk (environment in which I learned OOP). I didn't realize that the default work environments in a Xerox PARC were REPL lisp, smalltalk or mesa (not familiar with mesa except as reference to mesa3d -- the same?). This sounds like an awesome environment. Mathematica doesn't really capture this either, not being integrated with the OS or third party applications, does it? Sounds like MS is trying to do this with PowerShell but with a poor interface compared to smalltalk or lisp (and probably less standardized). Did Xerox control all the software or mandate programming languages or interfaces?

Xerox controlled the whole stack, they invented it for their systems.

So these workstations where an whole stack OS.

You can read about their implementations on the links I provided before.

Additionally, you can get the original Smalltalk-80 from from Stephane's web site.

http://stephane.ducasse.free.fr/FreeBooks.html

This is a very ruft idea how it felt to use a Smalltalk workstation:

https://www.youtube.com/watch?v=8yxCJfayW-8

https://www.youtube.com/watch?v=JLPiMl8XUKU

The REPL was what is known in Smalltalk as transcript.

Similarly the Interlisp-D workstation also had they own mixture of REPL and editing

https://www.youtube.com/watch?v=wlN0hHLZL8c

Some of the ideas that lived on the Lisp Machines afterwards

https://www.youtube.com/watch?v=NOysrxexTXg

https://www.youtube.com/watch?v=o4-YnLpLgtk

Mesa was a memory safe system programming language created at Xerox, as an evolution from Extended Algol to replace BCPL. Niklaus Wirth based his Modula-2 design on it.

Shortly thereafter they updated Mesa into Cedar, which added support for RC with local GC for cycle collection. The system then got called Mesa/Cedar.

It allowed the same interactive experience as the other workstations, but using a GC enabled systems programming language.

The REPL provided nice features, like auto-suggestion when a typo would cause a compilation failure. It is also probably the first graphical debugger for a strong typed language.

Any OS API could be used on the REPL by typing modulename.procedure , which could use other OS APIs to get its input from different sources.

An idea Wirth adopted into Oberon.

You can see how all three environments looked like here:

http://www.chilton-computing.org.uk/inf/literature/books/wm/...

And yes, Powershell alongside .NET is probably the closest we have in modern computers to those systems.

Followed by Swift Playgrounds and AppleScript on Apple systems.

Although Apple also provided similar experiences with Common Lisp, Hypercard and their NewtonScript and Dylan.

Personal historical favorite:

MIT Lisp Machine screenshots from 1980:

http://bitsavers.informatik.uni-stuttgart.de/pdf/symbolics/L...

Thanks, very nice screenshots.

Yet another proof that the success of AT&T meant a left turn.

Could somebody give some examples of things that the Axiom family, sympy, Maxima, Reduce, Sage, Giac/XCAS ... can't do but Mathematica can.

Basic computer algebra.

Symbolic integration, differentiation and simplification just works miles better in Mathematica than any of the open alternatives I've tried.

It's quite common that the open alternatives just choke on something quite simple. In Mathematica, you can just use Simplify[] to get a nicely simplified expression. In Maxima/Sympy, you might be able to get the same result but you need to understand how the simplification algorithms work, ie. you have to choose which simplification method to apply based on the task and might need to apply more than one method to reach the goal.

Additionally: Mathematica is pretty darn simple to use and well documented. Going through the docs of Maxima, Sympy, etc requires you to be a domain expert in computer algebra. I am not. I just want something to assist me when a basic algebra task would require several sheets of paper and I'd be likely to make mistakes.

I am not even going to talk about the UI, plotting and other features. I haven't used them much.

I have not tried XCas yet, perhaps it's more suitable for my purposes. Some commercial calculators (ie. physical devices) are based on XCas.

note: I've been mostly working on some basic university level physics related to space flight and orbital mechanics. Stuff like Kepler's equations or rocket equation solutions.

Also, if they don't go FOSS, their users are living under a Sword of Damocles: At any point, they can choose to change everything, break everything, and your recourse is one of a variety of impolite ways of saying you have no recourse.

Being a customer doesn't save you, BTW: Giving them money insulates you not one whit from their decision that a course of action which destroys your whole setup will get them more money. This doesn't really change even if you have a contract: Unless you convince a thoroughly insane court to order some form of specific performance, the most you're entitled to is, essentially, a refund, maybe some punitive damages, and Wolfram no longer caring about you even to the extent they previously did.

I see Wolfram's offerings as an incredibly convenient prototyping and educational tool. I can't really imagine anyone sane is using it for anything that would result in a disaster significant enough to evoke a sword hanging over ones head.

Reply on news.ycombinator.com