taliesinb

We are going to make the core Wolfram Language available as a cloud computation type platform in the near future. The main idea is that anybody should be able to run Wolfram Language code using their browser as a frontend, without necessarily having a license.

Specifically, I would be very surprised if small programs are not free for anyone to run, and if we don't offer some free tier (X minutes of kernel time a month).

Really, we have a strong interest in making WL ubiquitous for doing algorithmic and knowledge-based work. That does certainly mean opening it up more, though as some other people have pointed out, that is probably not going to take the form of open-sourcing the kernel [at least, not until we have different revenue streams in place].

Some corrections, from an enthusiast of some of the ideas of NKS and someone in the R&D division of Wolfram Research:

1. Cellular automata are not the "point" of the book. The point of the book is the study of simple computational systems: cellular automata, mobile automata, Turing machines, register machines, tag systems, string rewriting systems, graph automata, lattice automata, etc..

What are they?

How powerful are they?

What is the distribution of complexity of these systems?

How might they hint at an unexplored scientific territory that we could profitably investigate?

What systems in nature can they qualitatively model?

How well does traditional mathematics work in analyzing and predicting their behavior?

The complexity manifested by these systems have been sporadically encountered by various people. A classic example being of course being Conway's Game of Life. But that was seen as pure recreation. Not until much later did anyone engage in systematic enumeration of similar systems in order to determine if Conway's game of life was special in any sense, if its universality was a unique feature of such systems.

Even the genius John von Neumann, who invented CAs, didn't simulate any to see what they actually did. He just saw them as models for engineering, and went about designing a somewhat pointless and brittle self-reproducing CA that has about zero significance for the [then non-existent] field of artificial life.

Similarly, Turing used Turing machines them to build a universal machine, and that's as much as he cared about running Turing machines. He almost immediately went on to oracles and supercomputation, which are pretty abstract and arguably far less important ideas.

2. Wolfram didn't invent many of the things in the book, and as far as I'm aware, doesn't claim to. In the very detailed notes you'll find plenty of attribution (although many academic fields are necessarily summarized very briefly). What he does claim to be the first to have done is articulate a common thread of argument that connects these existing ideas in a coherent and explanatory way.

3. This isn't about simulation. Simulation is precisely the WRONG way to look at what he's saying. What follows is a particular case he covers in the book, one of many such cases, but it is a nice illustration.

Turbulence in fluids is one of those 'puzzling' things that we often describe as a complex and interesting phenomenon in nature [0]. The traditional approach to understanding what causes turbulence is to use non-linear partial differential equations to describe the interplay between velocity, density, and pressure in fluids [1].

When you ask what these equations "really mean", there is a sense in which you have to first explain some pretty complex mathematical machinery, culminating in the very idea of a real number -- that take tens of years for modern-day students to learn in detail, that took hundreds of years for professional mathematicians to construct. These aren't trivial, or even particularly natural, ideas [2].

For some very simple cases it is possible to use the same mathematical machinery that expresses these equations to then solve them -- although enough fundamental stuff is still missing to warrant a $1m Clay prize [3].

But many questions (include almost all practical ones) are far too difficult to solve using the same "mathematical machinery" that expressed the equations. So we end up instead discretizing time and space and finding approximations to the solutions of these equations (using floating point numbers that are themselves approximations to the reals).

But wait a minute! Fluids are COMPOSED of discrete particles. The Navier-Stokes are just an approximation in the case of large numbers and statistical independence of momenta. So we've built approximations of approximations of approximations, at great computational expense and effort, with very little of the clarity and elegance of Newton's original work in classical mechanics [4].

We've got this very tall tower of ideas, with many answers still unclear (eg [5]). Can we refactor?

A natural question to start with is the following: if you don't limit yourself to continuous mathematics, how much "computational machinery" do you need to get fluid turbulence? It turns out, as Wolfram and others showed, all you need a system that is simple enough to explain to a 5-year-old [6]. (In fact, if you have ever played Chinese Checkers, you have a rough feel for how it would work).

These lattice gas automata have subsequently led to industrially useful CFD solvers [7].

In short, you can discretize both space, time, momentum, and break Galilean invariance, and my god it still works! That is a remarkable result!

What's the point of all this? The point here is that many of the traditional assumptions one makes in mathematical physics turn out not to be critical to reproduce the behavior one is interested in. And this appears to be a general phenomenon in many fields: the "unreasonable success of mathematics" has lead us to in some sense define science to be about building idealized mathematical systems.

Sometimes our ignorance of the 'spikiness' of computation has lead to some wild goose chases. [8]

And science has in turn ignored many problems that weren't amenable to mathematical analysis. Not out of pride or incompetence, but because no-one had made a particular effort to articulate what other kinds of analysis were possible. That's what the book tries to do, and that is why it perhaps seems quite hubristic. It is a manifesto. You can't write a manifesto in the form of 60 consecutive peer-reviewed papers.

I think there are faults with the book, particularly its length, its pedantry, and the assumptions it makes about its readers. But it is a radical scientific view, makes some quite big claims and predictions, perhaps wrong, perhaps right. I think it is definitely worth dipping dipping into at random for an educated generalist (rather than reading sequentially, which can be quite tiring).

Anyway, this post is far too long, and I'm not getting paid for defending Wolfram's work. I have Mathematica kernel functionality to write, dammnit!

[0] http://www.wolframscience.com/nksonline/page-377

[1] https://en.wikipedia.org/wiki/Navier%E2%80%93Stokes_equation...

[2] http://plato.stanford.edu/entries/mathematics-constructive/

[3] http://www.claymath.org/millennium/Navier-Stokes_Equations/

[4] https://en.wikipedia.org/wiki/Philosophi%C3%A6_Naturalis_Pri...

[5] https://en.wikipedia.org/wiki/Turbulence#Kolmogorov.27s_theo...

[6] https://en.wikipedia.org/wiki/Lattice_gas_automaton ; http://www.wolframscience.com/nksonline/page-378#previous

[7] https://en.wikipedia.org/wiki/Lattice_Boltzmann_methods

[8] https://en.wikipedia.org/wiki/Principia_Mathematica

Can you provide a specific example of a scientific discovery he claims is his own but was in fact discovered by someone else?

According to Wolfram, Cook's job was to prove Rule 110 was universal. Cook didn't "discover" it as an independent action.

In a very direct sense, this was a software development task, where the program that was being written was precisely a universal program (through several layers of emulation, of course; rule 110 -> 'particle computer' -> cyclic tag system -> Turing machine). It's an impressive program, but also quite mechanical. More details: http://www.wolframscience.com/nksonline/page-681#previous

Here's the timeline as far as I've been able to determine, mostly from sources inside the company:

0. Wolfram tasked Cook with finding a proof that 110 was universal.

1. With encouragement and help, Cook finished the proof many years before the book was ready.

2. Cook tried to publish the program/proof at some complexity conference.

3. Wolfram asked him not to, because it was against the terms of his contract.

4. Cook agreed not to, but then did it anyway.

5. Wolfram threatened legal action to prevent the proof being published.

This kind of thing is clearly a breakdown of the relationship between Cook and Wolfram. Is there a good guy and a bad guy? Maybe, maybe not. But if we see this proof as a program, I'm sure many people would agree that publishing code one was paid to write, without permission, is kind of a no-no.

To my mind, this doesn't so much paint Wolfram as a lawsuit-happy egomaniac as it does expose some of the contradictions of a running a private company as a commercial venture and as a vehicle for one's own research.

edit: I work for Wolfram, as probably some HN users already know and as I mention elsewhere on this thread. But when I heard about the whole Cook thing a couple of years ago, I asked around about what happened to make sure I wasn't working for a company that was unethical. The above is what I was able to determine.

Here's a more hacker-ish explanation about what is different and cool about Mathematica: https://news.ycombinator.com/item?id=4015695

Your comment makes me feel really young! I'm younger than Wolfram Research but older than Mathematica.

Having just attended our 25 anniversary celebration in Illinois, it's clear there is an enthusiastic and long-lived community around Mathematica that is only going to grow.

With any luck, 2014 will be a "breakout" year in which the Wolfram Language that underlies Mathematica will start playing a more mainstream role in the software engineering industry.

Weirdly, Mathematica integrated support for high-performance numerical linear algebra libraries from Intel and others a couple years before MATLAB did.

So your point about symbolic versus numeric flavor might have been correct prior to version 6, but it is no longer true (Mathematica 10 will be released later this year). In fact, they now use many of the same libraries for pure numeric work.

Numeric (as well as hybrid numeric+symbolic) computation is possible in Mathematica. In MATLAB, symbolic math requires a clunky add-on that is far from competitive with what Mathematica can do.

I work at Wolfram.

Yes. Probably much longer.

It's more like the world is going to be radically different place in 50 years, and owning property in a formerly 'important' part of the world doesn't quite have that cachet anymore...

Though I'm not a singularitarian, I do think we are probably going to live an order of magnitude longer than our parents.

Given that, buying a house seems too... parochial.

2. Re zero value: If pointers were by default not nullable, then the zero value of a struct containing pointers would have to recursively allocate objects to fill those pointers. That's a deal-breaker for the notion of zero values.

3. Null values are almost always what result, as opposed to undefined or incorrect values. So it's a product type that is effectively a sum type. As for code smells: actually, there is a greppable equivalent in Go, and it is "... , _ := ... ". Future Go compilers might allow for a flag that treats "error" types specially in this regard.

Interesting to hear about fromJust, didn't know about it.

My own experience is that unchecked errors aren't where Go's 'debugging load' lies. The load is often in reasoning about concurrence, causal dependancies, and state machines.

4. If you're saying that because pattern matching isn't a core feature of Go, you wouldn't use it for everyday code, you're right. I'm a fan of pattern matching, I use it in Mathematica all the time (albiet unchecked). But my point stands: you can build sum types through interfaces, and when that makes a design significantly cleaner, a decent Go programmer will use it.

5. "Type systems don't become cages": well, a lot of people will disagree with you there. Ask C++ programmers about const. Ask Java programmers about checked exceptions. Ask Haskell programmers about logging. So I guess that's just, like, your opinion, man. :)

5.1. Go may not offer you as rich type verification as Haskell, but it does things Haskell can't: it can detect race conditions, for example, in concurrent code [I shudder to think how you even write concurrent code in Haskell].

Another closing thought: you focus a lot on Haskell. Haskell seems to be a very fecund place to devise type-theoretic patterns and techniques. And it seems to have taken the niche of "programming-language research testbed", which also results in a lot of interesting ideas. But there isn't nearly as much evidence that it is a smart choice for production code.

I've been impressed by what I've seen, read, and understood about Haskell. Still, I have an unshakable impression that a lot of the quality that is associated with Haskell code is explained better by the hypothesis that the bar for writing Haskell code is very high: bad programmers can't write Haskell programs that do the things they want to do, so we see a lot of nice Haskell code and little bad Haskell code because of the base competency of the audience.

But that doesn't imply that an experienced programmer would write higher quality code in Haskell than in Clojure or Go. We all have a complexity budget to spend on using tools to their full potential and doing things a certain 'proper' way. Simple languages leave more of that budget to spend on your program itself. Type-heavy languages leave less, with the promise that the cost is amortized. Calling Go a "sad language" and implying its authors are incompetent because you don't agree with their particular choice of budget demonstrates, to me, a certain arrogance and lack of imagination.

1. Parametric polymorphism may still come. I think we need to understand their connection with interfaces and builtins before it will be clear what to do. A thousand frameworks may need to bloom before we have enough data to do that.

2. Nullability IS a kind of sum... and the idea of a zero value would be impossible (or at least ruinously inefficient) without nullability. Secondly, the fact that all dereferences are checked gives you most of the same benefits, just minus all the type noise.

3. Products for errors are actually a GOOD thing, in that they allow developers to easily choose between pedantic error handling or relying on downstream panicing to handle situations that are unrecoverable anyway. Whereas with type-level enforcement, you HAVE to do all that extra checking, and that straitjacket costs you clarity and elegance.

4. Interfaces do allow for sum types, and type switch statements and/or dispatch allow for pattern matching. Not terribly elegant, but effective and understandable.

My closing thought: type systems are crutches for reasoning about the behavior of programs at compile time... and we need all the help we can get with that challenge, so yay for type systems!

At some point, however, they become cages, because there will always be a frontier beyond which they will be unable to encode contraints that are obvious to the person writing the program, and at that point, the resulting contortions will obscure, not enlighten.

I love how the dialogue on HN about Go has gone from pessimism and largely uninformed criticism, to regurgitation of the team's own talking points ["simple, orthogonal features"], to a more nuanced appreciation of what trade-offs Go makes.

Hopefully these are individual developers shifting through a continuum of enlightenment, rather than the conversation itself migrating to a more enlightened population.

This last question is testable, of course, though sadly HN does not offer an official API.

Speaking of bagging: Geoff Hinton talks a bit about model averaging and has a very stimulating pair of hypotheses related to this, which are:

a) genetic recombination in the form of sex might be "bagging across genes" that prevents tight co-adaptation (= overfitting on the evolutionary timescale)

b) the brain uses noisy discrete firing instead of continuous communication because it allows for "bagging across network topologies".

The bulk of his talk is about how dropout in neural nets can be used to accomplish some of the same performance benefits that model averaging gives on other algorithms (at a much smaller relative computational cost).

Here's the talk: https://www.youtube.com/watch?v=DleXA5ADG78

You're right, that was actually a really good explanation. I'm sorry, I misinterpreted your comment! Retroactively upvoted, for what it is worth.

> if you're going to launder money, you should launder a whole shitload of it

Yes. Or rather, if you're going to launder money, make sure you're making healthy campaign contributions, hiring ex-politicians and their staffers as lobbyists, oiling the great lobbying machine, etc...

For those who haven't read it, Lessig has written a rather nice book on exactly how this kind of "soft corruption" works: http://www.amazon.com/Republic-Lost-Money-Corrupts-Congress/...

Tooling should get us part of the way - it surely cannot be that difficult to interactively show which interfaces a type satisfies (and conversely, what you'd need to add to a type to have it satisfy an interface).

In general it would be neat if the "go" command supported some more advanced static analysis stuff beyond go fix, fmt, and vet.

A lot of this stuff could be done by reflect -- if only there was a repl!

A few years ago I spent a couple hours prototyping a tool for retrofitting some of these ideas onto existing forums. I didn't get super far, but I had some nice ideas.

Here, for example, is an HN discussion (hopefully what is going on should be self-evident): https://twitter.com/taliesinb/status/338742471574040577/phot...

We use a similar thing internally at Wolfram to diff Wolfram Language expressions, though I believe it doesn't have the subexpression matcher. But the diffs are much clearer when you have full m-expressions instead of s-expressions.

The coder who wrote it (@wmacura) now works at Tumblr.

That is exactly how I started! Today's walled gardens don't seem to contain the same sandpits we had in the days of DOS and friends.

Yes, I'm quite aware of this. Not to be rude or anything, but it is kinda obvious if you've thought at all finite computational systems.

Many of these are quite beautiful.

The NKS-style question to ask here is: what computations are these doing? Totally unique-unto-themselves computations? Potentially useful computations? Computations analogous to familiar human ones? How would we know? Can we know? Do we run into the limits of undecidability?

The cyclic boundary conditions somewhat 'spoil' things, though. Maybe a particular rule was "destined to multiply input by 5" or "calculate log-2 of input" or something (given suitable encoding of input on the tape), but the computation is foiled as soon as the machine wraps around and starts interfering with itself.

Then again, the self-interference is what makes many of these patterns do the cool things they do.

Wolframite here: these would make a good summer school project for our summer school (plug: the 2013 one is coming up, apply at https://www.wolframscience.com/summerschool/ if you're interested in this kind of stuff and want to do a 3-week project).

In fact I studied the space-filling behavior of precisely these 2D Turing machines in 2009. For low numbers of states and colors you can enumerate them exhaustively and calculate the distribution of space-filling efficiency (log-normal, if anyone is interested).

The full spectrum of behavior is quite fascinating to catalog. All these kinds of simple computational system have a character and 'zaniness' all their own.

Putting on a more scientific hat, it might be interesting to look at block-entropy measures and find interesting rulesets that way.

P.S. To the author: this is a great implementation! Really nice to play with. Check out http://reference.wolfram.com/mathematica/ref/TuringMachine.h... for the one built into the Wolfram Language.

In fact we (Wolfram) hired the primary Mathics developer to work on Mathematica Online, which will be released in the next few months: http://www.wolfram.com/mathematica-online/

Beta testers wanted!

After reading "The Blind Watchmaker" I did something similar to this project -- in Visual Basic! (My excuse for using such an abhorrent language is that I was only 10 and didn't know any better).

Which makes me think that this project is ripe for a "artificial selection" mode in which you get multiple mutated copies and pick the right one, slowly directing the parameters in a 'pleasing' direction. Here's a cool video that demos Dawkins' watchmaker program from way back when: https://www.youtube.com/watch?v=nUkL2bLFiOU

Wolfram and Kurzweil are betting on opposite horses here. I work for Wolfram, but I'd be delighted if the ML-approach works. Personally, I think we need a diversity of approaches to really understand how to crack the nut of 'hard AI' (if that is even a sensible thing to pursue).

I've been to a couple Kurzweil talks. He's got a good shtick, but it doesn't seem to change much. I tweeted "I saw this talk 10 years ago. Shouldn't it be 5 minutes long and 100x more interesting now?"

This is a great point. Gabe Newell makes the same point about the gaming industry: the short-term view is that gaming is about selling units. The long-term view is that gaming is a cultural activity that can be monetized in an almost infinite number of ways.

It's probably smaller than we all think. Google has a lot of products, even after graveyarding a large number of them.

These are interesting examples. Have you given any talks or written any blog posts about these results?