taliesinb

This is really cool! My question is: how much of the managed time philosophy could be bolted on to a functional reactive programming environment like Elm?

No, because all that is required for operator forms to do what you write is just ordinary function application. Which they already do just fine.

Rich is talking about something deeper, in which operators like filter become transformers that themselves operate on stateful processors (reducers) to produce new stateful processors that have incorporated the action described by the original operator.

Yeah, it's pretty easy to turn

    Map[#^2&] /* Select[PrimeQ] /* Select[OddQ] 

into

    Select[OddQ[#] && PrimeQ[#]&] /* Map[#^2&] 

and so on via rules that look like:

    Select[s1_] /* Select[s2_] :> Select[s1[#] && s2[#]&] 
    Map[f_ ? SideEffectFreeQ] /* s_Select :> s /* Map[f]
    Map[Extract[p_Integer | p_Key]] :> Extract[{All, p}]

I'm already doing a bunch of that for Dataset.

But in reality you need to move to a DSL for complex enough rules, and then Clojure and WL will be on the same footing (Clojure even a bit stronger, maybe, WL doesn't really have a proper macro system).

Does Clojure core do or allow for any of that kind of optimization already?

WL doesn't yet have a laziness/streaming/reducing framework, but the prototype we're working on uses 'operator forms' like Select, Map, GroupBy and so on in the way you describe.

I don't think the exact details are the same, because our operators don't actually evaluate to transformers (they remain totally symbolic). Rather, the conversion of composed operators to an actual reducer pipeline happens lazily 'at the right time', which I think will make optimization a bit easier to express.

We just did exactly the same thing in the Wolfram Language, for similar reasons (we called these things "operator forms" rather than "transducers") [0]

One major side effect has been to mitigate the kinds of heavy nesting you see in functional languages like WL and Clojure. Personally I think the resulting code resembles the phrase structure of English much more closely. It's a huge readability win.

The original motivations for operator forms were in fact writing Queries [1] against Datasets [2], for which you want to represent complex operations independent of their execution.

[0] http://reference.wolfram.com/language/guide/FunctionComposit...

[1] http://reference.wolfram.com/language/ref/Query.html

[2] http://reference.wolfram.com/language/ref/Dataset.html

When I was working on http://blog.stephenwolfram.com/2013/04/data-science-of-the-f..., a lot of bizarre results (such as a huge gender difference in number-of-friends for people older than 40) went away after I took into account the friendship paradox. It would be really interesting to revisit that anomaly and try to explain why.

Turns out the trial version is (probably accidentally) quite broken: http://community.wolfram.com/groups/-/m/t/290210?p_p_auth=uf...

It's true it can't use libraries like the Intel Performance Primitives on ARM, but that's not why the UI is slow on RPi. RPi's UI is slow because all graphics-intensive GUI stuff on RPi is slow. Firefox is unusable, for example.

Looks like it... I would delete both from disk, redownload 10, and reinstall it. Also check that you didn't run out of disk space.

If all else fails, phone our technical support at 1-800-WOLFRAM, they're pretty good.

Mathematica 8 doesn't have associations, so you must be testing symbol assignments.

Write a = <||> before running the above code to test this on associations; associations are persistent, the symbol table is not.

You can make your own opaque data objects, quite easily, thanks to HoldAll, UpValues, and Internal`SetNoEntry (the nuclear option).

What precisely becomes unmaintainable about Mathematica/Wolfram Language code after 10 lines? You could just be bad at programming in WL.

What is your value of $Version and SystemInformation["Small"]? I can't replicate these problems on any of the 3 platforms using freshly-downloaded versions.

I think you put your finger on it... people talking about Mathematica/Wolfram Language as a CAS are really chasing an idea of what it was 10 or 15 years ago.

Maybe I'm naive but I think that we really can just "all get along" -- notebook-based programming is just one example of how ideas can incubate in closed-source projects, and end up benefiting open-source projects. I'm hoping knowledge-based computation ends up being another.

And the reverse happens too -- Light Table is tremendously exciting, and I hope WRI can learn from that as it develops.

Yes, Associations use the same basic algorithm as Scala and Clojure's hashmaps, and are indeed persistent.

And b[x] there gives 1 as you would expect.

Well, unlike adding and changing definitions for symbols in the symbol table, associations are actually immutable. So you can pass them around as values, mutate them, and the old versions are still available unchanged. That's just not true of using downvalues of symbols.

Also, using definitions on symbols to store key-values is incredibly limiting, because you can't do reverse lookups without getting into very low level (and slow) hackery via DownValues. Or list the keys or values. Or even know how many keys you have.

Associations really will replace basically all uses of symbol downvalues for storing key-value pairs. And Associations have really nice general-purpose functions like GroupBy, Counts, Merge, and so on (mentioned below).

Associations even support function application syntax, so you should be able to drop them in into existing code that expects symbol downvalues as you're using them above.

Associations are the single most significant language-level improvement we've had in many versions. And we've integrated them really nicely across the system -- they're not just some special-purpose data structure.

Associations aren't just a data structure, they've been designed to fit in a sensible way into the rest of the language, via a principle I call the "central dogma". This means they work in a predictable way with a huge number of existing functions (though we still have more to do).

For example, Associations interact naturally with the hierarchical part-specification language used by Part (http://reference.wolfram.com/language/ref/Part.html):

   In[1]:= people = { 
      <|"name" -> "bob", "age" -> 20, "sex" -> "M"|>, 
      <|"name" -> "sue", "age" -> 25, "sex" -> "F"|>, 
      <|"name" -> "ann", "age" -> 18, "sex" -> "F"|>
   }; 
   
   In[2]:= people[[ All, "age" ]] (* extract list of ages *)
   Out[2]= {20, 25, 18}

   In[3]:= people[[ All, "sex" ]] (* extract list of sexes *)
   Out[3]= {"M", "F", "F"}

   In[4]:= people[[ 2, "age" ]] (* extract age of 2nd person *)
   Out[4]= 25

   In[5]:= people[[ 2, {"age","sex"} ]] (* extract age and sex *)
   Out[5]= <|"age" -> 25, "sex" -> "F"|>

This naturally generalizes to 'indexed tables', in which the outermost list becomes an association, because associations serve double-duty as "structs" and "hash-maps", just like lists are used for both "vectors" and "tuples":

   In[6]:= people = <|
      236234 -> <|"name" -> "bob", "age" -> 20, "sex" -> "M"|>, 
      253456 -> <|"name" -> "sue", "age" -> 25, "sex" -> "F"|>, 
      323442 -> <|"name" -> "ann", "age" -> 18, "sex" -> "F"|>
   |>; 
   
   In[7]:= people[[ All, "age" ]] (* extract association between ID and age *)
   Out[7]= <| 236234 -> 20, 253456 -> 25, 323442 -> 18|>

   In[8]:= people[[ All, "sex" ]] (* extract association between ID and sex *)
   Out[8]= <| 236234 -> "M", 253456 -> "F", 323442 -> "F"|>

   In[9]:= people[[ Key[323442], "age" ]] (* extract age of person with ID 323442 *)
   Out[9]= 18

   (* extract age and sex of person with ID 323442 *)
   In[10]:= people[[ Key[323442], {"age","sex"} ]] 
   Out[10]= <|"age" -> 18, "sex" -> "F"|>

The uniform addressing scheme behind Part (and Extract, Position, etc) is tremendously useful in day-to-day code, because it makes it much easier to write programs as functions that transform potentially complex, hierarchical data in a series of steps.

This is similar in some ways to the ideas behind Haskell's lens library, Clojure's assoc-in and friends, even the schemes used in JQuery and XPath. But it's core to WL.

The semantics of Part are also extended to become a full-fledged query language, as used by Dataset (http://reference.wolfram.com/language/ref/Dataset.html):

   (* load a dataset of passengers of the Titanic *)
   titanic = ExampleData[{"Dataset", "Titanic"}]

   (* produce a histogram of passenger ages *) 
   titanic[Histogram, "age"] 

   (* produce a histograms for 1st class, 2nd class, etc.. *)
   titanic[GroupBy[Key["class"]], Histogram[#, {0,80,4}]&, "age"]  

There are also some really nice functions to work with associations, like the map-reduce-like GroupBy (http://reference.wolfram.com/language/ref/GroupBy.html):

   (* split sentence into list of words *)
   In[16]:= words = StringSplit["it was the best of times it was the worst of times"] 
   Out[16]= {"it", "was", "the", "best", "of", "times", "it", "was", 
      "the", "worst", "of", "times"}

   (* group words that have the same length *) 
   In[17]:= GroupBy[words, StringLength] 
   Out[17]= <|
      2 -> {"it", "of", "it", "of"}, 
      3 -> {"was", "the", "was", "the"}, 
      4 -> {"best"}, 
      5 -> {"times", "worst", "times"}
   |>
   
   (* reduce each group into an association of counts *)
   In[18]:= GroupBy[words, StringLength, Counts] 
   Out[18]= <|
      2 -> <|"it" -> 2, "of" -> 2|>, 
      3 -> <|"was" -> 2, "the" -> 2|>, 
      4 -> <|"best" -> 1|>, 
      5 -> <|"times" -> 2, "worst" -> 1|>
   |>

And Counts and CountsBy (http://reference.wolfram.com/language/ref/CountsBy.html):

   In[21]:= CountsBy[words, StringLength]
   Out[21]= <|2 -> 4, 3 -> 4, 4 -> 1, 5 -> 3|>

And AssociationMap (http://reference.wolfram.com/language/ref/AssociationMap.htm...):

   In[23]:= AssociationMap[WordData[#, "PartsOfSpeech"]&, words]
   Out[23]= <|
      "it" -> {"Pronoun"}, "was" -> {"Verb"}, 
      "the" -> {"Determiner"}, 
      "best" -> {"Noun", "Adjective", "Verb", "Adverb"}, 
      "of" -> {"Preposition"}, "times" -> {"Noun"}, 
      "worst" -> {"Noun", "Adjective", "Verb", "Adverb"}
   |>

Here's some more info about associations: http://reference.wolfram.com/language/guide/Associations.htm...

Here's the list of what's new since version 9 of Mathematica: http://www.wolfram.com/mathematica/new-in-10/

Here's the list of what's new since version 9 of Mathematica:

http://www.wolfram.com/mathematica/new-in-10/

You're right, it is more of an emacs-like tool. Probably in the next year or so we will consolidate some ideas of what an IDE should (or could) be for WL. Some Brett Victor like things should be in there, no doubt...

And yes, we need to get PacletManager into a state where it is easy to use as a package manager. The only thing (and a big, crucial thing) it is missing right now is explicit dependencies, and a solver. If you have ideas about this please offer them (are you on the Mathematica stack exchange?)

I've been told I'm wrong, Mathematica 10 will have cloud integration, but with a product called Mathematica Online (see wolframcloud.com for the whole family of cloud products).

Mathematica 10 is being released shortly, that has plenty of new things (thanks to the Wolfram Language it shares with the Programming Cloud) to tempt you! GeoGraphics, Entity, computational geometry, machine learning, Association, Dataset, SemanticImport...

Note that's API and knowledgebase computation time, not notebook computation time, which doesn't cost cloud credits.

Cloud credits come with each plan, as well, so you have some leeway for development and testing before you have to buy additional cloud credits at a base price 0.03 cents per credit.

http://www.wolfram.com/cloud-credits/ has more info.

Entity is part of the documented language -- we've standardized the representation of millions of real-world things. Looking up properties of Entities and parsing text into Entities depends on Wolfram|Alpha, just like the ServiceExecute function relies on Twitter and Facebook to work, and just like Graphics relies on a front-end executable to actually render.

These aren't libraries as you might call (say) core.logic in Clojure, because Entity and co integrate tightly across the rest of the language, and because they all share the same System namespace. They aren't quite DSLs either, because if they were the entire language would be a DSL -- rather the language is symbolic, which gives it some properties in common with DSLs.

If there is nomenclature that is different or new, it is mostly because the language is different. The only thing I can't defend is the pure in our "pure functions". They're not pure functions, they are lambdas.

Those are concrete claims (wrong or right), but they are made after Wolfram has shown why he thinks they are so, not at the beginning of the post like the now-deleted parent quoted.

Jordigh, that's not the first time you've made that evidence-free claim. Let me make my own: your hostility has less to do with Wolfram and more to do with you. You develop an open source alternative to Matlab. But if the Wolfram Language is really successful, it would make the work you've done fall short of its goal, which is to put all science and math computation on a free, non-proprietary basis.

Let me correct you: Mathematica is a desktop product, specifically an IDE for the Wolfram Language. It is not a language, though before we had the name "Wolfram Language" it was used to mean that. Other such IDEs are Wolfram Desktop, the Eclipse-based Workbench, and the cloud-based Programming Cloud that is being announced here.

And although I'm biased, I don't think it is unfair for our language to meter access to an expensive resource in the form of real world data. There are complex translation layers that mediate access to a diversity of different databases, real time and static, and merge them to present a nice, clean logical view.

On top of that, cleaning and curating the data is expensive, and it has to be done in perpetuity -- think how often PersonData (http://reference.wolfram.com/language/ref/PersonData.html) needs to be updated, for example. Some data we also have to buy outright.

Should this prevent us building such real-world data into the language? I suspect in 20 years this idea will be common in new languages, and probably free.

For now, it needs to be subsidized, and the basis is fair: we charge Cloud Credits on the basis of the amount of time our servers are kept busy: 0.3 cents per second of computation time on our API servers. As we make our APIs faster, that effective cost will only go down.

You use that quote misleadingly. Stephen (not Steven) wrote:

"My goal with the Wolfram Language in general—and Wolfram Programming Cloud in particular—is to redefine the process of programming"

That's a goal, not a claim about the product. And one that many people in the industry share, and routinely espouse.

AFAIK, Mathematica 10 won't have cloud integration. And the semantic stuff will be present, but will be "trial grade".

The emphasis in Discovery and Data Science platforms will be around workflow and reporting, though the features there haven't been finalized yet.

Wolfram Desktop has built-in integration with the cloud -- it can save and load straight from your cloud files.

It also sheds a lot of the historical computer algebra baggage of Mathematica -- palettes for entering equations and so on are gone.

And although it doesn't support this yet, Wolfram Desktop will soon let you connect straight to a cloud kernel. It'll be really cool: you'll write ParallelMap, and (say) 32 kernels in the cloud will be summoned for you.

I should also add that the "Things to try" notebook has some simple examples that demo various kinds of functionality: https://programming.wolframcloud.com/objects/7ea9a27a-1ea8-4...