taliesinb

This is 'just' a fun project, and there's a veritable cottage industry of fun projects like this to illustrate and play with new ML techniques. They're not meant to be technical contributions, they're playful doodles.

Plus, this one is actually really well done! It's got a great oldschool dictionary typeface, a tumblr reblogging the tweets, nice details like that.

What delightfully deranged nonsense!

ojno: n. (pl. ojnos) 1 a small piece of metal with a long glazed stem and a pointed snout, used for making soft fabrics.

It's like funhouse mirror Borges.

I was told we were eating a quagga, an extinct species of zebra that was being bred back on the grounds of various facilities in the area. They have to periodically cull the quagga that roam free on the grounds of the particle accelerator.

When people ask how it tasted I say "rare".

Classic.

http://www.nature.com/ncomms/2015/151027/ncomms9661/full/nco...

Short 3 minute summary video from the supplementary materials, which has a fair amount of eye-candy: http://www.nature.com/ncomms/2015/151027/ncomms9661/extref/n...

Yeah, it seems impossible because changing the stride changes the size of the output tensor completely.

I see, that's cool. So you could define a 'symmetry operator' the defines a regularization term as e.g. k * sum_{ij}[(w_{ij} - w_{i+i',j+j'})^2] (modulo my hand-wavy tensor indexing). Then the dk tells you how much a layer wants to be a convolution and di', dj' tells you what convolution kernel size it wants to be.

(edit: made less badly articulated)

You kind of allude to things like this with the "discrete parameters" comment, but here's how I imagine one could use this to meta-optimize layer sizes, which are after all continuousish: make all layers too large, then compute the hyper-derivative dn_i of a per-layer cutoff that starts regularizing units more and more strongly beyond some unit index n_i for each layer i (as in the regularization strength is sigmoidal, or better exponential, centered around n_i). Does that make sense?

Also, I immediately wanted to see vector gradient field plots in hyperparameter space of a bunch of toy models. It'd be fun to actually see "hyperconvexity".

Comedy gold.

It's quite a low-key affair (no GPU clusters and so on) but what I think is remarkable is that it described a fairly complex cognitive (neural) architecture that was able to mimic certain kinds of child-level cognition with TINY amounts of training data. Instead, human supervisors guided the evolution of its cognitive strategy in a white-box kind of fashion to encourage it to answer questions as children did.

In many respects (like training data volume, layer depth, and so on) it couldn't be further from the current deep learning trends, and yet it seemed much more along the lines of what I imagine an actual AGI would be doing, especially one we hope to control.

Of the algorithms used in this work, which would be touching on foundational aspects of human intelligence, in your view?

Thinking about the variant of MCTS used in this work, for example, it's not clear to me that tree search, no matter how clever, touches much on human cognition, at least significantly more than deep blue did.

On the other hand, the idea of bootstrapping a network with huge numbers of expert interactions before 'graduating' it to more complex training and architectural enhancement might turn out be an important part of the Game Over algorithm, as you call it. Even if it doesn't resembles much how humans beings learn.

Yes and no. MATLAB and Mathematica have access to the same BLAS libraries. If you're using packed arrays in Mathematica, you can expect similar performance for many pure numeric things. Mathematica is missing some of the higher level primitives, there's no built-in function that corresponds to DGEMM, as a random example.

Another weakness is that there's only a handful of packed array types, for a 64-bit system, you get tensors composed of doubles, signed 64-bit ints, and complex numbers; no bool arrays, no unsigned 32-bit or 16-bit, no enum or factor arrays. Now this is purely a performance thing -- you can make tensors/arrays containing anything you want, but their contents just won't be 'packed' into a contiguous region of memory, instead they'll be arrays of pointers to other memory on the heap. Such arrays are slower to work with, much less memory efficient, harder to pass over an FFI, etc.

There's also a lack of fast workhorse-ish functions involving tensors, so there isn't a way to broadcast a condition (say 4 <= x <= 10) onto an input matrix to get a 'mask matrix' as is so common in MATLAB and R and so on, and one of their secrets to writing fastish code in a interpreted language.

On the other hand, you can deal with ordinary lists (and Associations) that contain arbitrary types (as with any dynamically typed language), which makes many kinds of algorithmic work very straightforward. After all, not everything is best represented as an n-tensor of reals or ints or bools! But if your lists do happen to be homogeneous lists of ints or reals (of any rank), they will usually end up being stored more efficiently as mentioned above, a process that is, for better or worse, totally invisible unless you know about Developer/PackedArrayQ.

There is also Compile (https://reference.wolfram.com/language/ref/Compile.html?q=Co...) which still has a fairly limited type system, but will still let you bypass large amounts of interpreter overhead when you need to do something performance critical.

> you wouldn't analytically solve a sophisticated PDE so you would use numerics

The whole analytic/numerical distinction really isn't true anymore (e.g. https://reference.wolfram.com/language/ref/NDSolve.html, https://reference.wolfram.com/language/FEMDocumentation/tuto...).

> so that explains why MATLAB seems more prevalent.

I think the explanation lies more with MATLAB having exactly the right features (or the right toolboxes) at the right time to fill niches in thousands of engineering and research labs. If you're running a lab, and that postdoc who wrote the code driving your instrument has long since left, there is very little reason to embrace a new language, even if it has caught up in various ways, and especially if that new language isn't tailored exactly to what you are doing, and would prefer you to use some strange thing called "functional programming", etc.

It certainly is.

The advantages over C++ seemingly never end. I'd add to the list: rigorous portability, non-broken macros, ADTs and pattern matching, a careful language stability protocol, standard test and doc tooling, flexible namespacing via visibility and modules, the absense of vast lacunas of undefined behaviour, simple numeric promotion rules, meaningful compile-time errors for generic code. Plus things already mentioned in the post like Cargo.

People tend to focus on the safety side of Rust, but as impressive to me is just the... solidity on all these other axes. If you stripped out lifetimes and the borrow checker you would still have an incredibly compelling alternative to C++.

"Provocative"? No, I think there needs to be an element of bad faith or inauthenticity. 'Actual' trolls are supposed to be malicious creatures.

> homeschooling

I wouldn't call that scalable, it only worked for me because I was already coding by the time I convinced my parents to let me do my own thing. Though maybe there is an advanced body of theory I'm not aware of.

Also, I want to know, what are the best modern examples of enclaves? Are they all bespoke? Some likeminded parents getting together? Something more formulaic that has actually scaled out without becoming a self-parody or a lifestyle brand?

I also liked the sequence of posts explaining timely data flow and its implementation in Rust: https://github.com/frankmcsherry/blog/blob/master/posts/2015...

Haven't read it, but the author taught a class I attended and it was great fun.

I write a lot of Mathematica code for my job. To make life easier I've built in a macro system that makes day-to-day programming a whole lot more pleasant. It will be available in Mathematica 10.4 in the GeneralUtilities package that ships with Mathematica.

One of the built-in macros is a construct that automatically localizes any variables that occur as the L-value of an assignment:

  Needs["GeneralUtilities`"]
  foo[x_, y_] := Scope[
    z = x * y;
    PrimeQ[z + 1]
  ]

  foo[x_, y_] := Block[{z},
    z = x * y;
    PrimeQ[z + 1]
  ]

There are a couple other nice features of Scope, too, like the ability to use a single quote ' anywhere in the body of your function to Echo the value there to the current notebook without actually changing the behavior of the code -- essentially a very lightweight form of 'debugging print'. That also makes a huge difference.

And it has some more powerful variants, like "z '= 2 * 3" will echo a line that explicitly says "z = 6", rather than just "6".

Likewise '' means EchoHold, which prints the expression before being evaluated AND afterwards, so you'll see "z = 2 * 3 = 6" in the above case.

Lastly ''' means Tap, which is a higher-order form of EchoHold which instruments a function to echo its input and output (again without changing behavior). So Fold[Plus''', Range[10]] prints out all the intermediate steps in the fold.

There's a variety of features that add a very lightweight syntax for monadic and exception-based error handling, but that's not quite stable enough to advertise yet.

Same goes for debugging: there's some really powerful stack browsing, history capture and replay features, but again not quite stable enough to advertise for this version.

Even if Mathematica is never open sourced, there's still value in having a commercial language that can incubate new ideas or applications of existing ideas, for open source efforts to either ignore or riff on. The document+repl model used by IPython/Jupyter is an example of something that WRI didn't pioneer but certainly helped popularize in its modern form.

Another example that I think should be more widespread is the idea of 'superfunctions' whose arguments exploit a simple grammar to scale from simple invocations to arbitrarily complex ones. In effect you have a lightweight DSL -- a large or infinite family of functions fused into one -- which I think can buy a great deal of concision and flexibility.

GroupBy (https://reference.wolfram.com/language/ref/GroupBy.html) is one small example of this idea; in its simplest form it just partitions a list into equivalence classes, but scales up via its argument grammar to something more like (optionally nested) MapReduce. Part (https://reference.wolfram.com/language/ref/Part.html) would be another example. Again, this isn't something unique to Mathematica, you see variants of this idea in e.g. Clojure and Haskell, but my impression is that we've incorporated it into a wider cross-section of our language and made it more light-weight and convenient to use.

Another thing that I often think about is the notion of having a deep web of interrelated structures and behaviors, for example colors both have a visual display form (little colored boxes), IDE edit mechanics (you can click them to edit them in a palette), and polymorphic properties like distance, so that functions such as Nearest and FindClusters can 'automatically' operate on them. That's of course hardly novel, typeclasses in Haskell were designed to do precisely this, arguably more elegantly and user-extensibley.

But I think Mathematica shows how it is really valuable to have not just the capability to do this, but having it built in and working -- with many of those n^2 possible interactions between functions and data already curated and implemented. WRI is far from having done this completely, but again, we're trying, and I don't know of other groups that are trying to do this in an as integrated a way, across so many areas of computation.

Under the OP's view, atheists aren't scary because they might be violent -- most people aren't psychopaths, that's not what social signaling is for -- but because they've chosen to chuck out a large chunk of the signaling machinery that the "middle middle class and below" have habitually relied upon.

In other words, if you've sworn off the dominant canon of ethical and social judgements, your position now suddenly requires just-in-time evaluation by the other, and that is threatening to people who don't trust themselves to reliably pull it off.

Here's a toy Python parser I wrote many years ago actually listing out all valid parses of this sentence: https://github.com/taliesinb/chartreuse/blob/master/README.m...

If you want to see more videos, the supplementary section of the article has quite a few with descriptions: http://m.sciencemag.org/content/346/6208/1257998.figures-onl...

Here's a fascinating talk by the Nobel laureate behind this technique, titled "Imaging Life at High Spatiotemporal Resolutions": http://youtu.be/2R2ll9SRCeo

Indeed many of the interesting new papers are elaborations of old ideas along new themes, or useful and elegant new combinations of old ideas, with the occasional attention-grabbing paper when someone tries something that seems like it can't work and it does, and we now have an entirely new thing in our toolbox.

Here is a highly unusual paper that showed up on HN recently, to little discussion.

It tackles a related problem to the OP paper but in a way that could not be more different from the current default of gradients, linear algebra, large training data volumes, hands free training.

It may end up being less impressive than it seems, but it is still a bit mind bending.

http://journals.plos.org/plosone/article?id=10.1371/journal....