Hacker News Reader

Comment by stevenspasbo | Hacker News Reader

Is that 50 lines of code, or 50 lines of using a library that's thousands of lines of code?

It's a fair question, but from my heydays doing ML research even good libraries can be pretty revolutionary. The engineering-side of ML is still in relative infancy.

For a great example look at all the papers Andrew McCallum's group has been able to publish by building on top of FACTORIE: they get to focus their time on the problem at hand rather than all the math required to solve it. Basically they can write code that generates a model dynamically but the framework handles all the inference. Compare that to how these things are built without such a framework: you spend most of your time painstakingly hand-deriving update rules and then implementing them as code.

IMHO, the exciting thing is that ML is getting closer and closer to being an everyday tool for engineers rather than something that requires you to be a full-time math person to use effectively.

> ML is getting closer and closer to being an everyday tool for engineers rather than something that requires you to be a full-time math person

I so wish! As a Sr Data Scientist, I interview potential candidates quite often, many of these are 10x engineers.

    Me: (2,3,4) is a vector.  
    Eng: Ok.
    Me: Gimme a unit vector in the same direction.
    Eng 1: ???
    Eng 2: "It can be done. I don't know how, but with Spark it can be done".
    Eng 3: I will need R. ( given R, he fiddles with it for 5 minutes getting nowhere fast )

There are actual humans out there with self-professed ML expertise who cannot compute the eigens of a tiny 2 by 2 diagonal matrix. I kid you not. These people make 150k salaries, have "heard of an eigen vector", but cannot find one to save their lives.

"to save their lives" is a little dramatic, considering they were in a closed tech interview. I'm sure if you gave them 20 seconds on the internet (you know, the resource they would have in a real life problem solving situation) they could find one.

My graduate adviser use to say, if you ask "how do you add 1+1?" a student with a bachelor's degree answers "2". A person with a master's degree will write a 10 line C program to calculate it, then tell you "2". A person with a PhD just says "I don't know, but I can tell you where to find the answer."

Well, um ,a diagonal matrix is already diagonalizaed (lol) so its P marix is {{1,0},{0,1}} and hence the you can use those two as your diagonal entries are the eigenvalues.

Wouldn't you just take the length of the vector with sqrt(4+9+16) and divide each component by it? Or is this a different kind of 'vector' than graphics people are familiar with?

But Obviously. The same chap can invoke the SparseVector and DenseVector Java APIs from Spark, but doesn't know what a simple unit vector is or how to compute one. I think we are entering into a bleak era where there are umpteen built-in ML APIs for everything under the sun but the engineers who use them have no idea what these things really are - they just invoke the constructor and write some code and think they are now doing ML.

Seems hard to believe that an "engineer" with a degree in engineering or CS wouldn't know what is a unit vector.

Why? There are plenty of directions you can take through a CS degree that will never have you dealing with vectors at all. And subsequently there are plenty of CS heavy careers where you will never deal with it.

What little I remember about vectors is from my high school maths classes, and in my 20 years of doing software engineering it's come up exactly once (for a GIS related project).

You are probably right about CS. I only have experience with engineering and wouldn't expect any engineer (i.e. a major in engineering) to not know that.

I wouldn't expect anyone working with machine learning to not know these concepts either.

Hmmm... I don't think I remember how to compute the eigenvectors of a 2x2 matrix, other than constructing a system of linear equations representing the outcome of the matrix multiplication and solving it.

But come on, everyone knows how to use the definition of the standard (l2) vector norm to normalize a vector! Don't joke!

What do we say about languages built on C? Is it 100 lines of code but there are hundreds of thousands of lines of code for that higher level language you just coded?

I don't think libraries count in terms of code. We all use code to program. Standing on the shoulder that preceded us. Using a library and a function should just count for the most part.

True, but the parent commenter is getting at something important. The article suggests that researchers have found a new, much more concise way to express the solutions to difficult problems. That's different from a library, which merely packages pre-built solutions to a finite set of problems.

It's like the difference between a complete kitchen that fits in your pocket and an iPhone app that lets you order a burrito. The article suggests something like the former. A library which encapsulates 1000 lines of code into a single function call is like the latter.

Some languages are almost UNUSABLE without libraries. Looking at R with the dplyr, ggplot2, reshape 2 packages being mandatory for my work flow.

Python also has some mandatory libraries if you want to do any specific. Numpy and pandas for statistical analysis make them required.

My code is concise and clean but it is because of these libraries.

I am also certain that there are exceptions.

Even a "kitchen that fits in your pocket" isn't general enough. A library that parses a DSL, such that you can then code in that DSL, is still a lame duck if that library, plus the encodings of all the useful solutions in the domain, add up to more code than just the solutions would be when expressed in a general language. The ROI of (learning the DSL + maintaining the library) would be negative.

On the other hand, there are things like Prolog. You can think of Prolog as a backtracking constraint-solving library, and then another library that parses a DSL for expressing facts and procedural constraints and feeds it to the first library. But Prolog's language isn't really a DSL, because it isn't particular to any domain: there's no closed solution-space where Prolog applies. The efficiency gains you get from Prolog's elision of proceduralized contraint-solution code can apply to any program you write. And so its value is unbounded; its ROI is certainly positive, whatever the cost was to implement it.

That's the comparison that's useful here, I think. Is this something that only solves problems in one domain? Or is this something that could be applied to (at least some little bits of) any problem you encounter?

If dependencies don't count as code, then I can implement the entire Linux kernel in one line.

How about C itself? Hello World is about 850 lines of code after you run the preprocessor on it.

You do not need 850 lines of code to output "hello world" unless you actually include a header. You could just stub printf instead of including and get it done with like 10 lines of code.

Assembly: Microsoft Macro Assembly "Hello World"

        .MODEL  TINY
        .CODE

CODE SEGMENT BYTE PUBLIC 'CODE' ASSUME CS:CODE,DS:CODE ORG 0100H DB 'HELLO WORLD$', 0 INC DH MOV AH,9 INT 21H RET

And then the interrupt calls into DOS code, which then calls into BIOS code, which then has the hardware do the work. The hardware itself will do stuff like translating instructions into microcode..

I think stevenspasbo did not mean to criticize the progress in this area, but rather just point out the misleading title of the article. Unless you program everything in pure ASM, Bios and up, you always stand on the shoulders of others, that's how since works. But the title of the article, "Probabilistic programming does in 50 lines of code what used to take thousands," makes it sound as if there was a solution to these complex problems that could be expressed in 50 lines of code all along, and only now did we figure it out. The truth is, people have written thousands of lines of code, and put them into libraries, and now those tools can be used and leveraged to express complex solutions more easily.

Thank you, that's exactly what I meant.

I agree -- I think lines of code is the wrong measure. A better measure would be how large is the set of problems that occur in practice that normally would take thousands of lines which can now solve be solved with 50 lines of code using a probabilistic programming language.

Generally it's anything that you can express using a probabilistic graphical model. It's mainly useful in engineering (real engineering), control systems, robotics, inference, machine learning and artifical intelligence circles.

Once a library becomes first class syntactical features of a language runtime then it shouldn't count towards line count anymore.

I was thinking the same thing. When is something just a library, and when is it something more? Think about concurrency in Go. Goroutines are cool, but what makes them more than just a cooperative threading library? Channels? Those are just concurrent data structures. The special sauce in Go is the select structure. It allows you to easily express control flow that cannot be expressed without select (or at least not easily). What is the special sauce in these probabilistic programming languages? What could not be implemented using a library?

You could make a similar comparison, for example, between 50 lines of C and 1000 bytes of microcode. (I just made those numbers up, but you get the point)

It makes sense to talk about lines of code when thinking about engineering hour efficiency. It costs the same amount for a programmer to write one line of C as one line of Assembly, and the rate of bugs tracks lines of code as well. The underlying libraries and abstractions should be considered if you're thinking about computing efficiency.

Using lines of code as a metric makes no sense at all. Bad programmers produce more lines of code. Does that make them more efficient?

A good programmer that writes less lines of code is more efficient than a bad programmer that writes more to do the same work, because you pay each per line of code. A library that allows both to write less lines of code will increase efficiency for everyone.

The time it takes to learn and master the language aren't irrelevant either.

That's what I was thinking. A good framework can do something comparable if your application fits its standard model.

The point is that the library is generic - it can be used for a wide variety of difficult problems.

Reply on news.ycombinator.com