Hacker News Reader

Comment by anfilt | Hacker News Reader

Like I have pretty much always thought this when I read about ML or uses of it. The process and end result is just math so it has to be explainable in some way. I feel like not enough effort is given to understand the important variables to ML models or how are results derived. When I see uses of it it's more like the following: "Oh, it works, that's good enough."

For-instance when a model starts being over fitted, one can say it's pretty much converging to something akin to a LUT (Look up table). While over fitting is generally undesirable it still might be interesting. Principally, if someone could figure out how basically it is indexing the data. Perhaps you could create a simpler hash function or find some rules for creating useful hash functions for that type of data.

View on HN

The whole point is that your models should perform well on data that they haven't seen before. You can't create a hash function for that. Let alone one that works for 100s or 1000s of dimensions.

Unless there is a self-modifying feedback loop you can indeed replace any ML model with a hash function. The problem is one of size rather than any fundamental limitation of a look-up table.

Shouldn't this fall into the "trust by verify" category of things?

Whilst your model may not be able to explain to you today why it can operate on future data, if it can explain to you why it works on any data you've already given it, it makes it easier to explain behaviour.

You have to trust it on new data.

You can always verify why it made the choices it did on old data.

That seems like something incredibly helpful, to me.

> You can always verify why it made the choices it did on old data.

This makes no sense. Your model typically isn't asked to make predictions on "old data" because it was trained on it. And understanding why it made certain prediction isn't any different on "new" vs old data.

You can look into bias-variance tradeoff, train-test splitting and cross validation to get a better picture of this.

> You can look into bias-variance tradeoff, train-test splitting and cross validation to get a better picture of this.

Or, like in the article, you can make use of introspective models that can tell you what dimensions were fitted. This can more easily show you where overfitting is occurring.

You can see the weights of the model and follow the decision network, which allows you to explain why something has occurred in a much clearer fashion.

An introspective model gives you a much clearer picture of why certain choices were made, rather than the fuzzy picture you get running statistical analysis against a black-box'd algorithm.

Uh, yeah you can. Hash functions handle data its never seen before.

I'm confused by what you're saying... by data that hasn't seen before, do you mean new variables which need to be hypothesis tested? Or just a larger sample?

I also don't see why you can't create a "function" hash or otherwise, that works for 1000s of dimensions. The model itself, if it was perfect would be an equation... the number of dimensions really doesn't matter that much.

Sorry, I'm just real confused by your statement.

By data it hasn't seen before I mean a sample which doesn't exist in the training set. Maybe I'm not understanding something in the parent's argument.

For example, you train an image recognition model to tell cats and dogs apart using images from the internet, then you take out your phone, snap an image of your dog and give it to your algorithm to determine the species. When preprocessed this picture is equivalent to one row in a table with thousands of columns and this specific combination of pixel values doesn't exist anywhere else. Where is the hash function looking and for what in that case?

Say you're building a self-driving car. You need to take an image of what the car sees, and figure out whether/where there is a pedestrian/car/bicycle/etc in the image. So your input/variable is the image itself.

Almost every image the car sees is going to be different. I'd guess that's why GP said that it doesn't make sense to use a hashing function - there's little value in mapping inputs to results, because your input images are pretty much always going to be different. So pretty much every time you look up an image in your hashmap, you wouldn't find a match.

That's the point of ML models - find patterns in data so that when you see a new example, you can predict what you're seeing based on what you've seen in the past.

It seems like you might be deep in the ML rabbit hole. Zoom out a bit. A hash function is "just a function." Every image labeling black box can be though of as a hash from images to label vectors.

OP's comment effective asks whether there is another, more grokkable function that maps/hashes inputs to the same labels.

Granted, that question boils down to "can we create human-understandable models?" which is the whole point of this discussion.

It's a good question, though. If we had black-box-like spaghetti code performing the same task, I predict that the comments here would be very different.

Thanks for explaining that perspective. I was seeing things a little too narrow-mindedly for sure.

Based on your phrasing of the issue, it seems like we could think of the problem as: can we reduce the number of parameters in an ML model to the point where humans can understand all of the parameters? That's related to an active research area - minimizing model size. ML models can have billions of parameters. It's unfeasible for a human to evaluate all of them. Research shows that (sometimes) you can reduce the number of parameters in a model 97% without hurting it's accuracy [0]. But, 3% of a billion parameters is still way too many for people to evaluate. So I think the answer so far is no, we can't create human-understandable models that perform as effectively as black boxes.

[0] https://openreview.net/forum?id=rJl-b3RcF7

Indeed. It would be nice to read an article about NN introspection/debuggers. I don't see why you couldn't use an introspected ML model as the basis for a behavior tree in some cases.

It's an old idea (22 years at least), the black box method can be used to remove noise and find a none bayes/compression/simplest info theory model, then the model is used to generate clean training for a decision tree or association rules.

Reply on news.ycombinator.com