Show HN: Lambdo – Feature engineering and machine learning together | Hacker News Reader

Show HN: Lambdo – Feature engineering and machine learning together

github.com

73 pointsasavinov24 comments

> Feature engineering is a mechanism of creating new levels of abstraction in knowledge representation because each (non-trivial) feature extract and makes explicit some piece of knowledge hidden in the data. It is almost precisely what deep learning is intended for. In this sense, feature engineering does what hidden layers of a neural network do or what the convolutional layer of a neural network does

Very intriguing and thoughtful statement. I hadn't ever thought of it that way.

This is essentially repeating your quote, but an "aha moment" clicked for me when I read that what successful neural networks are basically doing is such good feature learning that the problem can be solved by a simple linear model in the end.

E.g. if you have an N-layer neural network, N-1 layers are doing feature learning, and the Nth layer is a simple {logistic, multinomial/softmax, gaussian, poisson, ...} model

Related:"the kernel trick" [1].

"The kernel trick avoids the explicit mapping that is needed to get linear learning algorithms to learn a nonlinear function or decision boundary."

(what powers Support Vector Machines, the neural networks of the 90s, and still alive and kicking today)

[1] https://en.wikipedia.org/wiki/Kernel_method#Mathematics:_the...

A quick - and not 100% correct but it will do - way of looking at deep learning is to think of it as automated feature detection and weighing.

Have no worry. Little, if anything, is 100% correct.

We sometimes get hung up on correcting and contradicting people, often missing a deeper truth. It takes skill to find the grain of truth and build on it :-)

Lambdo is a workflow engine which simplifies data analysis by combining in one analysis pipeline

* Feature engineering and machine learning: Lambdo does not distinguish them and treats them as data transformations

* Model training and prediction: both feature definitions and ML models can be trained as part of one workflow

* Table population and column evaluation: workflow consists of nodes of these two types. This makes it similar to Bistro: https://github.com/asavinov/bistro

Lambdo is intended for the following use cases:

* Numerous derived features with parameters derived from the data

* Regular re-training is required by using the same features as those to be used during prediction

* Time series analysis because it is where the quality of derived features is especially important

* Customization via user-defined Python functions

I definitely see the need for packages like this. So much of a data scientist's time is spent on feature engineering, but there are relatively few tools out there that are trying to improve that step in the process compared to tools for the modeling step.

I see this tool as a something that can help with the deployment piece of the feature engineering. As things stand, it's "easy" to package and deploy modeling code, but much harder to package up your feature engineering workflow. In part because there is no agreed upon standard for developing feature engineering pipelines.

I'd be curious how this could be combined with a library like Featuretools (http://github.com/featuretools/featuretools/) which helps automate the discovery of features, but currently has less functionality related to deployment.

(full disclosure: I work on Featuretools)

I think there is a need for deployment approaches for both: feature engineering and modeling. For example, please consider feature scaling and case when the mean of the feature is drifting. Then the feature engineering and probably ML model needs to be updated. I'm not aware of ready solutions for such problems.

Things I always want when looking at something new:

- Where does it sit in relation to other data science components?

- What does it integrate with, and what is it agnostic to?

- Smallest self-contained use case? I see some examples in the repo readme but they are not self-contained so its harder for me to imagine its use

I'm also reading the readme and trying to guess why I need this package?

One more question, why treating feature engineering and ML models the same way is important?

Because they are so much similar that it is much easier to implement them as part of one mechanism. For example:

* Apply a (say, SVM) classification model to each object (row) by producing a new column

* Generate a new column as a difference between its values and its average over all rows.

In both cases, you produce a new column (=feature) by applying some transformation. Also, in both cases, you need to find parameters of this transformation from the data. In the first case, by training SVM model. In the second case, by find the average value.

Conclusion: there is no essential difference between defining/training a feature and a ML model.

I think it depends on your context. From theoretical point of view you can treat both (feature engineering and ML) as transformation defined by some function with parameters. From practical point of view, they can differ a lot. Imagine, you would like to handle in a different way: feature transformation that can be done on 1M rows and 2 columns in 5 seconds, and training of ML model that can take 7 days to train.

Your comment reminds me so much of my PM who asks questions like these to give an appearance that he is putting in some serious effort and energy to understand. Add a few words like 'value', 'leverage' and 'resonate' and the impersonation would be pitch perfect .... Looks up contact detail oops spoke to soon should have guessed.

Haha :) My comment is precisely aimed at reducing the energy needed to understand the value prop here. Why is that bad?

My assumption is that would be helpful to other readers and thus the author. Do you prefer having key information buried?

> 'value', 'leverage' and 'resonate'

You could not be more wrong about my perspective though.

My question is from this perspective: I use tensorflow, keras, numpy, scikit-learn, pandas, and Im looking to understand how I would integrate these.

Oh I know, from your past comments {you are more famous than you think :}

Not sure what you mean by this vague statement.

Nothing sinister. I take interest in ML related threads so I remember your HN handle.

By the same token, your comment reminds me of an engineer who puts serious effort into making his work look more important than it is. Even the term "feature engineering" is pretty ridiculous. As if academics found real-world data to be too messy and decided to outsource this little cost to.....a department of expert feature-makers(?)

Ask around in ML quarters how important is feature engineering that will sort things out for you.

Used to work in machine vision so I do kind of understand the basics of model development and testing. Hope your PM situation improves :)

Machine vision is actually a good place to be in this context. There is a lot of institutional knowledge on what good features are, and if you have enough data DNNs can do a lot of heavy lifting. If you arent doing DNNs, the decision on how you featurize the problem will probably play the most significant role.

Its some of the other areas of ML where its not known ahead of time what the good features would be that tooling for feature engineering would help a great deal. Especially when one doesn't have the luxury of throwing a DNN at it and be done with it. The situation has both good and bad parts, the good part is that there is a lot of space to be creative in the design of features.

Cool, thanks. I suppose financial/risk modeling is also primarily "feature engineering"; I've just never seen it as separate from the modeling itself. Perhaps it's because I've typically been as involved in the domain expertise as in the math part.

Feature engineering is about going from a concrete real world problem to a representation that a computer can operate on. Furthermore, this representation has to facilitate solving the problem.

DNNs are good at trying many alternative representations on their own, but you need a lot of data. When you cannot use these beasts, one has to be creative and think of good ways to represent the problem (either as numbers or syntactic structures such as graphs).

Once the representation part is done then one can throw computation and maths at the problem.

Reply on news.ycombinator.com