Hacker News Reader

I believe that random forests are still the primary way data scientists work with tabular data. Deep learning hasn't cracked tabular data like it has other areas.

View on HN

Everytime I look into this stuff it is just one table/dataframe. Nobody is modelling the relations between things. I think there is a huge opportunity for a product that can look at Customer, Items, Orders, Returns, Payment_Methods etc. and first of all show me when things tend to co-occur, so that it could generate synthetic customers that have the statistically correct number of registered payment methods where those payment details are also synthetically generated.

Next would be the ability to decompose or factor my entire db into the subcomponents and make suggestions for combining tables. The use case would be a legacy enterprise system that has grown so complex and tangled that devs are afraid to do basic db refactoring. From where I'm sitting and working this is the next gold rush, apply DL methods to the bread and butter computing, log flow analysis, etc.

hi john, your first paragraph is my phd (2021).

its a massively under researched area because relational data has more... dimensions to it and is understandably not as exciting to most.

happy to discuss (email in profile)

Reply on news.ycombinator.com